Protecting the private data from publishing has become a requisite in todays world of data. This solves the problem of divulging the sensitive data when it is mined. Amongst the prevailing privacy models, ϵ differential privacy outfits one of the strongest privacy guarantees. In this paper, we address the problem of private data publishing on vertically partitioned data, where different attributes exist for same set of individuals. This operation is simulated between two parties. In specific, we present an algorithm with differentially private data release for vertically partitioned data between two parties in the semi honest adversary model. First step towards achieving this is to present a two party protocol for exponential mechanism. By the same token, a two party algorithm that releases differentially private data in a secure way following secure multiparty computation is implemented. A set of investigational results on the real life data indicate that the proposed algorithm can effectively safeguard the information for a data mining task.