The modified likelihood ratio test, which is based on penalized likelihood function, is usually used for testing homogeneity of the mixture models. The efficiency of this test is seriously affected by the shape of penalty function that is used in penalized likelihood function. The selection of penalty function is usually based on avoiding of complexity and increasing tractability, hence the results may be far from optimality. In this paper, we consider a more general form of penalty function that depends on a shape parameter. Then this shape parameter and the parameters of mixture models are estimated by using Bayesian paradigm. It is shown that the proposed Bayesian approach is more efficient in comparison to modified likelihood test. The proposed Bayesian approach is clearly more efficient, specially in nonidentifiability situation, where frequentist approaches are almost failed.