Mixture models for count data: the ZIP model

Unfortunately, estimating the parameters of a mixture model presents a number of obstacles: first, model identification is not guaranteed; second, estimates are sensitive to the started values used for the optimization algorithm. In this work I shortly reviews some of this problems.

A great care is needed in the specification and estimation of a mixture model: it is not guaranteed that the parameter space lies in an identifiable set. In fact a mixture model is unidentifiable when infinitely many parameters vectors lead to the same mixed distribution; to make an example, we cannot estimate a mixture of two or more Bernoulli distributions, without any additional assumptions. An accurate account of this issue is in Titterington et al., 1985.

Even when identifiability conditions hold, the identification may be weak. We pay the flexibility of this class of models with an unpredictable and multimodal likelihood surface (posterior distribution in a Bayesian analysis). As a consequence the models admitting a degenerate distribution in an arbitrary mass point (e.g., the Normal model) and the log-likelihood function is unbounded. For the same reasons, the estimates are extremely sensitive to model misspecification.

Another very important issue is that most algorithms (e.g., the standard EM) assume a known number of components: more complex procedures allow for k to be an estimate parameter, alternatively , we can choose different values of k and perform an ex-post model selection using the standard selection criteria (BIC, AIC etc.)

Finally, once the model structure has been chosen and a maximum of the log-likelihood function has been found, we have to face the problem of making inference about the model parameters. Also in very simple settings, no closed forms are available for the sampling variances of the estimators, and the Hessian matrix is generally hard to derive. Even with we have a quite large sample size, the distribution of the estimators may be heavily skewed.