Document Type

Conference Proceeding

Publication Date



We describe a unified probabilistic framework for statistical language modeling, the latent maximum entropy principle. The salient feature of this approach is that the hidden causal hierarchical dependency structure can be encoded into the statistical model in a principled way by mixtures of exponential families with a rich expressive power. We first show the problem formulation, solution, and certain convergence properties. We then describe how to use this machine learning technique to model various aspects of natural language, such as syntactic structure of sentences, semantic information in a document. Finally, we draw a conclusion and point out future research directions.


This paper was presented at the IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) in 2001.