March 1, 2013
One of the most dominant technologies in language modeling is N-gram models. In this work, we make an approach to continuous- space language models using continuous distributions. We give a suitable mapping method from the discrete word space to a continuous space and show how to model the resulting continuous vectors. Techniques and methods for mapping and dimensionality reduction such as Singular Value Decomposition and Linear Discriminant Analysis are being used.
Multivariate Gaussian distribution, Gaussian Mixture Models and tying parameters techniques are the main tools for this language modeling approach.