Not all words are created equal Here is our premise. We have a coherent sequence of words thought up by a human, for instance a book, an essay, a blog post such as this one, or a transcription of a speech.

Motivation Let’s say you have some data. You assume that this data comes from a specific probability distribution. You have an idea of what that distribution might be (e.g. Gaussian, Poisson, …), but you don’t know anything about its parameters.