To work our way up to GLMs, we will begin by defining exponential family distributions. We say that a class of distributions is in the exponential family if it can be written in the form

\(p(y ; \eta)=b(y) \exp \left(\eta^{T} T(y)-a(\eta)\right)\)

Here, \(\eta\) is called the [...] (also called the [...] ) of the distribution; T(y) is the sufficient statistic (for the distributions we consider, it will often be the case that T(y) = y); and \(a(\eta)\) is the log partition function. The quantity \(e^{-a(\eta)}\) essentially plays the role of a normalization constant, that makes sure the distribution \(p(y;\eta)\) sums/integrates over y to 1.

To work our way up to GLMs, we will begin by defining exponential family distributions. We say that a class of distributions is in the exponential family if it can be written in the form

\(p(y ; \eta)=b(y) \exp \left(\eta^{T} T(y)-a(\eta)\right)\)

Here, \(\eta\) is called the [...] (also called the [...] ) of the distribution; T(y) is the sufficient statistic (for the distributions we consider, it will often be the case that T(y) = y); and \(a(\eta)\) is the log partition function. The quantity \(e^{-a(\eta)}\) essentially plays the role of a normalization constant, that makes sure the distribution \(p(y;\eta)\) sums/integrates over y to 1.

natural parameter

canonical parameter