on 21-Feb-2017 (Tue)

The factors of Bayes’ rule have specific names that will be used regularly throughout the book, as indicated here: p(θ|D) posterior = p(D|θ) likelihood p(θ) prior / p(D) evidence

The “prior,” p(θ ), is the credibility of the θ values without the data D. The “posterior,” p(θ|D),isthe credibility of θ values with the data D taken into account.

The “likelihood,” p(D|θ),is the probability that the data could be generated by the model with parameter value θ

The “evidence” for the model, p(D), is the overall probability of the data according to the model, determined by averaging across all possible parameter values weighted by the strength of belief in those parameter value

the model, determined by averaging across all possible parameter values weighted by the strength of belief in those parameter values

The denominator of Bayes’ rule, labeled in Equation 5.7 as the evidence for the model, is also called the marginal likelihood.

The term “marginal likelihood” refers specifically to the operation of taking the average of the likelihood, p(D|θ), across all values of θ ,weighted by the prior probability of θ

For continuous variables, the only change in Bayes’ rule is that the marginal likelihood changes from the sum in Equation 5.8 to an integral: p(D) = dθ ∗ p(D|θ ∗ )p(θ ∗ )

suppose we observe some more data, which we’ll denote D . We can then update our beliefs again, from p(θ|D ) to p(θ|D , D). Here’s the question: Does our final belief depend on whether we update with D first and D second, or update with D first and D second? The answer is: It depends! In particular, it depends on the model function that defines the likelihood, p(D|θ).

In many models, the probability of data, p(D|θ), does not depend in any way on other data. That is, the joint probability p(D, D |θ) equals p(D|θ)·p(D |θ). In other words, in this sort of model, the data probabilities are independent (recall that independence was defined in Section 4.4.2). Under this condition, then the order of updating has no effect of the final posterior

One way of thinking about this assumption is as follows: We assume that every datum is equally representative of the underlying process, regardless of when the datum was observed, and regardless of any data observed before or after.

a general phenomenon in Bayesian inference: The posterior is a compromise between the prior distribution and the likelihood function.

The compromise favors the prior to the extent that the prior distribution is sharply peaked and the data are few. The compromise favors the likelihood function (i.e., the data) to the extent that the prior distribution is flat and the data are many.

the mode of the posterior distribution is between the mode of the prior distribution and the mode of the likelihood function, but the posterior mode is closer to the likelihood mode for larger sample sizes

the width of the posterior highest density intervals (HDI) is smaller for the larger sample size. Even though both samples of data showed 25% heads, the larger sample implied a smaller range of credible underlying biases

In general, the more data we have, the more precise is the estimate of the parameter(s) in the model. Larger sample sizes yield greater precision or certainty of estimation

You can use logical vectors to extract a selection of rows or columns from a ma- trix, for example, if a is the original 3-by-3 matrix defined above, the statement a(:, logical([1 0 1])) results in ans = 13 46 79 (first and third columns extracted)

the functions zeros, ones and rand generate matrices of 1s, 0s and random numbers, respectively. With a single argument n, they generate n × n (square) matrices. With two arguments n and m they generate n × m ma- trices.

For very large matrices repmat is usually faster than ones and zeros.

The function eye(n) generates an n × n identity matrix, i.e., a matrix with 1s on the main ‘diagonal’, and 0s everywhere else

pascal(n) generates a Pascal matrix of order n. Technically, this is a symmetric positive definite matrix with entries made up from Pascal’s triangle

magic(n) generates an n × n magic square

the statement all(a) results in ans = 001 For each column of a where all the elements are true (non-zero) all returns 1, otherwise it returns 0. all therefore returns a logical vector when it takes a matrix argument. To test if all the elements of a are true, use all twice

diag extracts or creates a diagonal

fliplr flips from left to right

flipud flips from top to bottom

rot90 rotates.

tril extracts the lower triangular part

triu extracts the upper triangular part

It helps to think of the 3-D array a as a series of ‘pages’, with a matrix on each page. The third dimension of a numbers the pages. This is analogous to a spreadsheet with multiple sheets: each sheet contains a table (matrix)

The matrix operation A 2 means A × A,whereA must be a square matrix. The operator ^ is used for matrix exponentiation

det determinant. eig eigenvalue decomposition. expm matrix exponential, i.e., e A ,whereA is a matrix. The matrix exponen- tial may be used to evaluate analytical solutions of linear ordinary differential equations with constant coefficients. inv inverse. lu LU factorization (into lower and upper triangular matrices). qr orthogonal factorization. svd singular value decomposition

although the overall conformation of each pro- tein is unique, two regular folding patterns are often found within them. Both pat- terns were discovered more than 60 years ago from studies of hair and silk

The first folding pattern to be discovered, called the α helix, was found in the protein α-keratin, which is abundant in skin and its derivatives—such as hair, nails, and horns.

Within a year of the discovery of the α helix, a second folded structure, called a β sheet, was found in the protein fibroin, the major constituent of silk. These two patterns are particularly common because they result from hydro- gen-bonding between the N–H and C=O groups in the polypeptide backbone, without involving the side chains of the amino acids