on 02-Mar-2017 (Thu)

Recursive functions are usually written in this way: an if statement handles the general recursive definition; the else part handles the special case (n = 1)

where the number of repetitions must be determined in advance, is sometimes called determinate repetition.

it often happens that the condition to end a loop is only satisfied during the execution of the loop itself . Such a structure is called indeterminate.

If there are a number of different conditions to stop a while loop you may be tempted to use a for with the number of repetitions set to some accepted cut-off value (or even Inf) but enclosing if statements which break outofthe for when the various conditions are met. Why is this not regarded as the best programming style? The reason is simply that when you read the code months later you will have to wade through the whole loop to find all the conditions to end it, rather than see them all paraded at the start of the loop in the while clause

Graphs (in 2-D) are drawn with the plot statement. In its simplest form, it takes a single vector argument as in plot(y)

plot(y). In this case the elements of y are plotted against their indexes, e.g., plot(rand(1, 20)) plots 20 random num- bers against the integers 1–20, joining successive points with straight lines,

Probably the most common form of plot is plot(x, y) where x and y are vectors of the same length, e.g., x = 0:pi/40:4*pi; plot(x, sin(x)) In this case, the co-ordinates of the ith point are x i , y i

Straight-line graphs are drawn by giving the x and y co-ordinates of the end- points in two vectors.

MATLAB has a set of ‘easy-to-use’ plotting commands, all starting with the string ‘ez’. The easy-to-use form of plot is ezplot, e.g., ezplot(’tan(x)’)

9.1 Basic 2-D graphs 199 gtext(’text’) writes a string (’text’) in the graph window. gtext puts a cross-hair in the graph window and waits for a mouse button or keyboard key to be pressed.

Text may also be placed on a graph interactively with Tools -> Edit Plot from the figure window

grid adds/removes grid lines to/from the current graph. The grid state may be toggled

text(x, y, ’text’) writes text in the graphics window at the point speci- fied by x and y. If x and y are vectors, the text is written at each point. If the text is an indexed list, successive points are labeled with corresponding rows of the text

title(’text’) writes the text as a title on top of the graph

xlabel(’horizontal’) labels the x-axis

ylabel(’vertical’) labels the y-axis

There are at least three ways of drawing multiple plots on the same set of axes (which may however be rescaled if the new data falls outside the range of the previous data). 1. The easiest way is simply to use hold to keep the current plot on the axes. All subsequent plots are added to the axes until hold is released, either with hold off,orjusthold, which toggles the hold state. 2. The second way is to use plot with multiple arguments

The third way is to use the form plot(x, y) where x and y may both be matrices, or where one may be a vector and one a matrix. If one of x or y is a matrix and the other is a vector, the rows or columns of the matrix are plotted against the vector, using a different color for each.

If x is not specified, as in plot(y),wherey is a matrix, the columns of y are plotted against the row index

If x and y are both matrices of the same size, the columns of x are plotted against the columns of y

plot(x, y, ’--’) joins the plotted points with dashed lines, whereas plot(x, y, ’o’) draws circles at the data points with no lines joining them

The available colors are denoted by the symbols c, m, y, k, r, g, b, w

axis( [xmin, xmax, ymin, ymax] ) which sets the scaling on the current plot, i.e., draw the graph first, then reset the axis limits.

If you want to specify one of the minimum or maximum of a set of axis limits, but want MATLAB to autoscale the other, use Inf or -Inf for the autoscaled limit

You can return to the default of automatic axis scaling with axis auto

The statement v = axis returns the current axis scaling in the vector v.

Scaling is frozen at the current limits with axis manual so that if hold is turned on, subsequent plots will use the same limit

in MATLAB the word ‘axes’ refers to a particular graphics object, which includes not only the x-axis and y-axis and their tick marks and labels, but also everything drawn on those particular axes: the actual graphs and any text included in the figure

You can show a number of plots in the same figure window with the subplot function. It looks a little curious at first, but it’s quite easy to get the hang of it. The statement subplot(m, n, p) divides the figure window into m × n small sets of axes, and selects the pth set for the current plot (numbered by row from the left of the top row)

figure(h),whereh is an integer, creates a new figure window, or makes figure h the current figure

clf clears the current figure window. It also resets all properties associated with the axes, such as the hold state and the axis state

cla deletes all plots and text from the current axes, i.e., leaves only the x- and y-axes and their associated information

The command [x, y] = ginput allows you to select an unlimited number of points from the current graph us- ing a mouse or arrow keys. A movable cross-hair appears on the graph. Clicking saves its co-ordinates in x(i) and y(i). Pressing Enter terminates the input.

The command [x, y] = ginput(n) works like ginput except that you must select exactly n points

he command semilogy(x, y) plots y with a log 10 scale and x with a linear scale

The command polar(theta, r) generates a polar plot of the points with angles in theta and magnitudes in r

Plotting rapidly changing mathematical functions: fplot

The function plot3 is the 3-D version of plot. The command plot3(x, y, z) draws a 2-D projection of a line in 3-D through the points whose co-ordinates are the elements of the vectors x, y and z

The function comet3 is similar to plot3 except that it draws with a moving ‘comet head’.

The function mesh draws a surface as a ‘wire frame’.

An alternative visualization is provided by surf, which generates a faceted view of the surface (in color), i.e. the wire frame is covered with small tiles

contour(u) You should get a contour plot of the heat distribution

The function contour can take a second input variable. It can be a scalar spec- ifying how many contour levels to plot, or it can be a vector specifying the values at which to plot the contour levels

You can get a 3-D contour plot with contour3

Contour levels may be labeled with clabel

A 3-D contour plot may be drawn under a surface with meshc or surfc

If a matrix for a surface plot contains NaNs, these elements are not plotted. This enables you to cut away (crop) parts of a surface

The function quiver draws little arrows to indicate a gradient or other vec- tor field

The mesh function can also be used to ‘visualize’ a matrix

The function spy is useful for visualizing sparse matrices

The view function enables you to specify the angle from which you view a 3- D graph

The function view takes two arguments. The first one, az in this example, is called the azimuth or polar angle in the x-y plane (in degrees). az rotates the viewpoint (you) about the z-axis—i.e. about the ‘pinnacle’ at (15,15) in Fig- ure 9.12—in a counter-clockwise direction. The default value of az is −37.5 ◦ . The program therefore rotates you in a counter-clockwise direction about the z-axis in 15 ◦ steps starting at the default position. The second argument of view is the vertical elevation el (in degrees). This is the angle a line from the viewpoint makes with the x-y plane. A value of 90 ◦ for el means you are directly overhead. Positive values of the elevation mean you are above the x-y plane; negative values mean you are below it. The default value of el is 30 ◦

The command pause(n) suspends execution for n seconds.

You can rotate a 3-D figure interactively as follows. Click the Rotate 3-D button in the figure toolbar (first button from the right). Click on the axes and an outline of the figure appears to help you visualize the rotation. Drag the mouse

the cell can form a hydrogel that pulls these and other molecules into punctate structures called intracellular bodies, or granules. Specific mRNAs can be seques- tered in such granules, where they are stored until made available by a controlled disassembly of the core amyloid structure that holds them together.

the FUS protein, an essential nuclear protein with roles in the tran- scription, processing, and transport of specific mRNA molecules.

Over 80 per- cent of its C-terminal domain of two hundred amino acids is composed of only four amino acids: glycine, serine, glutamine, and tyrosine. This low complexity domain is attached to several other domains that bind to RNA molecules.

At high enough concentrations in a test tube, FUS protein forms a hydrogel that will associate with either itself or with the low complexity domains from other proteins.

FUS low complexity domain binds most tightly to itself

both the homotypic and the heterotypic bindings are mediated through a β-sheet core structure forming amyloid fibrils, and that these structures bind to other types of repeat sequences

Many of these interactions appear to be controlled by the phosphorylation of ser- ine side chains in the one or both of the interacting partners.

he amyloid fibril is a long unbranched structure assembled through a repeating aggre- gate of β sheets

The substance that is bound by the protein—whether it is an ion, a small molecule, or a macromolecule such as another protein—is referred to as a ligand for that protein (from the Latin word ligare, meaning “to bind”)

Even small changes to the amino acids in the interior of a protein molecule can change its three-dimensional shape enough to destroy a binding site on the surface

the interaction of neighboring parts of the polypeptide chain may restrict the access of water molecules to that protein’s ligand-binding sites. Because water molecules readily form hydrogen bonds that can compete with ligands for sites

a ligand will form tighter hydrogen bonds (and electro- static interactions) with a protein if water molecules are kept away

In effect, a protein can keep a ligand-binding site dry, increasing that site's reactivity, because it is energetically unfavorable for individual water mole- cules to break away from this network—as they must do to reach into a crevice on a protein’s surface

the clustering of neighboring polar amino acid side chains can alter their reactivity. If protein folding forces together a number of negatively charged side chains against their mutual repulsion, for example, the affinity of the site for a positively charged ion is greatly increased.

when amino acid side chains interact with one another through hydrogen bonds, normally unreactive groups (such as the –CH 2 OH on the serine shown in Figure 3–39) can become reactive, enabling them to be used to make or break selected covalent bonds

even when the amino acid sequence identity falls to 25%, the backbone atoms in a domain can follow a common protein fold within 0.2 nanometers (2 Å)

We can use a method called evolutionary tracing to identify those sites in a protein domain that are the most crucial to the domain’s function. Those sites that bind to other molecules are the most likely to be maintained, unchanged as organisms evolve. Thus, in this method, those amino acids that are unchanged, or nearly unchanged, in all of the known protein family members are mapped onto a model of the three-dimensional structure of one family member. When this is done, the most invariant positions often form one or more clusters on the protein surface, as illustrated in Figure 3–40A for the SH2 domain described previously (see Figure 3–6). These clusters generally correspond to ligand-binding sites.

The amino acids located at the binding site for the phosphorylated polypeptide have been the slowest to change during the long evolutionary process

n many cases, a portion of the surface of one protein contacts an extended loop of polypeptide chain (a “string”) on a second protein

A second type of protein–protein interface forms when two α helices, one from each protein, pair together to form a coiled-coil

The most common way for proteins to interact, however, is by the precise matching of one rigid surface with that of another

Different antibod- ies generate an enormous diversity of antigen-binding sites by changing only the length and amino acid sequence of these loops, without altering the basic protein structure.

A detailed examination of the antigen-binding sites of antibodies reveals that they are formed from several loops of polypeptide chain that protrude from the ends of a pair of closely juxtaposed protein domains

Antibodies are Y-shaped molecules with two identical binding sites that are complementary to a small portion of the surface of the antigen molecule

Loops of this kind are ideal for grasping other molecules. They allow a large number of chemical groups to surround a ligand so that the protein can link to it with many weak bonds. For this reason, loops often form the ligand-binding sites in proteins

Strong interactions occur in cells when- ever a biological function requires that molecules remain associated for a long time—for example, when a group of RNA and protein molecules come together to make a subcellular structure such as a ribosome

Self-information deals only with a single outcome. W e can quantify the amoun t of uncertain t y in an en tire probabilit y distribution using the Shannon entrop y : H ( ) = x E x ∼ P [ ( )] = I x − E x ∼ P [log ( )] P x . (3.49) also denoted H ( P )

the Shannon en trop y of a distribution is the exp ected amoun t of information in an ev en t dra wn from that distribution

It gives a lo w er bound on the num b er of bits (if the logarithm is base 2, otherwise the units are differen t) needed on av erage to enco de symbols drawn from a distribution P

Distributions that are nearly deterministic (where the outcome is nearly certain) ha v e lo w en trop y; distributions that are closer to uniform hav e high entrop y

When 3.5 x is con tin uous, the Shannon entrop y is kno wn as the differen tial entrop y

If we hav e t w o separate probability distributions P ( x ) and Q ( x ) ov er the same random v ariable x , we can measure ho w different these t w o distributions are using the Kullbac k-Leibler (KL) div ergence : D KL ( ) = P Q E x ∼ P log P x ( ) Q x ( ) = E x ∼ P [log ( ) log ( )] P x − Q x

Kullbac k-Leibler (KL) div ergence : \(D_{KL}( P||Q) = E_{x\sim P} = [log\frac{ P (x )}{ Q (x )}] = E_{x \sim P} [log P(x) - log Q(x)] \) (3.50) In the case of discrete v ariables, it is the extra amount of information (measured in bits if we use the base 2 logarithm, (but in machine learning w e usually use nats and the natural logarithm) needed to send a message containing symbols drawn from probability distribution P , when w e use a co de that w as designed to minimize the length of messages dra wn from probabilit y distribution Q.

The KL div ergence has many useful prop erties, most notably that it is non- negativ e.

The KL divergence is 0 if and only if P and Q are the same distribution in the case of discrete v ariables, or equal “almost ev erywhere” in the case of con tin uous v ariables

KL div ergence is non-negativ e and measures the difference b et w een t w o distributions, it is often conceptualized as measuring some sort of distance b etw een these distributions. How ever, it is not a true distance measure b ecause it is not symmetric

A quan tit y that is closely related to the KL div ergence is the cross-en trop y \(H ( P , Q ) = H ( P ) + D_{KL}( P|| Q )\) , whic h is similar to the KL div ergence but lac king the term on the left: \(H (P, Q ) = − E _{x \sim P} \space log Q(x)\)

Minimizing the cross-entrop y with resp ect to Q is equiv alent to minimizing the KL div ergence, b ecause do es not participate in the omitted term. Q

the form 0 log 0 . By con v en tion, in the con text of information theory , w e treat these expressions as lim x → 0 x x log = 0

we can greatly reduce the cost of representing a distribution if w e are able to find a factorization in to distributions o v er few er v ariables

When we represen t the factorization of a probabilit y distribution with a graph, we call it a structured probabilistic mo del or graphical mo del

There are t w o main kinds of structured probabilistic mo dels: directed and undirected.

graphical models use a graph G in which each no de in the graph corresp onds to a random v ariable, and an edge connecting tw o random v ariables means that the probability distribution is able to represen t direct in teractions b et w een those t w o random v ariables

Directed mo dels use graphs with directed edges, and they represen t fac- torizations into conditional probability distributions

a directed mo del contains one factor for ev ery random v ariable x i in the distribution, and that factor consists of the conditional distribution ov er x i giv en the paren ts of x i , denoted P a G ( x i ) : p ( ) = x i p ( x i | P a G ( x i ))

Undirected mo dels use graphs with undirected edges, and they represen t factorizations into a set of functions

Any set of no des that are all connected to each other in G is called a clique.

Each clique C ( ) i in an undirected mo del is asso ciated with a factor φ ( ) i ( C ( ) i )

Each clique C ( ) i in an undirected mo del is asso ciated with a factor φ ( ) i ( C ( ) i ) . These factors are just functions, not probabilit y distributions

The probability of a configuration of random v ariables is prop ortional to the pro duct of all of these factors—assignments that result in larger factor v alues are more lik ely

W e therefore divide by a normalizing constant Z , defined to b e the sum or integral o v er all states of the pro duct of the φ functions, in order to obtain a normalized probabilit y distribution: p ( ) = x 1 Z i φ ( ) i C ( )

numerical compu- tation. This t ypically refers to algorithms that solve mathematical problems b y metho ds that update estimates of the solution via an iterative pro cess, rather than analytically deriving a form ula providing a symbolic expression for the correct so- lution.

optimization (finding the v alue of an argument that minimizes or maximizes a function

The fundamen tal difficulty in p erforming contin uous math on a digital computer is that w e need to represent infinitely many real num b ers with a finite num ber of bit patterns

Underflo w o ccurs when n um b ers near zero are rounded to zero

Overflo w o ccurs when n um bers with large magnitude are appro ximated as ∞ or −∞

The softmax function is often used to predict the probabilities asso ciated with a multinoulli distribution

The softmax function is defined to b e softmax( ) x i = exp( x i ) n j =1 exp( x j )

Both of these difficulties can b e resolved by instead ev aluating softmax ( z ) where z = x − max_i x_i . Simple algebra shows that the v alue of the softmax function is not c hanged analytically by adding or subtracting a scalar from the input vector. Subtracting max i x i results in the largest argument to exp b eing 0, whic h rules out the p ossibility of ov erflo w. Lik ewise, at least one term in the denominator has a v alue of 1, which rules out the p ossibility of underflow in the denominator leading to a division by zero

Theano ( , ; , ) is an example Bergstra et al. 2010 Bastien et al. 2012 of a softw are pack age that automatically detects and stabilizes man y common n umerically unstable expressions that arise in the context of deep learning

Conditioning refers to how rapidly a function c hanges with resp ect to small changes in its inputs

Consider the function f ( x ) = A − 1 x . When A ∈ R n n × has an eigenv alue decomp osition, its condition num ber is max i,j λ i λ j . (4.2) This is the ratio of the magnitude of the largest and smallest eigen v alue

Optimization refers to the task of either minimizing or maximizing some function f ( x ) b y altering x .

W e usually phrase most optimization problems in terms of minimizing f ( x ) . Maximization ma y b e accomplished via a minimization algorithm by minimizing − f (x)

The function we wan t to minimize or maximize is called the ob jectiv e func- tion or criterion . When we are minimizing it, w e may also call it the cost function , loss function , or error function .

W e often denote the v alue that minimizes or maximizes a function with a sup erscript . F or example, w e might say ∗

The deriv ativ e is therefore useful for minimizing a function b ecause it tells us how to change x in order to mak e a small improv emen t in y . F or example, w e know that f ( x − sign ( f ( x ))) is less than f ( x ) for small enough . W e can th us reduce f ( x ) b y moving x in small steps with opp osite sign of the deriv ativ e. This tec hnique is called gradien t descen t

When f ( x ) = 0 , the deriv ative provides no information ab out which direction to mov e. Poin ts where f ( x ) = 0 are known as critical p oints or stationary p oin ts .

A lo cal minim um is a p oint where f ( x ) is low er than at all neighboring p oin ts, so it is no longer p ossible to decrease f ( x ) b y making infinitesimal steps

A lo cal maximum is a p oint where f ( x ) is higher than at all neighboring p oin ts

Some critical p oin ts are neither maxima nor minima. These are kno wn as saddle p oints

The concept of representing a distribution by a large representative sample is foundational for the approach we take to Bayesian analysis of complex models

What is new in the present application is that the population from which we are sampling is a mathematically defined distribution, such as a posterior probability distributio

Here is a summary of our algorithm for moving from one position to another. We are currently at position θ current . We then propose to move one position right or one position left. The specific proposal is determined by flipping a coin, which can result in 50% heads (move right) or 50% tails (move left). The range of possible proposed moves, and the probability of proposing each, is called the proposal distribution. In the present algorithm, the proposal distribution is very simple: It has only two values with 50-50 probabilities. Having proposed a move, we then decide whether or not to accept it. The acceptance decision is based on the value of the target distribution at the proposed position, relative to the value of the target distribution at our current position. Specifically, if the target distribution is greater at the proposed position than at our current position, then we definitely accept the proposed move: We always move higher if we can. On the other hand, if the target position is less at the proposed position than at our current position, we accept the move probabilistically: We move to the proposed position with probability p move =P(θ proposed )/P(θ current ),whereP(θ) is the value of the target distribution at θ .We can combine these two possibilities, of the target distribution being higher or lower at the proposed position than at our current position, into a single expression for the probability of moving to the proposed position: p move = min P(θ proposed ) P(θ current ) ,1 (7.1) Notice that Equation 7.1 says that when P(θ proposed )>P(θ current ),thenp move =1. Notice also that the target distribution, P(θ), does not need to be normalized, which means it does not need to sum to 1 as a probability distribution must. This is because what matters for our choice is the ratio, P(θ proposed )/P(θ current ), not the absolute magnitude of P(θ)

Notice what we must be able to do in the random-walk process: • We must be able to generate a random value from the proposal distribution, to create θ proposed . • We must be able to evaluate the target distribution at any proposed position, to compute P(θ proposed )/P(θ current ). • We must be able to generate a random value from a uniform distribution, to accept or reject the proposal according to p move

Suppose we are at position θ. The probability of moving to θ + 1, denoted p(θ → θ + 1), is the probability of proposing that move times the probability of accepting it if proposed, which is p(θ → θ + 1) =0.5 · min ( P(θ + 1)/P(θ ),1 )

On the other hand, if we are presently at position θ + 1, the probability of moving to θ is the probability of proposing that move times the probability of accepting it if proposed, which is p(θ + 1 → θ)=0.5 · min ( P(θ)/P(θ + 1),1 ) .

The ratio of the transition probabilities is \(\frac{p(θ → θ +1)}{ p(θ +1 → θ)} = \frac{0.5 min ( P(θ +1)/P(θ),1 )}{ 0.5 min ( P(θ)/P(θ +1),1 )}\) =
\(\frac {1} {P(\theta)/P(\theta+1)} \) (if P(\theta+1) > P(\theta))

\(\frac {P(\theta)/P(\theta+1)} {1}\) (if P(\theta+1) < P(\theta))

= \(\frac{P(\theta+1)} {P(\theta)}\)

When the vector of position probabilities is the target distribution, it stays that way on the next time step! In other words, the position probabilities are stable at the target distribution

Sample values from the target distribution are generated by taking a random walk through the parameter space. The walk starts at some arbitrary point, specified by the user. The starting point should be someplace where P(θ ) is nonzero. The random walk progresses at each time step by proposing a move to a new position in parameter space and then deciding whether or not to accept the proposed move. Proposal distributions can take on many different forms, with the goal being to use a proposal distribution that efficiently explores the regions of the parameter space where P(θ ) has most of its mass.

Having generated a proposed new position, the algorithm then decides whether or not to accept the proposal. The decision rule is exactly what was already specified in Equation 7.1. In detail, this is accomplished by computing the ratio p move =P(θ proposed )/P(θ current ).

Here’s how to get handles: The functions that draw graphics objects can also be used to return the handle of the object drawn, e.g., x = 0:pi/20:2*pi; hsin = plot(x, sin(x)) hold on hx = xlabel(’x’)

gcf gets the handle of the current figure, e.g., hf = gcf

gca gets the handle of the current axes

gco gets the handle of the current graphics object, which is the last graph- ics object created or clicked on. For example, draw the sine graph above and get its handle hsin. Click on the graph in the figure win- dow. Then enter the command ho = gco

Handle Graphics objects are the basic elements used in MATLAB graphics. The objects are arranged in a parent-child inheritance structure

To see all the property names of an object and their current values use get(h) where h is the object’s handle

You can change any property value with the set function: set(handle, ‘PropertyName’, PropertyValue)

The command set(handle) lists all the possible property values (where appro- priate)

If a graphics object has a number of children the get command used with the children property returns a vector of the children’s handles

The answer is that the handles of children of the axes are returned in the reverse order in which they are created

If you are desperate and don’t know the handles of any of your graphics ob- jects you can use the findobj function to get the handle of an object with a property value that uniquely identifies it

you can specify the parent of an object when you create it