Edited, memorised or added to reading queue

on 02-Mar-2017 (Thu)

Do you want BuboFlash to help you learning these things? Click here to log in or create user.

#matlab #programming
Recursive functions are usually written in this way: an if statement handles the general recursive definition; the else part handles the special case (n = 1)
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
where the number of repetitions must be determined in advance, is sometimes called determinate repetition.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
it often happens that the condition to end a loop is only satisfied during the execution of the loop itself . Such a structure is called indeterminate.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
If there are a number of different conditions to stop a while loop you may be tempted to use a for with the number of repetitions set to some accepted cut-off value (or even Inf) but enclosing if statements which break outofthe for when the various conditions are met. Why is this not regarded as the best programming style? The reason is simply that when you read the code months later you will have to wade through the whole loop to find all the conditions to end it, rather than see them all paraded at the start of the loop in the while clause
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
Graphs (in 2-D) are drawn with the plot statement. In its simplest form, it takes a single vector argument as in plot(y)
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
plot(y). In this case the elements of y are plotted against their indexes, e.g., plot(rand(1, 20)) plots 20 random num- bers against the integers 1–20, joining successive points with straight lines,
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
Probably the most common form of plot is plot(x, y) where x and y are vectors of the same length, e.g., x = 0:pi/40:4*pi; plot(x, sin(x)) In this case, the co-ordinates of the ith point are x i , y i
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
Straight-line graphs are drawn by giving the x and y co-ordinates of the end- points in two vectors.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
MATLAB has a set of ‘easy-to-use’ plotting commands, all starting with the string ‘ez’. The easy-to-use form of plot is ezplot, e.g., ezplot(’tan(x)’)
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
9.1 Basic 2-D graphs 199 gtext(’text’) writes a string (’text’) in the graph window. gtext puts a cross-hair in the graph window and waits for a mouse button or keyboard key to be pressed.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
Text may also be placed on a graph interactively with Tools -> Edit Plot from the figure window
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
grid adds/removes grid lines to/from the current graph. The grid state may be toggled
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
text(x, y, ’text’) writes text in the graphics window at the point speci- fied by x and y. If x and y are vectors, the text is written at each point. If the text is an indexed list, successive points are labeled with corresponding rows of the text
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
title(’text’) writes the text as a title on top of the graph
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
xlabel(’horizontal’) labels the x-axis
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
ylabel(’vertical’) labels the y-axis
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
There are at least three ways of drawing multiple plots on the same set of axes (which may however be rescaled if the new data falls outside the range of the previous data). 1. The easiest way is simply to use hold to keep the current plot on the axes. All subsequent plots are added to the axes until hold is released, either with hold off,orjusthold, which toggles the hold state. 2. The second way is to use plot with multiple arguments
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The third way is to use the form plot(x, y) where x and y may both be matrices, or where one may be a vector and one a matrix. If one of x or y is a matrix and the other is a vector, the rows or columns of the matrix are plotted against the vector, using a different color for each.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
If x is not specified, as in plot(y),wherey is a matrix, the columns of y are plotted against the row index
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
If x and y are both matrices of the same size, the columns of x are plotted against the columns of y
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
plot(x, y, ’--’) joins the plotted points with dashed lines, whereas plot(x, y, ’o’) draws circles at the data points with no lines joining them
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The available colors are denoted by the symbols c, m, y, k, r, g, b, w
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
axis( [xmin, xmax, ymin, ymax] ) which sets the scaling on the current plot, i.e., draw the graph first, then reset the axis limits.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
If you want to specify one of the minimum or maximum of a set of axis limits, but want MATLAB to autoscale the other, use Inf or -Inf for the autoscaled limit
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
You can return to the default of automatic axis scaling with axis auto
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The statement v = axis returns the current axis scaling in the vector v.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
Scaling is frozen at the current limits with axis manual so that if hold is turned on, subsequent plots will use the same limit
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
in MATLAB the word ‘axes’ refers to a particular graphics object, which includes not only the x-axis and y-axis and their tick marks and labels, but also everything drawn on those particular axes: the actual graphs and any text included in the figure
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
You can show a number of plots in the same figure window with the subplot function. It looks a little curious at first, but it’s quite easy to get the hang of it. The statement subplot(m, n, p) divides the figure window into m × n small sets of axes, and selects the pth set for the current plot (numbered by row from the left of the top row)
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
figure(h),whereh is an integer, creates a new figure window, or makes figure h the current figure
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
clf clears the current figure window. It also resets all properties associated with the axes, such as the hold state and the axis state
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
cla deletes all plots and text from the current axes, i.e., leaves only the x- and y-axes and their associated information
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The command [x, y] = ginput allows you to select an unlimited number of points from the current graph us- ing a mouse or arrow keys. A movable cross-hair appears on the graph. Clicking saves its co-ordinates in x(i) and y(i). Pressing Enter terminates the input.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The command [x, y] = ginput(n) works like ginput except that you must select exactly n points
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
he command semilogy(x, y) plots y with a log 10 scale and x with a linear scale
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The command polar(theta, r) generates a polar plot of the points with angles in theta and magnitudes in r
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
Plotting rapidly changing mathematical functions: fplot
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The function plot3 is the 3-D version of plot. The command plot3(x, y, z) draws a 2-D projection of a line in 3-D through the points whose co-ordinates are the elements of the vectors x, y and z
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The function comet3 is similar to plot3 except that it draws with a moving ‘comet head’.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The function mesh draws a surface as a ‘wire frame’.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
An alternative visualization is provided by surf, which generates a faceted view of the surface (in color), i.e. the wire frame is covered with small tiles
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
contour(u) You should get a contour plot of the heat distribution
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The function contour can take a second input variable. It can be a scalar spec- ifying how many contour levels to plot, or it can be a vector specifying the values at which to plot the contour levels
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
You can get a 3-D contour plot with contour3
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
Contour levels may be labeled with clabel
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
A 3-D contour plot may be drawn under a surface with meshc or surfc
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
If a matrix for a surface plot contains NaNs, these elements are not plotted. This enables you to cut away (crop) parts of a surface
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The function quiver draws little arrows to indicate a gradient or other vec- tor field
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The mesh function can also be used to ‘visualize’ a matrix
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The function spy is useful for visualizing sparse matrices
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The view function enables you to specify the angle from which you view a 3- D graph
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The function view takes two arguments. The first one, az in this example, is called the azimuth or polar angle in the x-y plane (in degrees). az rotates the viewpoint (you) about the z-axis—i.e. about the ‘pinnacle’ at (15,15) in Fig- ure 9.12—in a counter-clockwise direction. The default value of az is −37.5 ◦ . The program therefore rotates you in a counter-clockwise direction about the z-axis in 15 ◦ steps starting at the default position. The second argument of view is the vertical elevation el (in degrees). This is the angle a line from the viewpoint makes with the x-y plane. A value of 90 ◦ for el means you are directly overhead. Positive values of the elevation mean you are above the x-y plane; negative values mean you are below it. The default value of el is 30 ◦
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The command pause(n) suspends execution for n seconds.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
You can rotate a 3-D figure interactively as follows. Click the Rotate 3-D button in the figure toolbar (first button from the right). Click on the axes and an outline of the figure appears to help you visualize the rotation. Drag the mouse
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
the cell can form a hydrogel that pulls these and other molecules into punctate structures called intracellular bodies, or granules. Specific mRNAs can be seques- tered in such granules, where they are stored until made available by a controlled disassembly of the core amyloid structure that holds them together.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
the FUS protein, an essential nuclear protein with roles in the tran- scription, processing, and transport of specific mRNA molecules.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
Over 80 per- cent of its C-terminal domain of two hundred amino acids is composed of only four amino acids: glycine, serine, glutamine, and tyrosine. This low complexity domain is attached to several other domains that bind to RNA molecules.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
At high enough concentrations in a test tube, FUS protein forms a hydrogel that will associate with either itself or with the low complexity domains from other proteins.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
FUS low complexity domain binds most tightly to itself
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
both the homotypic and the heterotypic bindings are mediated through a β-sheet core structure forming amyloid fibrils, and that these structures bind to other types of repeat sequences
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
Many of these interactions appear to be controlled by the phosphorylation of ser- ine side chains in the one or both of the interacting partners.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
he amyloid fibril is a long unbranched structure assembled through a repeating aggre- gate of β sheets
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
The substance that is bound by the protein—whether it is an ion, a small molecule, or a macromolecule such as another protein—is referred to as a ligand for that protein (from the Latin word ligare, meaning “to bind”)
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
Even small changes to the amino acids in the interior of a protein molecule can change its three-dimensional shape enough to destroy a binding site on the surface
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
the interaction of neighboring parts of the polypeptide chain may restrict the access of water molecules to that protein’s ligand-binding sites. Because water molecules readily form hydrogen bonds that can compete with ligands for sites
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
a ligand will form tighter hydrogen bonds (and electro- static interactions) with a protein if water molecules are kept away
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
In effect, a protein can keep a ligand-binding site dry, increasing that site's reactivity, because it is energetically unfavorable for individual water mole- cules to break away from this network—as they must do to reach into a crevice on a protein’s surface
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
the clustering of neighboring polar amino acid side chains can alter their reactivity. If protein folding forces together a number of negatively charged side chains against their mutual repulsion, for example, the affinity of the site for a positively charged ion is greatly increased.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
when amino acid side chains interact with one another through hydrogen bonds, normally unreactive groups (such as the –CH 2 OH on the serine shown in Figure 3–39) can become reactive, enabling them to be used to make or break selected covalent bonds
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
even when the amino acid sequence identity falls to 25%, the backbone atoms in a domain can follow a common protein fold within 0.2 nanometers (2 Å)
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
We can use a method called evolutionary tracing to identify those sites in a protein domain that are the most crucial to the domain’s function. Those sites that bind to other molecules are the most likely to be maintained, unchanged as organisms evolve. Thus, in this method, those amino acids that are unchanged, or nearly unchanged, in all of the known protein family members are mapped onto a model of the three-dimensional structure of one family member. When this is done, the most invariant positions often form one or more clusters on the protein surface, as illustrated in Figure 3–40A for the SH2 domain described previously (see Figure 3–6). These clusters generally correspond to ligand-binding sites.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
The amino acids located at the binding site for the phosphorylated polypeptide have been the slowest to change during the long evolutionary process
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
n many cases, a portion of the surface of one protein contacts an extended loop of polypeptide chain (a “string”) on a second protein
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
A second type of protein–protein interface forms when two α helices, one from each protein, pair together to form a coiled-coil
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
The most common way for proteins to interact, however, is by the precise matching of one rigid surface with that of another
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
Different antibod- ies generate an enormous diversity of antigen-binding sites by changing only the length and amino acid sequence of these loops, without altering the basic protein structure.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
A detailed examination of the antigen-binding sites of antibodies reveals that they are formed from several loops of polypeptide chain that protrude from the ends of a pair of closely juxtaposed protein domains
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
Antibodies are Y-shaped molecules with two identical binding sites that are complementary to a small portion of the surface of the antigen molecule
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
Loops of this kind are ideal for grasping other molecules. They allow a large number of chemical groups to surround a ligand so that the protein can link to it with many weak bonds. For this reason, loops often form the ligand-binding sites in proteins
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#biochem #biology #cell
Strong interactions occur in cells when- ever a biological function requires that molecules remain associated for a long time—for example, when a group of RNA and protein molecules come together to make a subcellular structure such as a ribosome
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Self-information deals only with a single outcome. W e can quantify the amoun t of uncertain t y in an en tire probabilit y distribution using the Shannon entrop y : H ( ) = x E x ∼ P [ ( )] = I x − E x ∼ P [log ( )] P x . (3.49) also denoted H ( P )
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
the Shannon en trop y of a distribution is the exp ected amoun t of information in an ev en t dra wn from that distribution
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
It gives a lo w er bound on the num b er of bits (if the logarithm is base 2, otherwise the units are differen t) needed on av erage to enco de symbols drawn from a distribution P
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Distributions that are nearly deterministic (where the outcome is nearly certain) ha v e lo w en trop y; distributions that are closer to uniform hav e high entrop y
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
When 3.5 x is con tin uous, the Shannon entrop y is kno wn as the differen tial entrop y
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
If we hav e t w o separate probability distributions P ( x ) and Q ( x ) ov er the same random v ariable x , we can measure ho w different these t w o distributions are using the Kullbac k-Leibler (KL) div ergence : D KL ( ) = P Q E x ∼ P log P x ( ) Q x ( ) = E x ∼ P [log ( ) log ( )] P x − Q x
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Kullbac k-Leibler (KL) div ergence : \(D_{KL}( P||Q) = E_{x\sim P} = [log\frac{ P (x )}{ Q (x )}] = E_{x \sim P} [log P(x) - log Q(x)] \) (3.50) In the case of discrete v ariables, it is the extra amount of information (measured in bits if we use the base 2 logarithm, (but in machine learning w e usually use nats and the natural logarithm) needed to send a message containing symbols drawn from probability distribution P , when w e use a co de that w as designed to minimize the length of messages dra wn from probabilit y distribution Q.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
The KL div ergence has many useful prop erties, most notably that it is non- negativ e.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
The KL divergence is 0 if and only if P and Q are the same distribution in the case of discrete v ariables, or equal “almost ev erywhere” in the case of con tin uous v ariables
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
KL div ergence is non-negativ e and measures the difference b et w een t w o distributions, it is often conceptualized as measuring some sort of distance b etw een these distributions. How ever, it is not a true distance measure b ecause it is not symmetric
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
A quan tit y that is closely related to the KL div ergence is the cross-en trop y \(H ( P , Q ) = H ( P ) + D_{KL}( P|| Q )\) , whic h is similar to the KL div ergence but lac king the term on the left: \(H (P, Q ) = − E _{x \sim P} \space log Q(x)\)
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Minimizing the cross-entrop y with resp ect to Q is equiv alent to minimizing the KL div ergence, b ecause do es not participate in the omitted term. Q
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
the form 0 log 0 . By con v en tion, in the con text of information theory , w e treat these expressions as lim x → 0 x x log = 0
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
we can greatly reduce the cost of representing a distribution if w e are able to find a factorization in to distributions o v er few er v ariables
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
When we represen t the factorization of a probabilit y distribution with a graph, we call it a structured probabilistic mo del or graphical mo del
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
There are t w o main kinds of structured probabilistic mo dels: directed and undirected.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
graphical models use a graph G in which each no de in the graph corresp onds to a random v ariable, and an edge connecting tw o random v ariables means that the probability distribution is able to represen t direct in teractions b et w een those t w o random v ariables
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Directed mo dels use graphs with directed edges, and they represen t fac- torizations into conditional probability distributions
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
a directed mo del contains one factor for ev ery random v ariable x i in the distribution, and that factor consists of the conditional distribution ov er x i giv en the paren ts of x i , denoted P a G ( x i ) : p ( ) = x i p ( x i | P a G ( x i ))
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Undirected mo dels use graphs with undirected edges, and they represen t factorizations into a set of functions
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Any set of no des that are all connected to each other in G is called a clique.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Each clique C ( ) i in an undirected mo del is asso ciated with a factor φ ( ) i ( C ( ) i )
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Each clique C ( ) i in an undirected mo del is asso ciated with a factor φ ( ) i ( C ( ) i ) . These factors are just functions, not probabilit y distributions
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
The probability of a configuration of random v ariables is prop ortional to the pro duct of all of these factors—assignments that result in larger factor v alues are more lik ely
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
W e therefore divide by a normalizing constant Z , defined to b e the sum or integral o v er all states of the pro duct of the φ functions, in order to obtain a normalized probabilit y distribution: p ( ) = x 1 Z i φ ( ) i C ( )
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
numerical compu- tation. This t ypically refers to algorithms that solve mathematical problems b y metho ds that update estimates of the solution via an iterative pro cess, rather than analytically deriving a form ula providing a symbolic expression for the correct so- lution.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
optimization (finding the v alue of an argument that minimizes or maximizes a function
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
The fundamen tal difficulty in p erforming contin uous math on a digital computer is that w e need to represent infinitely many real num b ers with a finite num ber of bit patterns
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Underflo w o ccurs when n um b ers near zero are rounded to zero
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Overflo w o ccurs when n um bers with large magnitude are appro ximated as ∞ or −∞
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
The softmax function is often used to predict the probabilities asso ciated with a multinoulli distribution
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
The softmax function is defined to b e softmax( ) x i = exp( x i ) n j =1 exp( x j )
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Both of these difficulties can b e resolved by instead ev aluating softmax ( z ) where z = x − maxi xi . Simple algebra shows that the v alue of the softmax function is not c hanged analytically by adding or subtracting a scalar from the input vector. Subtracting max i x i results in the largest argument to exp b eing 0, whic h rules out the p ossibility of ov erflo w. Lik ewise, at least one term in the denominator has a v alue of 1, which rules out the p ossibility of underflow in the denominator leading to a division by zero
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Theano ( , ; , ) is an example Bergstra et al. 2010 Bastien et al. 2012 of a softw are pack age that automatically detects and stabilizes man y common n umerically unstable expressions that arise in the context of deep learning
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Conditioning refers to how rapidly a function c hanges with resp ect to small changes in its inputs
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Consider the function f ( x ) = A − 1 x . When A ∈ R n n × has an eigenv alue decomp osition, its condition num ber is max i,j λ i λ j . (4.2) This is the ratio of the magnitude of the largest and smallest eigen v alue
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Optimization refers to the task of either minimizing or maximizing some function f ( x ) b y altering x .
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
W e usually phrase most optimization problems in terms of minimizing f ( x ) . Maximization ma y b e accomplished via a minimization algorithm by minimizing − f (x)
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
The function we wan t to minimize or maximize is called the ob jectiv e func- tion or criterion . When we are minimizing it, w e may also call it the cost function , loss function , or error function .
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
W e often denote the v alue that minimizes or maximizes a function with a sup erscript . F or example, w e might say ∗
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
The deriv ativ e is therefore useful for minimizing a function b ecause it tells us how to change x in order to mak e a small improv emen t in y . F or example, w e know that f ( x − sign ( f ( x ))) is less than f ( x ) for small enough . W e can th us reduce f ( x ) b y moving x in small steps with opp osite sign of the deriv ativ e. This tec hnique is called gradien t descen t
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
When f ( x ) = 0 , the deriv ative provides no information ab out which direction to mov e. Poin ts where f ( x ) = 0 are known as critical p oints or stationary p oin ts .
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
A lo cal minim um is a p oint where f ( x ) is low er than at all neighboring p oin ts, so it is no longer p ossible to decrease f ( x ) b y making infinitesimal steps
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
A lo cal maximum is a p oint where f ( x ) is higher than at all neighboring p oin ts
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#deeplearning #neuralnetworks
Some critical p oin ts are neither maxima nor minima. These are kno wn as saddle p oints
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#bayes #programming #r #statistics
The concept of representing a distribution by a large representative sample is foundational for the approach we take to Bayesian analysis of complex models
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#bayes #programming #r #statistics
What is new in the present application is that the population from which we are sampling is a mathematically defined distribution, such as a posterior probability distributio
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#bayes #programming #r #statistics
Here is a summary of our algorithm for moving from one position to another. We are currently at position θ current . We then propose to move one position right or one position left. The specific proposal is determined by flipping a coin, which can result in 50% heads (move right) or 50% tails (move left). The range of possible proposed moves, and the probability of proposing each, is called the proposal distribution. In the present algorithm, the proposal distribution is very simple: It has only two values with 50-50 probabilities. Having proposed a move, we then decide whether or not to accept it. The acceptance decision is based on the value of the target distribution at the proposed position, relative to the value of the target distribution at our current position. Specifically, if the target distribution is greater at the proposed position than at our current position, then we definitely accept the proposed move: We always move higher if we can. On the other hand, if the target position is less at the proposed position than at our current position, we accept the move probabilistically: We move to the proposed position with probability p move =P(θ proposed )/P(θ current ),whereP(θ) is the value of the target distribution at θ .We can combine these two possibilities, of the target distribution being higher or lower at the proposed position than at our current position, into a single expression for the probability of moving to the proposed position: p move = min P(θ proposed ) P(θ current ) ,1 (7.1) Notice that Equation 7.1 says that when P(θ proposed )>P(θ current ),thenp move =1. Notice also that the target distribution, P(θ), does not need to be normalized, which means it does not need to sum to 1 as a probability distribution must. This is because what matters for our choice is the ratio, P(θ proposed )/P(θ current ), not the absolute magnitude of P(θ)
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#bayes #programming #r #statistics
Notice what we must be able to do in the random-walk process: • We must be able to generate a random value from the proposal distribution, to create θ proposed . • We must be able to evaluate the target distribution at any proposed position, to compute P(θ proposed )/P(θ current ). • We must be able to generate a random value from a uniform distribution, to accept or reject the proposal according to p move
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#bayes #programming #r #statistics
Suppose we are at position θ. The probability of moving to θ + 1, denoted p(θ → θ + 1), is the probability of proposing that move times the probability of accepting it if proposed, which is p(θ → θ + 1) =0.5 · min ( P(θ + 1)/P(θ ),1 )
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#bayes #programming #r #statistics
On the other hand, if we are presently at position θ + 1, the probability of moving to θ is the probability of proposing that move times the probability of accepting it if proposed, which is p(θ + 1 → θ)=0.5 · min ( P(θ)/P(θ + 1),1 ) .
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#bayes #programming #r #statistics
The ratio of the transition probabilities is \(\frac{p(θ → θ +1)}{ p(θ +1 → θ)} = \frac{0.5 min ( P(θ +1)/P(θ),1 )}{ 0.5 min ( P(θ)/P(θ +1),1 )}\) =
\(\frac {1} {P(\theta)/P(\theta+1)} \) (if P(\theta+1) > P(\theta))

\(\frac {P(\theta)/P(\theta+1)} {1}\) (if P(\theta+1) < P(\theta))

= \(\frac{P(\theta+1)} {P(\theta)}\)
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#bayes #programming #r #statistics
When the vector of position probabilities is the target distribution, it stays that way on the next time step! In other words, the position probabilities are stable at the target distribution
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#bayes #programming #r #statistics
Sample values from the target distribution are generated by taking a random walk through the parameter space. The walk starts at some arbitrary point, specified by the user. The starting point should be someplace where P(θ ) is nonzero. The random walk progresses at each time step by proposing a move to a new position in parameter space and then deciding whether or not to accept the proposed move. Proposal distributions can take on many different forms, with the goal being to use a proposal distribution that efficiently explores the regions of the parameter space where P(θ ) has most of its mass.
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#bayes #programming #r #statistics
Having generated a proposed new position, the algorithm then decides whether or not to accept the proposal. The decision rule is exactly what was already specified in Equation 7.1. In detail, this is accomplished by computing the ratio p move =P(θ proposed )/P(θ current ).
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
Here’s how to get handles: The functions that draw graphics objects can also be used to return the handle of the object drawn, e.g., x = 0:pi/20:2*pi; hsin = plot(x, sin(x)) hold on hx = xlabel(’x’)
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
gcf gets the handle of the current figure, e.g., hf = gcf
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
gca gets the handle of the current axes
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
gco gets the handle of the current graphics object, which is the last graph- ics object created or clicked on. For example, draw the sine graph above and get its handle hsin. Click on the graph in the figure win- dow. Then enter the command ho = gco
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
Handle Graphics objects are the basic elements used in MATLAB graphics. The objects are arranged in a parent-child inheritance structure
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
To see all the property names of an object and their current values use get(h) where h is the object’s handle
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
You can change any property value with the set function: set(handle, ‘PropertyName’, PropertyValue)
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The command set(handle) lists all the possible property values (where appro- priate)
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
If a graphics object has a number of children the get command used with the children property returns a vector of the children’s handles
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
The answer is that the handles of children of the axes are returned in the reverse order in which they are created
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
If you are desperate and don’t know the handles of any of your graphics ob- jects you can use the findobj function to get the handle of an object with a property value that uniquely identifies it
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs




#matlab #programming
you can specify the parent of an object when you create it
statusnot read reprioritisations
last reprioritisation on suggested re-reading day
started reading on finished reading on

pdf

cannot see any pdfs