Course Description:

The course MTH 741 will lay out the theoretical foundation of statistics. After taking this course, students are expected to: 1. Understand common statistical models and applications of probability; commonly used sampling distributions and density functions. 2. Determine moments and use moment generating functions. 3. Utilize exponential families, marginal and conditional distributions, transformation and change of variables. 4. Understand convergence concepts and large sample theory. This course is required for all graduate students in statistics. Prerequisite: Advanced calculus.

Tentative course policy statement is available here. 

Textbooks:

• No textbook required, lecture notes will be available.

Announcements:

• A simulation on Monty Hall Example: http://www.grand-illusions.com/simulator/montysim.htm

Topics:

• Lecture 1 (Aug. 18th): Course policy statement. sigma-algebra and its properties.

• Lecture 2 (Aug. 20th): Axioms of probability, and related properties.

• Lecture 3 (Aug. 25th): continuous from below/above and countable activity of probability measure. conditional probability and examples.

• Lecture 4 (Aug. 27th): Independent events, definition and examples, properties.

• Lecture 5 (Sept. 1st): Random Variable, definition and examples. Discrete Random variable and mass function. Examples.

• Lecture 6 (Sept. 3rd): Continuous random variables, density functions, and examples. Distribution functions: definition, calculation, examples. Transformed random variables.

• Lecture 7 (Sept. 8th): Density function of transformed random variables, examples. Calculating probability from distribution function. Characteristics of distribution function.

• Lecture 8 (Sept. 10th): joint discrete distribution, joint probability density function, related probabilities. Joint cumulative distribution function.

• Lecture 9 (Sept. 15th): Marginal p.d.f. and Example. Independent random variables, Examples. Factorization theorem for independent random variables: proof. Conditional p.d.f., explanation and examples. Bivariant transformation, Jacobian matrix, and examples.

• Lecture 10 (Sept. 17th): Bivariant transformation and examples. Generating normal random variable from uniform random variable. Joint pdf of n random variable and related concepts. Independence of n random variable, iid random variables. Examples. Expectation of discrete random variables, Examples.

• Lecture 11 (Sept. 22nd): Expectation of continuous random variables, Examples. Expectation of function, Examples. Variance: definition and its understanding, calculation and Examples. Expectation of multivariant random variables, Examples. Properties of Expectation: proof and Various Examples. Properties of Expectation: proof and Various Examples. Variance of sum of independent R. V.

• Lecture 12 (Sept. 24th): Covariance of two random variables: definition, explanation, formulas. Properties of covariance, Cauchy-Schwarz inequality and the proof, connections to the related concepts in linear algebra and calculus. Correlation coefficient: definition, explanation, and Example.

• Lecture 13 (Sept. 29th): Moment generating function: definition and examples for Bernoulli, Binomial, exponential, and normal. Calculating moments from moment generating function, examples. Moment generating function of linear combination of independent random variables. Determining distribution from moment generating function, examples. Bernoulli, Binomial, and Poisson Distribution: definition and Property.

• Lecture 14 (Oct. 1st): Normal distribution: definition, relation to N(0, 1), expectation, variance, and moment generating function. Linear combination of independent normal. Markov Inequality, Chebyshev inequality, converge in probability, weak law of large numbers.

• Lecture 15 (Oct. 6th): Proof of Weak Law of Large Numbers, and its explanation.

• Lecture 16 (Oct. 13th): Variations of Weak Law of Large Numbers, some examples.

• Lecture 17 (Oct. 15th): convex functions, Jensen's inequality, examples. Chernoff's idea.

• Lecture 18 (Oct. 20th): Application to sum of Bernoulli random variables, and sum of bounded random variables.

• Lecture 19 (Oct. 22nd): Chernoff's inequality, Hoeffding's inequality, Bernstein, Bennett.

• Lecture 20 (Oct. 27th): Large deviation rate function: definition, properties, and examples; Kullback Leibler divergence. Application to bound the sum of independent random variables. Borel-Cantelli Lemma and its proof.

• Lecture 21 (Oct. 29th): Application of Borel-Cantelli Lemma to random walk. Second Borel-Cantelli Lemma and its proof. Convergene with probability 1: definition and explanations.

• Lecture 22 (Nov. 3rd): Equivalent statements of almost sure convergence, examples. Kolmogorov's inequality and proof.

• Lecture 23 (Nov. 5th): Strong law of large numbers and proof.

• Lecture 24 (Nov. 10th): Glivenko-Cantelli theorem and its proof.

• Lecture 25 (Nov. 12th): Levy's inequality and its proof. The statement of law of iterated logrithm.

• Lecture 26 (Nov. 17th): Proof of Law of Iterated Logrithm for standard normal random variables.

• Lecture 27 (Nov. 19th): convergence in distribution: definition, explanation, example. Relation between different modes of convergence. Central Limit Theorem and its proof.