MTH 540/640: Statistical Theory I

Mon/Wed/Fri: 9:05 am -- 9:55 am, Spring 2024, Cheek Hall 173

Course Description:

The course MTH 540/640 is devoted to probability theory, which is widely used in modern sciences and technologies. Students will be equipped with probability theory, thoughts, and methodology when they leave the course; also students are expected to be able to solve practical application problems. The prerequisite for this course is MTH 302 and MTH 315, or equivalent; students are expected to be familiar with calculus (differentiation, integral, different properties of sum of infinite series). Previous courses about probability and statistics are not required but definitely a plus.

Tentative syllabus is available here. 

Textbooks:

• Probability and Statistics, 4th Edition, by Morris H. DeGroot, and Mark J. Schervish. E-version available here.

• Lecture Notes will be available.

Announcements:

• Notes: #1  , #2  , #3 ,   #4 ,    #5 ,   #6 ,  #7    , #8    , #9   .

• HW#1 is available here. Due: Fri. Feb. 2nd.

• HW#2 is available here. Due: Fri. Feb. 9th.

• A simulation on Monty Hall Example: http://www.mathwarehouse.com/monty-hall-simulation-online/

• HW#3 is available here. Due: Fri., Feb. 16th.

• Exam #1: Fri., Feb. 23rd, 8:55 -- 10:10. Coverage: note #1 to note #3.

• HW#4 is available here. Due: Fri., Mar. 1st. 

• HW#5 is available here. Due: Mon., Mar. 18th.

• Project: Implementing several Normal Random number generating method using R. Due: Wed., Apr. 10th.

• Please install and learn how to use R by yourselves! You can find tutorial materials by Google: just type in "R tutorial filetype:pdf", you will have a lot of tutorial files available. For example: http://www.nceas.ucsb.edu/files/scicomp/Dloads/RProgramming/BestFirstRTutorial.pdf

• HW#6 is available here, Due: Wed., Mar. 27th.

• HW#7 is available here. Due: Mon. Apr. 8th.

• Exam #2: Wed., Apr. 10th, 8:55 -- 10:10. Coverage: note #4 to note #6.

• HW#8 is available here. Due: Fri. Apr. 26th.

• HW#9 is available here. Due: Mon. May 6th.

• Final Exam: Monday, May 6th, 8:30am! Comprehensive.

Topics:

• Lecture 1 (Jan. 17th): Course Statement. Basic concepts: Experiment, sample space, event, Examples. Relation between events, Event Operations

• Lecture 2 (Jan. 19th): Examples for Event Operations, DeMorgan Laws. Axioms for Probability. Properties of Probability, Proofs.

• Lecture 3 (Jan. 24th): Probability for Union of events: proof, explanations, examples.

• Lecture 4 (Jan. 26th): Finite sample space, probability for equally likely events, Examples. Multiplication rule and Examples.

• Lecture 5 (Jan. 29th): Examples for Multiplication rule. Permutation: definition and calculation. Permutation and Combinations: definition and calculation. Examples.

• Lecture 6 (Jan. 31st): Examples for combination problems. Partition problems and formula. Proof of Partition formula, Example. Binomial coefficients, Example.

• Lecture 7 (Feb. 2nd): Multinomial coefficients and example. Conditional probability: motivation, definition, explanation and examples.

• Lecture 8 (Feb. 5th): Multiplication formula for probability, example. Independence: explanation, definition. Examples.

• Lecture 9 (Feb. 7th): Independence of more than two events, pairwise independence. Examples. Partition, Total probability formula and its proof. Example.

• Lecture 10 (Feb. 9th): Bayesian formula and proof. Examples for Bayes rule. Monty Hall Problem.

• Lecture 11 (Feb. 12th): Random variable: definition and notations. Examples of Random variable. Probability function of discrete Random variables, Examples.

• Lecture 12 (Feb. 14th): Bernoulli R.V., Binomial R.V, Uniform R. V., Examples for Bernoulli R.V. and Binomial R.V. Cumulative distribution function of discrete R.V., calculation and example.

• Lecture 13 (Feb. 16th): Definition of Continuous R.V. Properties of probability density function, differences between continuous and discrete random variables.Cumulative distribution function of continuous random variable. Examples: Uniform, Cauchy, and Exponential distribution.

• Lecture 14 (Feb. 21st): Standard Normal distribution. p.d.f. of transformed random variable, Example.

• Lecture 15 (Feb. 26th): Exam #1 problems. p.d.f. of transformed random variable, Examples.

• Lecture 16 (Feb. 28th): Joint discrete random variables and distribution, example. Joint pdf and related probabilities, examples.

• Lecture 17 (Mar. 1st): Joint pdf and related probabilities, examples.

• Lecture 18 (Mar. 4th): Bivariate Distribution function and properties. Example. Marginal density function: definition, explanation and example.

• Lecture 19 (Mar. 6th): Independent random variables, Examples. Factorization theorem for independent random variables: proof and example.

• Lecture 20 (Mar. 8th): Conditional p.d.f., explanation and examples.

• Lecture 21 (Mar. 18th): Bivariant transformation and examples. Generating normal random variable from uniform random variable.

• Lecture 22 (Mar. 20th): Continuous Bayes formula, example. Joint pdf of n random variable and related concepts.

• Lecture 23 (Mar. 22nd): Independence of n random variable, iid random variables. Examples.

• Lecture 24 (Mar. 25th): Expectation of discrete and continuous random variables, Examples. Expectation of function, Examples.

• Lecture 25 (Mar. 27th):Variance: definition and its understanding, calculation and Examples. Expectation of multivariant random variables, Examples. Properties of Expectation.

• Lecture 26 (Apr. 1st): HW#6 problems. Properties of Expectation: proof and Various Examples.

• Lecture 27 (Apr. 3rd): Variance of sum of independent R. V. Covariance of two random variables: definition, explanation, formulas. Properties of covariance.

• Lecture 28 (Apr. 5th): Properties of covariance, Cauchy-Schwarz inequality and the proof, connections to the related concepts in linear algebra and calculus. Correlation coefficient: definition, explanation, and Example. Examples for calcuating covariances and correlation coefficient.

• Lecture 29 (Apr. 8th): Markov Inequality and Chebyshev Inequality. Weak law of large numbers and explanation. Moment generating function: definition and examples for Bernoulli and Binomial.

• Lecture 30 (Apr. 12th): Exam #2 problems. Moment generating function for exponential and normal. Moments, and calculating moments from moment generating function.

• Lecture 31 (Apr. 15th): Calculating moments from moment generating function, examples. Moment generating function of linear combination of independent random variables. Determining distribution from moment generating function, examples.

• Lecture 32 (Apr. 17th): Central Limit Theorem: motivation, statement, remarks. Proving Central Limit Theorem using MGF.

• Lecture 33 (Apr. 19th): Bernoulli, Binomial, Geometric, and Poisson Distributions: physical model, distribution functions, moment generating functions, properties.

• Lecture 34 (Apr. 22nd): Normal distribution: definition, relation to N(0, 1), expectation, variance, and moment generating function. Linear combination of independent normal. Definition of bivariate normal distribution.

• Lecture 35 (Apr. 24th): Relation between Bivariate Normal and standard normal. Marginal distributions in Bivariate normal distribution; correlation coefficient. Conditional distribution in 2D normal. Notes for 2D Normal.

• Lecture 36 (Apr. 26th): Gamma function and properties. Gamma distribution: pdf, mgf, Expectation, variance, and the additive property. Exponential distributions as special case of Gamma distribution.

• Lecture 37 (Apr. 29th): Chi square distribution as special case of Gamma distribution. Normal distribution and Chi square distribution. Beta distribution, log normal distribution.  

• Lecture 38 (May 1st): HW#8 problems. Multinomial distribution: definition, probability function, Marginal distribution, covariances. Notes.