MTH 543/653: Stochastic Modeling

Mon/Wed/Fri: 1:25pm -- 2:15pm, Spring 2024, Cheek Hall 173

Course Description:

 The course MTH 543/653 is devoted to applications of probability and statistics from a modeling point of view. Topics include generating functions, branching processes, discrete time Markov chains, classification of states, estimation of transition probabilities, continuous time Markov Chains, Poisson processes, birth and death processes, renewal theory, queuing systems, Brownian motion, and stationary processes. The prerequisite for this course is MTH 540, or equivalent; students are expected to be familiar with calculus (differentiation, integral, different properties of sum of infinite series).

Tentative Syllabus is available here.

Textbooks:

• An Introduction to Stochastic Modeling, 4-th Edition, by Pinsky and Karlin. (Required) Available here.

• Introduction to Probability Models, 10-th Edition, by S. Ross.  (Reference)

Announcements:

• HW#1 is available here. Due: Wed., Feb. 21st.

• HW#2 is available here. Due: Wed., Mar. 6th.

• HW#3 is available here. Due: Wed. Mar. 20th.

• Computer Simulation (for Graduate Students): Please generate a Possion process with rate 1, and plot a graph. You can use any computer language (R, matlab, Python, ... ), but do not use the built-in function, if any.

• HW#4 is available here. Due: Mon. Apr. 8th.

• HW#5 is available here. Due: Fri. Apr. 19th.

• Computer Simulation: Please generate a pure birth process, the rate at each step is 1 more than the current population size. Please generate a pure death process, starting with 50, and the rate at each step is 1.

• HW#6 is available here. Due: Fri. Apr. 26th.

• HW#7 is available here. Due: Mon. May 6th.

• Computer Simulation: Please simulation a standard Brownian Motion in 1D. You can refer to the paper here.

• Final Exam: Monday, May 6th, 11:00am.

Topics:

• Lecture 1 (Jan. 17th): Statement of class policy. Experiment, sample space, event, operation of events, Axioms of probability, conditional probability.

• Lecture 2 (Jan. 19th): independence. Probability of union events, formula and examples.

• Lecture 3 (Jan. 24th): Random variables, discrete random variable and examples. Commonly used discrete distributions. Continuous random variable and properties.

• Lecture 4 (Jan. 26th): Continuous random variable and commonly used continuous distributions, expectation of discrete random variables. Expectation of Countinous distributions, variance, examples. Joint distribution of random variables

• Lecture 5 (Jan. 29th): Joint distribution of random variables. Independent random variables, covariance and properties. Moment generating function: definition and example.

• Lecture 6 (Jan. 31st): Markov Inequality, Chebyshev Inequality, Laws of large numbers. Central Limit theorem, Explanations and Examples.

• Lecture 7 (Feb. 2nd): Conditional distribution and conditional expectation: defintion and examples.

• Lecture 8 (Feb. 5th): Wald's Equation. Using conditional expectation to find expectations, examples. Conditional Variance: definition.

• Lecture 9 (Feb. 7th): Conditional Variance, VEV formula: proof and example. Properties of conditional expectation, Examples using conditional expectation.

• Lecture 10 (Feb. 9th): Stochastic process, martingale: properties and examples.

• Lecture 11 (Feb. 12th): Further properties of Martingale. Markov Inequality and maximal inequality for martingale and proof.

• Lecture 12 (Feb. 14th): Exponential distribution and its properties. Memoryless of Exponential distribution, Distribution of remaining life time, examples.

• Lecture 13 (Feb. 16th): Distribution of remaining life time, examples. Hazard function and exponential distribution, example.

• Lecture 14 (Feb. 21st): Further properties of Exponential distribution. More examples. Counting Process and properties.

• Lecture 15 (Feb. 23rd): Definition of Poisson process. The equivalence between the two definitions of Poisson process. 

• Lecture 16 (Feb. 26th): Understanding of the Poisson process. Waiting time, Sojourn time. The distribution of the waiting time and its proof.

• Lecture 17 (Feb. 29th): Sojourn time and their distributions. Distribution of number of events during a period of time. Joint distribution of waiting time given N events.

• Lecture 18 (Mar. 1st): Distribution of number of events during a period of time; order statistic, joint distribution of order statistic for uniform random variables.

• Lecture 19 (Mar. 4th): Application of Poisson process. Compound Poisson process.

• Lecture 20 (Mar. 6th): Non-homogeneous Poisson process. Renewal Process and related concepts. 

• Lecture 21 (Mar. 8th): HW#1 problems. Renewal Process and related random variables, Poisson process as a special case of Renewal process.

• Lecture 22 (Mar. 18th): Markov Chain, transition probability matrix, determining distributions over states. Chapman-Kolmogorov Equation, state distribution after n step.

• Lecture 23 (Mar. 20th): Chapman-Kolmogorov Equation, state distribution after n step. Examples of Markov chain.

• Lecture 24 (Mar. 22nd): HW#2 problems. First step analysis: an example.

• Lecture 25 (Mar. 25th): First step analysis: examples. One dimensional Random walk and gambler's ruin.

• Lecture 26 (Mar. 27th): First step analysis for gambler's ruin problem.

• Lecture 27 (Apr. 1st): HW#3 Problems. Branching process: mean and variance of population size.

• Lecture 28 (Apr. 3rd): Regular transition probability matrix, limiting distribution. Classification of states.

• Lecture 29 (Apr. 5th): Communicative and equivalent classes, Period. Recurrent and Transient states, probability of first return time.

• Lecture 30 (Apr. 10th): Rules to determine recurrent and transient. Recurrent and transient in 1D random walk.

• Lecture 31 (Apr. 12th): Recurrent and transient in 2D random walk. Continuous time Markov process, Birth process.

• Lecture 32 (Apr. 15th):  Probability of the population size at any time, the corresponding differential equations and proof. Example of Birth Process.

• Lecture 33 (Apr. 17th): Pure Death process and example. Birth and Death process, definition and explanation. Infinitesimal generator.

•  Lecture 34 (Apr. 19th): Kolmogorov's backward and forward differentia34 equations, their matrix forms. Example.

• Lecture 35 (Apr. 22nd): HW#4 problems. Brief history of Brownian motion. Limiting case of 1D symmetric random walk, definition of Brownian motion, mean and variance of Brownian motion, covariance at two time points.

• Lecture 36 (Apr. 24th): Symmetric property of Brownian motion, Brownian motion and heat equation. Brownian motion as a continuous time martingale. Examples.

• Lecture 37 (Apr. 26th): First hitting time and Maximum value of Brownian motion.

• Lecture 38 (Apr. 29th): zeros of Brownian motion, arcsin law and the Proof. Brownian motion with drift: definition, and probability below a level.   

• Lecture 39 (May 1st): Brownian motion with drift: first hitting time and its probability.