M171, University of California, Los Angeles, Spring 2021
Abstract: A stochastic process is a sequence of random variables, evolving over time according to some underlying mechanism that is designed to capture some natural phenomena. We will study some essential stochastic processes such as Markov chains, Poisson Processes, Renewal Processes, Martingales, and Brownian Motion.
Textbook: Rick Durrett, Essentials of Stochastic Processes, 2nd edition
TAs: Haiyu Huang and Daniel Raban
Homework and exams:
Homework #1 (04/10)
Homework #2 (04/17)
Homework #3 (04/24)
Midterm (05/03)
Homework #4 (05/08)
Homework #5 (05/15)
Homework #6 (05/22)
Homework #7 (05/29)
Final exam (06/08)
Table of contents:
I Markov chains
1 Definition and examples
2 Multistep transition probability
3 Classification of states
4 Stationary distributions
5 Convergence theorem
6 Exit times and exit distributions
7 Markov chains with infinite state space
II Martingales
1 Conditional expectations
2 Martingales: definition and properties
3 Gambling strategies and stopping times
4 Applications of martingales and stopping times
5 Convergence of martingales
III Poisson process
1 Reminder on exponential distribution
2 Poisson process: definition as an arrival process
3 Poisson process: memoryless property and alternative characterization
4 Transformations of Poisson processes
5 Compound Poisson processes
IV Renewal processes
1 Defintion and laws of large numbers
2 Applications to queuing theory: GI/G/1 and M/G/1
V Continuous-time Markov chains
1 Definitions and properties
2 Limiting behavior
3 Exit times and exit distributions
4 Markovian queues
VI Brownian motion
Schedule:
Course #1 (03/29, 12pm-1pm): Introduction
Course #2 (03/31, 12pm-1pm): Sections I.1-I.2
Course #3 (04/02, 12pm-1pm): Sections I.3
Course #4 (04/05, 12pm-1pm): Section I.3
Course #5 (04/07, 12pm-1pm): Section I.3-I.4
Course #6 (04/09, 12pm-1pm): Section I.4
Course #7 (04/12, 12pm-1pm): Section I.5
Course #8 (04/14, 12pm-1pm): Section I.5
Course #9 (04/16, 12pm-1pm): Section I.5-I.6
Course #10 (04/19, 12pm-1pm): Section I.6-I.7
Course #11 (04/21, 12pm-1pm): Section I.7-II.1
Course #12 (04/23, 12pm-1pm): Section II.1-II.2
Course #13 (04/26, 12pm-1pm): Section II.2-II.3
Course #14 (04/28, 12pm-1pm): Section II.4
Course #15 (04/30, 12pm-1pm): Section II.4
Course #17 (05/05, 12pm-1pm): Section II.5-III.1
Course #18 (05/07, 12pm-1pm): Section III.1-III.2
Course #19 (05/10, 12pm-1pm): Section III.3
Course #20 (05/12, 12pm-1pm): Section III.4
Course #21 (05/14, 12pm-1pm): Section III.5-IV.1
Course #22 (05/17, 12pm-1pm): Section IV.1-IV.2
Course #23 (05/19, 12pm-1pm): Section IV.2
Course #24 (05/21, 12pm-1pm): Section IV.2-V.1
Course #25 (05/26, 12pm-1pm): Section V.1
Course #26 (05/28, 12pm-1pm): Section V.2-V.4
Course #27 (06/02, 12pm-1pm): Section VI
Course #28 (06/04, 12pm-1pm): Section VI