**Stochastic processes**

M171, University of California, Los Angeles, Spring 2021

**Abstract:**A stochastic process is a sequence of random variables, evolving over time according to some underlying mechanism that is designed to capture some natural phenomena. We will study some essential stochastic processes such as Markov chains, Poisson Processes, Renewal Processes, Martingales, and Brownian Motion.

**Textbook:**Rick Durrett,

*Essentials of Stochastic Processes*, 2nd edition

**TAs:**Haiyu Huang and Daniel Raban

**Homework and exams:**

Homework #1 (04/10)

Homework #2 (04/17)

Homework #3 (04/24)

Midterm (05/03)

Homework #4 (05/08)

Homework #5 (05/15)

Homework #6 (05/22)

Homework #7 (05/29)

Final exam (06/08)

**Table of contents:**

I

*Markov chains*

1 Definition and examples

2 Multistep transition probability

3 Classification of states

4 Stationary distributions

5 Convergence theorem

6 Exit times and exit distributions

7 Markov chains with infinite state space

II

*Martingales*

1 Conditional expectations

2 Martingales: definition and properties

3 Gambling strategies and stopping times

4 Applications of martingales and stopping times

5 Convergence of martingales

III

*Poisson process*

1 Reminder on exponential distribution

2 Poisson process: definition as an arrival process

3 Poisson process: memoryless property and alternative characterization

4 Transformations of Poisson processes

5 Compound Poisson processes

IV

*Renewal processes*

1 Defintion and laws of large numbers

2 Applications to queuing theory: GI/G/1 and M/G/1

V

*Continuous-time Markov chains*

1 Definitions and properties

2 Limiting behavior

3 Exit times and exit distributions

4 Markovian queues

VI

*Brownian motion*

**Schedule:**

Course #1 (03/29, 12pm-1pm): Introduction

Course #2 (03/31, 12pm-1pm): Sections I.1-I.2

Course #3 (04/02, 12pm-1pm): Sections I.3

Course #4 (04/05, 12pm-1pm): Section I.3

Course #5 (04/07, 12pm-1pm): Section I.3-I.4

Course #6 (04/09, 12pm-1pm): Section I.4

Course #7 (04/12, 12pm-1pm): Section I.5

Course #8 (04/14, 12pm-1pm): Section I.5

Course #9 (04/16, 12pm-1pm): Section I.5-I.6

Course #10 (04/19, 12pm-1pm): Section I.6-I.7

Course #11 (04/21, 12pm-1pm): Section I.7-II.1

Course #12 (04/23, 12pm-1pm): Section II.1-II.2

Course #13 (04/26, 12pm-1pm): Section II.2-II.3

Course #14 (04/28, 12pm-1pm): Section II.4

Course #15 (04/30, 12pm-1pm): Section II.4

Course #17 (05/05, 12pm-1pm): Section II.5-III.1

Course #18 (05/07, 12pm-1pm): Section III.1-III.2

Course #19 (05/10, 12pm-1pm): Section III.3

Course #20 (05/12, 12pm-1pm): Section III.4

Course #21 (05/14, 12pm-1pm): Section III.5-IV.1

Course #22 (05/17, 12pm-1pm): Section IV.1-IV.2

Course #23 (05/19, 12pm-1pm): Section IV.2

Course #24 (05/21, 12pm-1pm): Section IV.2-V.1

Course #25 (05/26, 12pm-1pm): Section V.1

Course #26 (05/28, 12pm-1pm): Section V.2-V.4

Course #27 (06/02, 12pm-1pm): Section VI

Course #28 (06/04, 12pm-1pm): Section VI