Recorded Lectures: DTMC

In the following you will find an introduction to DTMCs

1. Definition of DTMC
   
2. Examples
   
3. Kolmogorov equations
   
4. Absorption probabilities
   
5. Communicating classes and irreducibility
   
6. Positive recurrence and aperiodicity
   
7. invariant measures and steady state
   
8. computation of the steady state
   
9. steady state for general DTMC
   
10. simulation of DTMC