A chain whose transition probabilities are independent of time n.

A chain whose state space is continuous.

A chain where all states are transient.

It is Transient and Aperiodic.

It is Null Recurrent and Aperiodic.

It is Recurrent and Periodic.

It is Positive Recurrent and Aperiodic.

Recurrent State

Aperiodic State

Absorbing State

Transient State

The transition probability is constant over time.

The conditional distribution of the future state depends only on the present state.

The state space must be continuous.

The distribution of depends on , , and all prior states.

The entire set of states is closed and contains no smaller, proper closed subsets.

All states are transient.

It is periodic.The entire set of states is closed and contains no smaller, proper closed subsets.

It contains at least one absorbing state.

All diagonal elements must be zero.

All row sums must equal 1

The matrix must be symmetric

All column sums must equal 1

125

10

25

5

0.2 steps

5 steps

0.8 steps

4 steps

The exact path the system will take in the next 10 steps.

The probabilities of being in each state after the system has run for a very long time

The probability of a system crashing.

The time taken to reach a specific state for the first time.

The probability of ever returning to j: 

The chain will visit j infinitely often.

There is a non-zero probability that the chain will never return to j: