IB Mathematics: Applications and Interpretation · Topic 4 · Statistics and probability
4.19 Transition matrices and Markov chains
Represent transitions with diagrams and matrices, calculate state probabilities after several steps, find a steady state, and apply Markov chains to real processes.
1/5A Markov chain moves between states with fixed probabilities
- A Markov chain moves between states with fixed probabilities
- A transition matrix stores every transition probability
- Multiplying by the matrix moves the chain forward one step
- Over many steps the probabilities settle to a steady state
- Markov chains model real processes that move between states
- Check your recall
- Practice: 10 questions
Part 1 of 5: A Markov chain moves between states with fixed probabilities
Tomorrow’s weather often depends on today’s. Suppose each day is either sunny or rainy. After a sunny day, the chance of another sunny day is 0.8. After a rainy day, it is 0.4.
This is a Markov chain. Each possible situation is a state. The chance of the next state depends only on the current state, not on earlier days. The probabilities stay the same from step to step.
Once we know today’s weather, yesterday’s adds nothing. Customers switching between shops and people moving between a city and the countryside can be modelled in the same way.
Key terms · tap one for its definition
Markov chain
A Markov chain is a sequence of random steps between states in which the next state depends only on the current state.
state
A state is one of the possible conditions in which a system can be found at a given step.
Part 2 of 5: A transition matrix stores every transition probability
A diagram shows these chances clearly. In a transition diagram, arrows join the states and each arrow carries a probability. The probabilities leaving any state add to 1.
The same information goes into a transition matrix, . Each column gives the probabilities of moving from one state, and each row gives a state moved to. Rows and columns follow the same order of states. For sunny then rainy, .
Key terms · tap one for its definition
transition diagram
A transition diagram is a diagram of states with arrows labelled by the probabilities of moving between them.
transition matrix
A transition matrix is a square matrix of probabilities; entry is the chance of moving from state to state .

Quick check
One question on the part you just read
Easy
Show answer and explanation
Answer: A · Worked explanation
Part 3 of 5: Multiplying by the matrix moves the chain forward one step
Now suppose today is sunny, and we want the weather over the next few days. The state vector lists the probability of each state after steps. The starting vector is . The entries of give the probabilities of moving between each pair of states in exactly steps.
One step is . After steps, . Every state vector has entries that add to 1, and a calculator finds matrix powers directly. Multiply in the order , with the matrix first.
Worked example
Find a state vector after three days
- Use .
- Calculator: .
- Multiply: .
Answer: After 3 days, the probability of sunny weather is 0.688 and of rainy weather is 0.312.
Key term
state vector
A state vector is a column of probabilities, summing to 1, that gives the chance of each state after steps.
Part 4 of 5: Over many steps the probabilities settle to a steady state
Continue the weather calculation for many days. The probability of sun goes 0.8, 0.72, 0.688, and so on. The changes get smaller each day. This is the long-term behaviour of the chain.
The vector approaches a steady state , which satisfies . Once reached, it no longer changes. This happens when some power of has only positive entries. Such a chain is a regular Markov chain.
For a regular chain, the steady state does not depend on . A large power has columns that are almost equal to the steady state.
Worked example
Find the steady state
- Let , with .
- From : , so and .
- Substitute: , so and .
Answer: The steady state is .
Key terms · tap one for its definition
steady state
The steady state is the state vector satisfying , so it does not change from one step to the next.
regular Markov chain
A regular Markov chain is a chain in which some power of the transition matrix has only positive entries.

Quick check
One question on the part you just read
Easy
Show answer and explanation
Answer: A · Worked explanation
Part 5 of 5: Markov chains model real processes that move between states
The same method works for any process with states and fixed probabilities. Define the states, build the matrix, then use powers or the steady state.
Examples of Markov models
- Process
- Weather
- States
- Sunny, rainy
- Steady state shows
- Long-run share of sunny days
- Process
- Shop choice
- States
- Shop A, shop B
- Steady state shows
- Long-term share of customers
- Process
- Population
- States
- City, countryside
- Steady state shows
- Long-run share living in the city
Interpret answers in context. For example, “in the long run, about two thirds of days are sunny”. The model assumes the probabilities never change, so real behaviour may differ. The steady state also does not say how quickly the long-run shares are reached.
Check your recall[14 marks]
6 written questions. Answer each one in your head or on paper, then open the mark scheme.
- 1.State[2 marks]
State two properties that every transition matrix, with columns for the state moved from, must have.
Show mark schemeHide mark scheme
Model answer
Every entry is a probability between 0 and 1, and each column adds to 1.
- 1 mark for stating that every entry is between 0 and 1
- 1 mark for stating that each column adds to 1
- 2.Write down[2 marks]
A chain has states A and B. From A, the probability of staying in A is 0.7. From B, the probability of moving to A is 0.1. Write down the transition matrix , with columns for the state moved from and rows for the state moved to, in the order A, B.
Show mark schemeHide mark scheme
Model answer
.
- 1 mark for placing 0.7 and 0.1 correctly in the first row
- 1 mark for placing 0.3 and 0.9 correctly in the second row
- 3.Calculate[3 marks]
The chain starts in state A, so , and uses . Calculate .
Show mark schemeHide mark scheme
Model answer
. Then .
- 1 mark for using
- 1 mark for finding correctly
- 1 mark for finding correctly
- 4.Determine[3 marks]
Determine the steady state of .
Show mark schemeHide mark scheme
Model answer
Let with . From , , so . Then , so and .
- 1 mark for setting up with
- 1 mark for obtaining
- 1 mark for obtaining and
- 5.Explain[2 marks]
Explain why a regular Markov chain has the same long-term probabilities whatever its starting state.
Show mark schemeHide mark scheme
Model answer
For a regular chain, the state vector approaches the steady state, which satisfies . A large power of has columns almost equal to it, so the start has no effect.
- 1 mark for stating that the state vector approaches the steady state
- 1 mark for linking this to the columns of a large power of being almost equal, so the start has no effect
- 6.Discuss[2 marks]
Discuss one limitation of using a Markov chain to model customers switching between two shops.
Show mark schemeHide mark scheme
Model answer
The model assumes the switching probabilities never change. Real customers may change behaviour, for example after a price change, so long-term predictions may be unreliable.
- 1 mark for stating an assumption such as fixed probabilities or dependence on the current state only
- 1 mark for explaining why real behaviour may not match, so predictions are less reliable

Practice this topic
10 quick-fire multiple-choice questions, easy to hard. Tap an answer to see if you're right and the worked explanation. (The written questions with mark schemes are under “Check your recall” above.)
0 right0 of 10 answered
Includes the 2 quick checks in the note above.
EasyIdentify[1 mark]
Show answer and explanation
Answer: A · Worked explanation
MediumCalculate[2 marks]
Show answer and explanation
Answer: A · Worked explanation
MediumDetermine[2 marks]
Show answer and explanation
Answer: A · Worked explanation
MediumIdentify[1 mark]
Show answer and explanation
Answer: A · Worked explanation
MediumCalculate[2 marks]
Show answer and explanation
Answer: A · Worked explanation
HardIdentify[2 marks]
Show answer and explanation
Answer: A · Worked explanation
HardCalculate[2 marks]
Show answer and explanation
Answer: A · Worked explanation
HardDetermine[2 marks]
Show answer and explanation
Answer: A · Worked explanation