Scribe logoScribeNotes & practice
Sign inStart free
  1. Notes
  2. ›IB Mathematics: Applications and Interpretation
  3. ›4 Statistics and probability
  4. ›4.19 Transition matrices and Markov chains

IB Mathematics: Applications and Interpretation · Topic 4 · Statistics and probability

4.19 Transition matrices and Markov chains

Represent transitions with diagrams and matrices, calculate state probabilities after several steps, find a steady state, and apply Markov chains to real processes.

  • 4 min read
  • 5 parts
  • 10 questions
  • Draft
  • HL only
Start a Quick 5 on this topic→Jump to the 10 questions
1/5A Markov chain moves between states with fixed probabilities
  1. 1A Markov chain moves between states with fixed probabilities
  2. 2A transition matrix stores every transition probability
  3. 3Multiplying by the matrix moves the chain forward one step
  4. 4Over many steps the probabilities settle to a steady state
  5. 5Markov chains model real processes that move between states
  6. Check your recall
  7. Practice: 10 questions
1

Part 1 of 5: A Markov chain moves between states with fixed probabilities

Tomorrow’s weather often depends on today’s. Suppose each day is either sunny or rainy. After a sunny day, the chance of another sunny day is 0.8. After a rainy day, it is 0.4.

This is a Markov chain. Each possible situation is a state. The chance of the next state depends only on the current state, not on earlier days. The probabilities stay the same from step to step.

Once we know today’s weather, yesterday’s adds nothing. Customers switching between shops and people moving between a city and the countryside can be modelled in the same way.

Key terms · tap one for its definition

Markov chain

A Markov chain is a sequence of random steps between states in which the next state depends only on the current state.

state

A state is one of the possible conditions in which a system can be found at a given step.

Part 1 done

2

Part 2 of 5: A transition matrix stores every transition probability

A diagram shows these chances clearly. In a transition diagram, arrows join the states and each arrow carries a probability. The probabilities leaving any state add to 1.

Figure 1. Each arrow shows the probability of moving from one state to the next, and the arrows leaving each state add to 1.

The same information goes into a transition matrix, T\mathbf{T}T. Each column gives the probabilities of moving from one state, and each row gives a state moved to. Rows and columns follow the same order of states. For sunny then rainy, T=(0.80.40.20.6)\mathbf{T}=\begin{pmatrix}0.8&0.4\\0.2&0.6\end{pmatrix}T=(0.80.2​0.40.6​).

Common mistake

Check that every column of T\mathbf{T}T adds to 1 before any calculation. If the rows add to 1 instead, the matrix is the wrong way round.

Key terms · tap one for its definition

transition diagram

A transition diagram is a diagram of states with arrows labelled by the probabilities of moving between them.

transition matrix

A transition matrix T\mathbf{T}T is a square matrix of probabilities; entry TijT_{ij}Tij​ is the chance of moving from state jjj to state iii.

Part 2 done

Quick check

One question on the part you just read

Easy

Identify the valid transition matrix, with columns for the state moved from.
Show answer and explanation

Answer: A · Worked explanation

Every column must add to 1: 0.6 + 0.4 and 0.5 + 0.5 both work. The second matrix has rows that add to 1 but columns that add to 1.1 and 0.9, so it is set up the wrong way round.
3

Part 3 of 5: Multiplying by the matrix moves the chain forward one step

Now suppose today is sunny, and we want the weather over the next few days. The state vector sn\mathbf{s}_nsn​ lists the probability of each state after nnn steps. The starting vector is s0\mathbf{s}_0s0​. The entries of Tn\mathbf{T}^{n}Tn give the probabilities of moving between each pair of states in exactly nnn steps.

One step is sn+1=Tsn\mathbf{s}_{n+1}=\mathbf{T}\mathbf{s}_nsn+1​=Tsn​. After nnn steps, sn=Tns0\mathbf{s}_n=\mathbf{T}^{n}\mathbf{s}_0sn​=Tns0​. Every state vector has entries that add to 1, and a calculator finds matrix powers directly. Multiply in the order Tsn\mathbf{T}\mathbf{s}_nTsn​, with the matrix first.

Worked example

Find a state vector after three days

Use T=(0.80.40.20.6)\mathbf{T}=\begin{pmatrix}0.8&0.4\\0.2&0.6\end{pmatrix}T=(0.80.2​0.40.6​) with sunny first. Today is sunny, so s0=(10)\mathbf{s}_0=\begin{pmatrix}1 \\ 0\end{pmatrix}s0​=(10​). Find the probabilities of sunny and rainy weather after 3 days.
  1. 1
    Use s3=T3s0\mathbf{s}_3=\mathbf{T}^{3}\mathbf{s}_0s3​=T3s0​.
  2. 2
    Calculator: T3=(0.6880.6240.3120.376)\mathbf{T}^{3}=\begin{pmatrix}0.688&0.624\\0.312&0.376\end{pmatrix}T3=(0.6880.312​0.6240.376​).
  3. 3
    Multiply: s3=(0.6880.6240.3120.376)(10)=(0.6880.312)\mathbf{s}_3=\begin{pmatrix}0.688&0.624\\0.312&0.376\end{pmatrix}\begin{pmatrix}1 \\ 0\end{pmatrix}=\begin{pmatrix}0.688 \\ 0.312\end{pmatrix}s3​=(0.6880.312​0.6240.376​)(10​)=(0.6880.312​).

Answer: After 3 days, the probability of sunny weather is 0.688 and of rainy weather is 0.312.

Key term

state vector

A state vector sn\mathbf{s}_nsn​ is a column of probabilities, summing to 1, that gives the chance of each state after nnn steps.

Part 3 done

4

Part 4 of 5: Over many steps the probabilities settle to a steady state

Continue the weather calculation for many days. The probability of sun goes 0.8, 0.72, 0.688, and so on. The changes get smaller each day. This is the long-term behaviour of the chain.

The vector approaches a steady state s\mathbf{s}s, which satisfies Ts=s\mathbf{T}\mathbf{s}=\mathbf{s}Ts=s. Once reached, it no longer changes. This happens when some power of T\mathbf{T}T has only positive entries. Such a chain is a regular Markov chain.

For a regular chain, the steady state does not depend on s0\mathbf{s}_0s0​. A large power Tn\mathbf{T}^{n}Tn has columns that are almost equal to the steady state.

Worked example

Find the steady state

Find the steady state of T=(0.80.40.20.6)\mathbf{T}=\begin{pmatrix}0.8&0.4\\0.2&0.6\end{pmatrix}T=(0.80.2​0.40.6​).
  1. 1
    Let s=(ab)\mathbf{s}=\begin{pmatrix}a \\ b\end{pmatrix}s=(ab​), with a+b=1a+b=1a+b=1.
  2. 2
    From Ts=s\mathbf{T}\mathbf{s}=\mathbf{s}Ts=s: 0.8a+0.4b=a0.8a+0.4b=a0.8a+0.4b=a, so 0.4b=0.2a0.4b=0.2a0.4b=0.2a and a=2ba=2ba=2b.
  3. 3
    Substitute: 2b+b=12b+b=12b+b=1, so b=13b=\frac{1}{3}b=31​ and a=23a=\frac{2}{3}a=32​.

Answer: The steady state is s=(2313)≈(0.6670.333)\mathbf{s}=\begin{pmatrix}\frac{2}{3} \\ \frac{1}{3}\end{pmatrix}\approx\begin{pmatrix}0.667 \\ 0.333\end{pmatrix}s=(32​31​​)≈(0.6670.333​).

Go deeperAbsorbing states

A state that can never be left, with probability 1 of staying, is called absorbing. Chains with such a state are not regular, and their long-term behaviour can depend on where they start.

The equation Ts=s\mathbf{T}\mathbf{s}=\mathbf{s}Ts=s says that s\mathbf{s}s is an eigenvector of T\mathbf{T}T with eigenvalue 1.

Key terms · tap one for its definition

steady state

The steady state is the state vector s\mathbf{s}s satisfying Ts=s\mathbf{T}\mathbf{s}=\mathbf{s}Ts=s, so it does not change from one step to the next.

regular Markov chain

A regular Markov chain is a chain in which some power of the transition matrix has only positive entries.

Part 4 done

Quick check

One question on the part you just read

Easy

State what the next state of a Markov chain depends on.
Show answer and explanation

Answer: A · Worked explanation

In a Markov chain the probabilities for the next step depend only on the present state. Once today’s state is known, earlier states and the starting state add no further information. The number of steps taken does not change the probabilities either, because they stay fixed from step to step.
5

Part 5 of 5: Markov chains model real processes that move between states

The same method works for any process with states and fixed probabilities. Define the states, build the matrix, then use powers or the steady state.

Examples of Markov models

ProcessStatesSteady state shows
WeatherSunny, rainyLong-run share of sunny days
Shop choiceShop A, shop BLong-term share of customers
PopulationCity, countrysideLong-run share living in the city
Process
Weather
States
Sunny, rainy
Steady state shows
Long-run share of sunny days
Process
Shop choice
States
Shop A, shop B
Steady state shows
Long-term share of customers
Process
Population
States
City, countryside
Steady state shows
Long-run share living in the city

Interpret answers in context. For example, “in the long run, about two thirds of days are sunny”. The model assumes the probabilities never change, so real behaviour may differ. The steady state also does not say how quickly the long-run shares are reached.

Exam tip

Define each state clearly and say which way the columns run before building the matrix. Then describe the result in the words of the problem. Check that every answer vector has entries adding to 1.

Note read

Check your recall[14 marks]

6 written questions. Answer each one in your head or on paper, then open the mark scheme.

  1. 1.State[2 marks]

    State two properties that every transition matrix, with columns for the state moved from, must have.

    Show mark schemeHide mark scheme

    Model answer

    Every entry is a probability between 0 and 1, and each column adds to 1.

    • 1 mark for stating that every entry is between 0 and 1
    • 1 mark for stating that each column adds to 1
  2. 2.Write down[2 marks]

    A chain has states A and B. From A, the probability of staying in A is 0.7. From B, the probability of moving to A is 0.1. Write down the transition matrix T\mathbf{T}T, with columns for the state moved from and rows for the state moved to, in the order A, B.

    Show mark schemeHide mark scheme

    Model answer

    T=(0.70.10.30.9)\mathbf{T}=\begin{pmatrix}0.7&0.1\\0.3&0.9\end{pmatrix}T=(0.70.3​0.10.9​).

    • 1 mark for placing 0.7 and 0.1 correctly in the first row
    • 1 mark for placing 0.3 and 0.9 correctly in the second row
  3. 3.Calculate[3 marks]

    The chain starts in state A, so s0=(10)\mathbf{s}_0=\begin{pmatrix}1 \\ 0\end{pmatrix}s0​=(10​), and uses T=(0.70.10.30.9)\mathbf{T}=\begin{pmatrix}0.7&0.1\\0.3&0.9\end{pmatrix}T=(0.70.3​0.10.9​). Calculate s2\mathbf{s}_2s2​.

    Show mark schemeHide mark scheme

    Model answer

    s1=Ts0=(0.70.3)\mathbf{s}_1=\mathbf{T}\mathbf{s}_0=\begin{pmatrix}0.7 \\ 0.3\end{pmatrix}s1​=Ts0​=(0.70.3​). Then s2=Ts1=(0.520.48)\mathbf{s}_2=\mathbf{T}\mathbf{s}_1=\begin{pmatrix}0.52 \\ 0.48\end{pmatrix}s2​=Ts1​=(0.520.48​).

    • 1 mark for using sn+1=Tsn\mathbf{s}_{n+1}=\mathbf{T}\mathbf{s}_nsn+1​=Tsn​
    • 1 mark for finding s1\mathbf{s}_1s1​ correctly
    • 1 mark for finding s2\mathbf{s}_2s2​ correctly
  4. 4.Determine[3 marks]

    Determine the steady state of T=(0.70.10.30.9)\mathbf{T}=\begin{pmatrix}0.7&0.1\\0.3&0.9\end{pmatrix}T=(0.70.3​0.10.9​).

    Show mark schemeHide mark scheme

    Model answer

    Let s=(ab)\mathbf{s}=\begin{pmatrix}a \\ b\end{pmatrix}s=(ab​) with a+b=1a+b=1a+b=1. From 0.7a+0.1b=a0.7a+0.1b=a0.7a+0.1b=a, 0.1b=0.3a0.1b=0.3a0.1b=0.3a, so b=3ab=3ab=3a. Then 4a=14a=14a=1, so a=0.25a=0.25a=0.25 and b=0.75b=0.75b=0.75.

    • 1 mark for setting up Ts=s\mathbf{T}\mathbf{s}=\mathbf{s}Ts=s with a+b=1a+b=1a+b=1
    • 1 mark for obtaining b=3ab=3ab=3a
    • 1 mark for obtaining a=0.25a=0.25a=0.25 and b=0.75b=0.75b=0.75
  5. 5.Explain[2 marks]

    Explain why a regular Markov chain has the same long-term probabilities whatever its starting state.

    Show mark schemeHide mark scheme

    Model answer

    For a regular chain, the state vector approaches the steady state, which satisfies Ts=s\mathbf{T}\mathbf{s}=\mathbf{s}Ts=s. A large power of T\mathbf{T}T has columns almost equal to it, so the start has no effect.

    • 1 mark for stating that the state vector approaches the steady state
    • 1 mark for linking this to the columns of a large power of T\mathbf{T}T being almost equal, so the start has no effect
  6. 6.Discuss[2 marks]

    Discuss one limitation of using a Markov chain to model customers switching between two shops.

    Show mark schemeHide mark scheme

    Model answer

    The model assumes the switching probabilities never change. Real customers may change behaviour, for example after a price change, so long-term predictions may be unreliable.

    • 1 mark for stating an assumption such as fixed probabilities or dependence on the current state only
    • 1 mark for explaining why real behaviour may not match, so predictions are less reliable

Practice this topic

10 quick-fire multiple-choice questions, easy to hard. Tap an answer to see if you're right and the worked explanation. (The written questions with mark schemes are under “Check your recall” above.)

0 right0 of 10 answered

Includes the 2 quick checks in the note above.

3.

Easy·Identify[1 mark]

Identify the equation for the state vector after nnn steps, with T\mathbf{T}T the transition matrix and s0\mathbf{s}_0s0​ the starting vector.
Show answer and explanation

Answer: A · Worked explanation

Each step multiplies the state vector on the left by T\mathbf{T}T, so nnn steps give Tns0\mathbf{T}^{n}\mathbf{s}_0Tns0​. Multiplying T\mathbf{T}T by nnn would only scale the entries and does not repeat the steps. Writing s0Tn\mathbf{s}_0\mathbf{T}^{n}s0​Tn puts the vector on the wrong side for a column vector.
4.

Medium·Calculate[2 marks]

A chain has T=(0.90.30.10.7)\mathbf{T}=\begin{pmatrix}0.9&0.3\\0.1&0.7\end{pmatrix}T=(0.90.1​0.30.7​) and s0=(0.80.2)\mathbf{s}_0=\begin{pmatrix}0.8 \\ 0.2\end{pmatrix}s0​=(0.80.2​). Calculate s1\mathbf{s}_1s1​.
Show answer and explanation

Answer: A · Worked explanation

First entry: 0.9×0.8+0.3×0.2=0.72+0.06=0.780.9\times0.8+0.3\times0.2=0.72+0.06=0.780.9×0.8+0.3×0.2=0.72+0.06=0.78. Second entry: 0.1×0.8+0.7×0.2=0.08+0.14=0.220.1\times0.8+0.7\times0.2=0.08+0.14=0.220.1×0.8+0.7×0.2=0.08+0.14=0.22. The vector with 0.74 and 0.38 comes from multiplying with the matrix the wrong way round, and its entries do not add to 1, which signals an error.
5.

Medium·Determine[2 marks]

Determine the steady state of T=(0.60.20.40.8)\mathbf{T}=\begin{pmatrix}0.6&0.2\\0.4&0.8\end{pmatrix}T=(0.60.4​0.20.8​).
Show answer and explanation

Answer: A · Worked explanation

Solving 0.6a+0.2b=a0.6a+0.2b=a0.6a+0.2b=a gives 0.2b=0.4a0.2b=0.4a0.2b=0.4a, so b=2ab=2ab=2a. With a+b=1a+b=1a+b=1, we get 3a=13a=13a=1, so a=13a=\frac{1}{3}a=31​ and b=23b=\frac{2}{3}b=32​. The vector with 0.6 and 0.4 is just the first column of T\mathbf{T}T, which describes one step from one state, not the steady state.
6.

Medium·Identify[1 mark]

A model of two shops has steady state (0.250.75)\begin{pmatrix}0.25 \\ 0.75\end{pmatrix}(0.250.75​), with shop A first. Identify the correct interpretation.
Show answer and explanation

Answer: A · Worked explanation

The steady state gives the long-term share of each state, so about 25% of customers use shop A in the long run. It says nothing about the first month or the starting shares. It also does not mean that shop A loses 25% of its customers each month, because switching rates come from the matrix entries.
7.

Medium·Calculate[2 marks]

A chain has T=(0.50.30.50.7)\mathbf{T}=\begin{pmatrix}0.5&0.3\\0.5&0.7\end{pmatrix}T=(0.50.5​0.30.7​) and starts in state 1. Calculate the probability of being in state 2 after two steps.
Show answer and explanation

Answer: A · Worked explanation

After one step, s1=(0.5,0.5)T\mathbf{s}_1=(0.5, 0.5)^{\mathsf T}s1​=(0.5,0.5)T. The second entry of s2\mathbf{s}_2s2​ is 0.5×0.5+0.7×0.5=0.25+0.35=0.600.5\times0.5+0.7\times0.5=0.25+0.35=0.600.5×0.5+0.7×0.5=0.25+0.35=0.60. The value 0.50 is the one-step probability, and 0.25 counts only the route that stays in state 1 first and then moves to state 2.
8.

Hard·Identify[2 marks]

Identify the correct statement about the long-term behaviour of a regular Markov chain.
Show answer and explanation

Answer: A · Worked explanation

A regular chain settles to one steady state whatever the start, so the state vector stops changing and the result does not depend on the starting vector. It does not end in a single state with probability 1, which happens only with an absorbing state. The probabilities are equal only for special matrices.
9.

Hard·Calculate[2 marks]

A three-state chain has T=(0.50.2500.50.50.500.250.5)\mathbf{T}=\begin{pmatrix}0.5&0.25&0\\0.5&0.5&0.5\\0&0.25&0.5\end{pmatrix}T=​0.50.50​0.250.50.25​00.50.5​​ and s0=(100)\mathbf{s}_0=\begin{pmatrix}1 \\ 0 \\ 0\end{pmatrix}s0​=​100​​. Calculate the probability of being in state 3 after two steps.
Show answer and explanation

Answer: A · Worked explanation

After one step, s1=(0.5,0.5,0)T\mathbf{s}_1=(0.5, 0.5, 0)^{\mathsf T}s1​=(0.5,0.5,0)T. The third entry of s2\mathbf{s}_2s2​ is 0×0.5+0.25×0.5+0.5×0=0.1250\times0.5+0.25\times0.5+0.5\times0=0.1250×0.5+0.25×0.5+0.5×0=0.125. The answer 0 wrongly assumes that a chain with no direct move from state 1 to state 3 cannot reach it in two steps. Compute the vector one step at a time rather than guessing from the matrix.
10.

Hard·Determine[2 marks]

A transition matrix is T=(0.7x0.3y)\mathbf{T}=\begin{pmatrix}0.7&x\\0.3&y\end{pmatrix}T=(0.70.3​xy​), and its steady state is (0.50.5)\begin{pmatrix}0.5 \\ 0.5\end{pmatrix}(0.50.5​). Determine xxx.
Show answer and explanation

Answer: A · Worked explanation

From Ts=s\mathbf{T}\mathbf{s}=\mathbf{s}Ts=s, the first row gives 0.7×0.5+x×0.5=0.50.7\times0.5+x\times0.5=0.50.7×0.5+x×0.5=0.5, so 0.35+0.5x=0.50.35+0.5x=0.50.35+0.5x=0.5 and x=0.3x=0.3x=0.3. The value 0.15 comes from subtracting 0.35 from 0.5 but forgetting to divide by 0.5. The value 0.7 is the entry already in the first column. Check the answer by confirming that each column of the matrix adds to 1.

Keep going

Next topic5.1 Limits and the derivativePrevious topic4.18 Hypothesis testing

More in Topic 4 · Statistics and probability

  1. 4.11Chi-squared and t-testsNotes · 10 questions
  2. 4.12Designing data collectionNotes · 10 questions· HL only
  3. 4.13Non-linear regressionNotes · 10 questions· HL only
  4. 4.14Linear combinations of random variablesNotes · 10 questions· HL only
  5. 4.15Central limit theoremNotes · 10 questions· HL only
  6. 4.16Confidence intervalsNotes · 10 questions· HL only
  7. 4.17Poisson distributionNotes · 10 questions· HL only
  8. 4.18Hypothesis testingNotes · 10 questions· HL only
  9. 4.19Transition matrices and Markov chainsNotes · 10 questions· HL only

All IB Mathematics: Applications and Interpretation topics →

Draft: written by Scribe and checked automatically, not yet by a subject reviewer.

Practice questions are original, written by Scribe and checked automatically for correctness.

Topic codes follow the official numbering. Not endorsed by IB Diploma Programme or any exam board.

On this page

0/5 read

  1. 1A Markov chain moves between states with fixed probabilities
  2. 2A transition matrix stores every transition probability
  3. 3Multiplying by the matrix moves the chain forward one step
  4. 4Over many steps the probabilities settle to a steady state
  5. 5Markov chains model real processes that move between states
  6. Check your recall
  7. Practice: 10 questions
All coursesScribeHow it worksPricingPrivacyTerms

Notes and questions are written by Scribe in our own words. Topic codes follow the official course numbering so you can find your place. Not produced or endorsed by the IB, College Board, AQA, OCR, Pearson Edexcel or any exam board.

© 2026 Scribe