A Markov Model is a probabilistic model used to represent systems that change over time, where the future state depends only on the current state and not on the past states.
This property is called the Markov Property.
Markov Property
P(Xt+1ᯣXt,Xtᯒ1,…,X0)=P(Xt+1ᯣXt)P(X_{t+1} | X_t, X_{t-1}, …, X_0) = P(X_{t+1} | X_t)
Meaning:
퇽혉 The next state depends only on the present state.
Components of a Markov Model
A Markov Model consists of:
1. States
Possible conditions of the system.
Example: Weather = {Sunny, Rainy, Cloudy}
2. Transition Probabilities
Probability of moving from one state to another.
Example:
- P(Rainy ᭒ Sunny)
- P(Sunny ᭒ Cloudy)
3. Transition Probability Matrix
A matrix representing all transition probabilities.
Example:
| From \ To | Sunny | Rainy | Cloudy |
|---|---|---|---|
| Sunny | 0.6 | 0.3 | 0.1 |
| Rainy | 0.4 | 0.4 | 0.2 |
| Cloudy | 0.5 | 0.2 | 0.3 |
Types of Markov Models
1. Markov Chain
- States are directly observable.
- Example: weather prediction.
2. Hidden Markov Model (HMM)
- States are hidden and not directly observable.
- Only outputs (observations) are visible.
- Example: speech recognition.
Applications of Markov Models
- Weather forecasting
- Stock market prediction
- Speech recognition (HMM)
- Natural language processing
- DNA sequence analysis
- Queueing systems
- Robotics and navigation
Advantages
- Simple and mathematically strong
- Useful for sequential and time-based data
- Efficient for modeling dynamic systems
ℌ Disadvantages
- Assumption of memoryless property may not fit all real-world problems
- Complex for large state spaces
- Limited if past history influences future strongly
