Support Vector Machines (SVM)
Support Vector Machine (SVM) is a powerful supervised machine learning algorithm used for classification and regression. It is mainly used for classification problems and works effectively in both linear and non-linear datasets.
᪅ Key Idea
SVM finds the best decision boundary (hyperplane) that separates different classes with the maximum margin.
Hyperplane
A hyperplane is a line (2D), plane (3D), or higher-dimensional boundary that separates classes.
᪅ Support Vectors
Support vectors are the data points closest to the hyperplane.
They are critical because they influence the position and direction of the hyperplane.
᪅ Maximum Margin Classifier
SVM tries to maximize the distance between:
- the hyperplane and
- the nearest data points (support vectors)
This improves generalization and reduces overfitting.
᪅ Types of SVM
1. Linear SVM
Used when data is linearly separable.
2. Non-Linear SVM
Used when data is not linearly separable.
It uses the kernel trick to map data into higher dimensions.
᪅ Kernel Functions in SVM
Kernel functions help SVM solve non-linear problems.
Common kernels:
- Linear Kernel
- Polynomial Kernel
- RBF (Radial Basis Function) Kernel
- Sigmoid Kernel
᪅ Advantages
- Works well for high-dimensional data
- Effective for small and medium datasets
- Strong theoretical foundation
- Handles non-linear classification using kernels
ᫌ Disadvantages
- Training is slow for large datasets
- Selecting the right kernel is difficult
- Not easily interpretable
- Sensitive to parameter tuning (C, gamma)
᪅ Applications
- Face recognition
- Text classification (spam detection)
- Medical diagnosis
- Handwriting recognition
- Intrusion detection systems
Conclusion
Support Vector Machines are highly effective classification algorithms that separate data using an optimal hyperplane with maximum margin. With kernel functions, SVM can also handle complex non-linear problems.
