Machine Learning Algorithms
A practical map of the most important algorithms: what each is good at, where it struggles, and when to reach for it.
By the end of this lesson you will be able to
- recognise the main algorithms in each learning type
- pick a sensible starting algorithm for a real problem
- know which algorithms appear most often in ML interviews
There are hundreds of ML algorithms, but you don't need to memorise them all. Start with the pictures below: most algorithms are variations on a handful of simple ideas. Then use the tables as a reference you come back to.
Eight core algorithms in pictures
How to read the tables below
Regression predicts a number (a price, a temperature). Classification predicts a category (spam or not spam). Both means the algorithm has versions for either. The stars show how often the algorithm comes up in interviews.
1. Supervised learning algorithms
| Algorithm | Problem type | Used for | Advantages | Disadvantages | Best used when | Interview |
|---|---|---|---|---|---|---|
| Linear Regression | Regression | Predict continuous values | Simple, interpretable, fast | Assumes a linear relationship | House prices, sales forecasting | ★★★★★ |
| Multiple Linear Regression | Regression | Several input variables | Easy to understand | Sensitive to multicollinearity | Salary prediction | ★★★★ |
| Polynomial Regression | Regression | Curved relationships | Fits curves well | Easily overfits | Growth trends | ★★★ |
| Logistic Regression | Classification | Binary classification | Fast, interpretable | Only a linear decision boundary | Spam detection, disease prediction | ★★★★★ |
| Decision Tree | Both | Classification and regression | Easy to explain | Overfits | Medical diagnosis | ★★★★★ |
| Random Forest | Both | Ensemble of trees | High accuracy, less overfitting | Slower, less interpretable | Banking, fraud detection | ★★★★★ |
| Extra Trees | Both | More randomised forest | Faster, lower variance | Harder to interpret | Large datasets | ★★★ |
| Support Vector Machine | Classification | High-dimensional data | Works well with small datasets | Slow on large datasets | Face recognition | ★★★★ |
| Support Vector Regression | Regression | Continuous prediction | Handles non-linearity | Computationally expensive | Financial forecasting | ★★★ |
| K-Nearest Neighbours | Both | Classification and regression | Very simple | Slow on large datasets | Recommendation systems | ★★★★ |
| Naive Bayes | Classification | Probability-based classification | Very fast | Assumes features are independent | Email spam filtering | ★★★★ |
| Gaussian Naive Bayes | Classification | Continuous features | Extremely fast | Sometimes lower accuracy | Medical diagnosis | ★★★ |
| Bernoulli Naive Bayes | Classification | Binary (yes/no) features | Good for text | Binary inputs only | Document classification | ★★ |
| Multinomial Naive Bayes | Classification | Count-based features | Excellent for NLP | Assumes independence | Sentiment analysis | ★★★★ |
| Gradient Boosting | Both | Ensemble boosting | High accuracy | Slow training | Credit scoring | ★★★★ |
| AdaBoost | Classification | Boosting weak learners | Simple boosting | Sensitive to noise | Fraud detection | ★★★ |
| XGBoost | Both | Gradient boosting | Extremely accurate | Many hyperparameters | Kaggle competitions | ★★★★★ |
| LightGBM | Both | Gradient boosting | Very fast | Can overfit small datasets | Large datasets | ★★★★ |
| CatBoost | Both | Categorical features | Handles categories automatically | Slower than LightGBM | Business datasets | ★★★★ |
| Neural Network (MLP) | Both | Complex patterns | Powerful | Needs more data | General prediction | ★★★★ |
| Deep Neural Network | Both | Large-scale prediction | Learns complex relationships | Computationally expensive | Images, speech | ★★★★ |
Bagging vs boosting, in one line each
Random forest (bagging): ask 100 independent experts and take a vote. Gradient boosting / XGBoost (boosting): each new expert studies the mistakes of the previous ones and fixes them.
2. Unsupervised learning algorithms
| Algorithm | Purpose | Advantages | Disadvantages | Best used for | Interview |
|---|---|---|---|---|---|
| K-Means | Clustering | Fast, simple | You must choose K | Customer segmentation | ★★★★★ |
| Hierarchical Clustering | Clustering | No K needed at the start | Slow | Gene analysis | ★★★★ |
| DBSCAN | Density clustering | Any cluster shape, flags outliers | Sensitive to parameters | Anomaly detection | ★★★★ |
| OPTICS | Density clustering | Handles varying densities | More complex | Spatial data | ★★★ |
| Mean Shift | Clustering | No K needed | Slow | Image segmentation | ★★★ |
| Gaussian Mixture Model | Probabilistic clustering | Soft (probabilistic) clusters | Computationally intensive | Customer behaviour | ★★★ |
| Spectral Clustering | Graph-based clustering | Handles complex shapes | Slow | Image processing | ★★★ |
| Affinity Propagation | Clustering | No K needed | High memory use | Recommendation systems | ★★ |
| PCA | Dimensionality reduction | Fast, reduces features | Linear only | Data visualisation | ★★★★★ |
| Kernel PCA | Non-linear reduction | Captures non-linear structure | More expensive | Image recognition | ★★★ |
| t-SNE | Visualisation | Excellent 2-D pictures of data | Slow, doesn't scale | Data exploration | ★★★★ |
| UMAP | Visualisation | Faster than t-SNE | Sensitive to parameters | Large datasets | ★★★ |
| ICA | Feature extraction | Separates mixed signals | Sensitive to noise | Signal processing | ★★★ |
| Autoencoder | Feature learning | Learns compressed representations | Needs neural networks | Anomaly detection | ★★★★ |
| Apriori | Association rules | Easy to understand | Slow on large data | Market basket analysis | ★★★★ |
| FP-Growth | Association rules | Faster than Apriori | More complex | Retail analytics | ★★★★ |
3. Semi-supervised learning algorithms
| Algorithm | Idea | Advantages | Disadvantages | Best used for |
|---|---|---|---|---|
| Self-Training | The model labels unlabelled data itself | Simple | Early errors can spread | Image classification |
| Label Propagation | Spreads labels through a similarity graph | Works with very few labels | Depends on graph quality | Social networks |
| Label Spreading | A more robust label propagation | Better robustness | Computationally expensive | Document classification |
| Co-Training | Two models teach each other | Improves accuracy | Needs two independent feature sets | Web page classification |
| Semi-Supervised SVM | SVM that also uses unlabelled data | High accuracy | Slow training | Medical imaging |
| Pseudo-Labelling | Confident predictions become labels | Easy to implement | Can reinforce mistakes | Deep learning pipelines |
| Mean Teacher | A teacher model guides a student model | Strong performance | Complex training | Image recognition |
| FixMatch | Pseudo-labelling + data augmentation | Excellent results | Depends on augmentation | Computer vision |
4. Reinforcement learning algorithms
| Algorithm | Type | Advantages | Disadvantages | Best used for |
|---|---|---|---|---|
| Q-Learning | Value-based | Simple | Doesn't scale to big problems | Games |
| Deep Q-Network (DQN) | Deep RL | Handles huge state spaces | Needs a lot of compute | Atari games |
| SARSA | On-policy | Stable learning | Slower to converge | Robotics |
| Monte Carlo | Model-free | Simple concept | Slow learning | Episodic tasks |
| Temporal Difference | Model-free | Efficient | Can be unstable | Sequential decisions |
| Policy Gradient | Policy-based | Continuous actions | High variance | Robotics |
| REINFORCE | Policy-based | Easy to implement | Slow convergence | Simple RL tasks |
| Actor-Critic | Hybrid | Stable and efficient | More complex | Autonomous driving |
| PPO | Policy optimisation | Stable, very widely used | Needs hyperparameter tuning | Robotics, games, LLM fine-tuning |
| TRPO | Policy optimisation | Strong theoretical guarantees | Computationally expensive | Research |
| A3C | Asynchronous RL | Faster training | Complex implementation | Game AI |
| DDPG | Continuous control | Handles continuous actions | Sensitive to hyperparameters | Industrial robots |
| TD3 | Improved DDPG | Better stability | More complex | Robotics |
| SAC (Soft Actor-Critic) | Off-policy | Sample-efficient | Computationally intensive | Autonomous systems |
5. Deep learning algorithms
These are covered in depth in the Deep Learning, Computer Vision, NLP, Transformer, GAN and VAE lessons later in the course.
| Algorithm | Used in | Best for | Advantages | Disadvantages |
|---|---|---|---|---|
| Artificial Neural Network | General ML | Tabular data | Flexible | Needs more data |
| CNN | Computer vision | Image classification | Excellent for images | High compute cost |
| RNN | Sequence data | Text, speech | Captures order in time | Vanishing gradients |
| LSTM | NLP, time series | Long sequences | Remembers long-term context | Slower training |
| GRU | NLP | Faster than LSTM | Simpler architecture | Slightly less expressive |
| Transformer | NLP and more | Language understanding | Parallel processing | Resource-intensive |
| Encoder-Decoder | Translation | Machine translation | Flexible | Training complexity |
| BERT | NLP | Text understanding | Context-aware | Large model |
| GPT | Generative AI | Text generation | Excellent language generation | Expensive to train |
| GAN | Image generation | Synthetic data | Very realistic outputs | Difficult to train |
| VAE | Data generation | Image synthesis | Stable training | Blurry outputs |
| Diffusion models | Generative AI | Image generation | High-quality images | Slow generation |
Which algorithm should I use?
A good starting point for common problems. In practice, you try two or three and compare them on held-out data.
| If your problem is… | Start with |
|---|---|
| Predicting house prices | Linear Regression, Random Forest Regressor, XGBoost |
| Predicting customer churn | Logistic Regression, Random Forest, XGBoost |
| Detecting spam emails | Naive Bayes, Logistic Regression |
| Classifying images | CNN, Vision Transformer |
| Translating languages | Transformer, Encoder-Decoder |
| Building a chatbot | Transformer, GPT-style LLM |
| Recommending products | KNN, Matrix Factorisation, Neural Collaborative Filtering |
| Segmenting customers | K-Means, DBSCAN, Hierarchical Clustering |
| Detecting fraud | Isolation Forest, One-Class SVM, Random Forest, XGBoost |
| Reducing the number of features | PCA, UMAP, t-SNE |
| Analysing shopping baskets | Apriori, FP-Growth |
| Forecasting stock prices | LSTM, GRU, Transformer (with caution: financial forecasting is inherently hard) |
| Controlling robots or game agents | PPO, DQN, Actor-Critic, SAC |
A professional tip
For tabular data (rows and columns, like a spreadsheet), gradient-boosted trees such as XGBoost, LightGBM and CatBoost still beat deep learning most of the time. Deep learning shines on unstructured data: images, text, audio and video.
The interview shortlist
If you are preparing for ML interviews, master these first. Together they cover most concepts expected at entry and intermediate level:
Classical ML
Linear Regression, Logistic Regression, Decision Tree, Random Forest, SVM, KNN, Naive Bayes, Gradient Boosting, XGBoost
Unsupervised
K-Means, DBSCAN, PCA
Deep learning
ANN, CNN, RNN, LSTM and GRU, the Transformer
Reinforcement
Q-Learning, Deep Q-Networks (DQN)
Key takeaways
- Most algorithms are variations of a few ideas: fit a line, ask yes/no questions, vote, find nearest neighbours, find groups.
- Start simple (linear/logistic regression, a tree), then try ensembles like Random Forest and XGBoost.
- Tabular data → boosted trees; images, text and audio → deep learning.
Check your understanding
1. You need to predict whether a patient has a disease (yes or no) and explain the result to doctors. A good first choice is…
It's a supervised classification problem where interpretability matters. K-Means and PCA are unsupervised.
2. What is the main weakness of K-Means?
K-Means asks you for K up front. DBSCAN and hierarchical clustering avoid this.
3. How does a Random Forest reduce overfitting compared with a single Decision Tree?
Individual trees overfit in different ways; averaging many of them cancels much of that out.