Machine Learning

Fundamentals

Supervised vs Unsupervised Learning - The basic paradigm distinction
Regression vs Classification - Two main supervised learning tasks
Cost Function - How we measure model performance
Gradient Descent - Core optimization algorithm

Feature Engineering & Preprocessing

Feature Engineering - Creating informative features from raw data
Feature Scaling - Normalizing features for better learning
Vectorization - Efficient computation through matrix operations
Polynomial Regression - Adding polynomial features for non-linear relationships

Model Performance & Diagnostics

Bias vs Variance - The fundamental tradeoff in model complexity
Learning Curves - Diagnostic tool for bias/variance problems
Human Level Performance - Benchmarking against human capabilities
Error Analysis - Systematic approach to understanding failures

Data Management

Development Set vs Test Set - Proper data splitting for evaluation
Single Number Evaluation Metric - Clear metrics for model comparison
Data Distribution Mismatch - When training and deployment data differ

Regularization & Overfitting

Regularization - Techniques to prevent overfitting

Advanced Topics

End-to-End Deep Learning - Single networks vs multi-stage pipelines