1. Automated Planning
Planning finds action sequences to achieve goals. STRIPS representations define states, actions (preconditions + effects), and goals. Partial-order planning handles concurrent actions.
2. Natural Language Processing
NLP enables machines to understand and generate human language. Pipeline stages: tokenization, stemming/lemmatization, part-of-speech tagging, parsing, named entity recognition, sentiment analysis.
Word embeddings (Word2Vec, GloVe) map words to dense vectors capturing semantic similarity. Modern systems use contextual embeddings from transformers.
3. Computer Vision
Vision tasks: classification, object detection, segmentation, pose estimation. Traditional pipeline: SIFT/HOG features + SVM. Deep learning (CNNs) now dominates — see our ML Advanced course.
4. Reinforcement Learning & MDPs
An agent learns by interacting with an environment. A Markov Decision Process is defined by \((S, A, P, R, \gamma)\): states, actions, transition probabilities, rewards, discount factor.
Q-learning learns action-values without a model of the environment. Policy gradient methods (PPO, SAC) optimize policies directly for continuous control.