Introduction
Introduction to Natural Language Processing
Natural Language Processing
Natural Language Processing (NLP) - from Tokenization, Embeddings, Transformers to LLMs
This course includes
12 articles
1h 2m reading time
23 videos • 7 hr 14 mins
Course content
12 sections • 93 topics • Read: 1h 2m • Video: 7 hr 14 mins
Introduction
Natural Language Processing
Corpus to Model
Evolution of LLM
Evolution of LLMs | From RNN to Transformer | Natural Language Processing (NLP)
Text Pre-Processing
Text Pre-Processing
Cleaning
Stemming
Lemmatization
Text Pre Processing for LLM | Text Cleaning | Stemming | Lemmatization
Tokenization
Tokenization
Word-Level Tokenization
Character-Level Tokenization
Sub-Word Tokenization
Byte Pair Encoding
WordPiece
SentencePiece
What are Tokens in LLM ? | How tokenization works ? | Byte Pair Encoding | Detailed Explanation
Text Embedding
Text Embedding
One Hot Encoding
Bag of Words
Term Frequency-Inverse Document Frequency (TF-IDF)
Distributed Text Representation
Word2Vec
Skip-Gram
Negative Sampling
How Negative Sampling Works ?
Global Vectors (GloVe)
Comparison: GloVe Vs Word2Vec
FastText
Comparison: FastText Vs GloVe Vs Word2Vec
Word Embeddings (Vector) | OHE | Bag of Words | Term Frequency Inverse Document Frequency (TFIDF)
Word2Vec Embedding | Detailed Explanation with Maths | Skip Gram | CBOW | Negative Sampling
Global Vectors (GloVe) Embedding | Detailed Explanation | Matrix Factorization | Word2Vec Vs GloVe
FastText Embedding for Rare (OOV) Words | Explained with Example | FastText Vs GloVe Vs Word2Vec
RNN
N-Gram Model
Recurrent Neural Network(RNN)
Back Propagation Through Time (BPTT)
Vanishing Gradient Problem
Exploding Gradient Problem
Applications of RNN
RNN Variants
Recurrent Neural Network (RNN) Architecture | Teacher Forcing | Detailed Explanation with Maths
Back Propagation Through Time (BPTT) in RNN | Vanishing Gradient | Exploding Gradient | Explained
Applications Of RNN | Text Generation | Sentiment Analysis | Machine Translation | Bi-RNN
LSTM
RNN Limitation
Long Short Term Memory (LSTM)
Forget Gate
Input Gate & Candidates
Output Gate
Back Propagation Through Time
Constant Error Carousel
Long Short Term Memory (LSTM) | Forget, Input & Output Gate | Constant Error Carousel | Explained
GRU
Issue with LSTM
Gated Recurrent Unit (GRU)
Reset Gate
Candidate Hidden State
Update Gate
Back Propagation Through Time
Constant Error Carousel
How GRU Works?
Gated Recurrent Unit (GRU) | Reset & Update Gate | Constant Error Carousel | Detailed Explanation
Attention
Issue with Encoder-Decoder RNNs
Attention Mechanism
Query, Key & Value
Bahdanau Attention
Dot Product Attention
Attention Mechanism | Query, Key & Value | Bahdanau Attention | Detailed Explanation with Maths
Self Attention
Issue with RNN based Architecture
Attention Is All You Need !
Need For Self-Attention
Self-Attention
Self-Attention Query, Key & Value
Self Attention in Transformer Architecture | Query, Key & Value | Detailed Explanation with Example
Transformer
Transformer
Attention Is All You Need
Self Attention
Limitations of Single Head Self Attention
Multi-Head Attention
Masked Multi-Head Attention
Encoder-Decoder Attention
Feed Forward Network
Residual Connection
Layer Normalization
Positional Encoding
Rotary Positional Embeddings (RoPE)
Attention with Linear Biases (ALiBi)
Multi Head Self Attention in Transformer Architecture | Detailed Explanation
Positional Encoding in Transformers | RoPE | ALiBi | Detailed Explanation with Maths
Transformer Architecture | Attention Is All You Need | Detailed Explanation with Maths
LLM
Transformer Variants
LLM Training Phases
Pre-Training
Transfer Learning
Supervised Fine-Tuning (SFT)
Chain of Thought
Tool/Function Calling
Limitations of SFT
Reinforcement Learning from Human Feedback (RLHF)
Direct Preference Optimization (DPO)
Parameter-Efficient Fine-Tuning (PEFT)
Supervised Fine Tuning (SFT) | Chain of Thought | Tool/Function Calling | Detailed Explanation
Reinforcement Learning From Human Feedback (RLHF) | Direct Preference Optimization (DPO) | Explained
Performance Efficient Fine Tuning (PEFT) | Low Rank Adaptation (LoRA) | Detailed Explanation
How Large Language Models (LLMs) are Trained ? | Pre-Training | Supervised Fine Tuning (SFT) | RLHF
BERT
Bidirectional Encoder Representations from Transformers (BERT)
Masked Language Modeling
Next Sentence Prediction
Special Tokens
Key Applications of BERT
Limitation of BERT
Sentence BERT
Bidirectional Encoder Representations from Transformers (BERT) | Masked Language Modeling |Explained
Sentence BERT (SBERT) | Siamese Network | Pooling Strategy | Detailed Explanation
Text Pre-Processing
Text Pre-Processing - Cleaning, Stemming & Lemmatization
Tokenization
Tokenization - Byte Pair Encoding, WordPiece & SentencePiece
Text Embedding
Text Embeddings - OHE, BoW, TF-IDF, Word2Vec, GloVe & FastText
RNN
Recurrent Neural Network
LSTM
Long Short Term Memory
GRU
Gated Recurrent Unit
Attention
Attention Mechanism (Bahdanau Attention)
Self Attention
Self Attention - Transformer Architecture
Transformer
Transformer Architecture - Attention Is All You Need !
LLM
Large Language Model
BERT
Bidirectional Encoder Representations from Transformers