Nearby lessons
23 of 30What is Attention?
Concept in One Line
Attention is a mechanism that helps the model focus on the most important words in a sentence while generating the output.
Why This Concept Is Important
- Solves memory problems of old models.
- Helps understand long sentences.
- Core reason why Transformers work well.
Key Points to Remember
- Attention looks at all words together.
- It decides which words are important.
- Important words get more weight.
- Helps understand long-distance relationships.
- Works in parallel (fast).
Problem Without Attention
Old models read word by word.
Early information gets forgotten.
Long sentence understanding fails.
Attention fixes this.
How Attention Works
Sentence: Durga teaches AI because he loves technology.
While predicting loves, attention focuses more on:
- Durga
- teaches
- AI
Less focus on:
- because
- he
The model understands who is doing what.
Simple Daily Life Example
Teacher asking a question.
Student listens carefully to important words only.
Attention = selective focus.
Simple Flow
All Words -> Importance Scores -> Focus on Important Words -> Better Understanding
Interview / Exam Points
Q1: What is attention in AI?
Attention is a mechanism that allows the model to focus on important parts of the input while generating output.
Q2: Why is attention better than RNN memory?
Attention looks at all words at once instead of passing memory step by step.
Common Confusions to Avoid
- Attention is not human attention.
- Attention does not mean awareness.
- Attention means weighted focus.
- It improves context understanding.
One Line to Remember
Attention helps the model focus on what matters most.
Quick Self-Check
For Students
Attention looks at all words together - Yes / No?
Attention means awareness - Yes / No?
(Correct answers: Yes, No)