مقدمه دوره: مقاله Attention Is All You Need
Attention Is All You Need
- Opening question: the language models you use every day all trace back to one 2017 paper. What problem were its authors trying to solve?
- What this course is: a slow, faithful walk through a single research paper, in the paper's own order, using its own figures and tables
- The big story in plain words: older translation models read a sentence one word at a time, which made them slow to train; this paper replaced that step-by-step reading with a way for every word to look at every other word at once
- How the pieces connect: a way of letting words look at each other, a way of telling the model where each word sits in the sentence, and a stack of repeated building blocks that turns one language into another
- Roadmap: Module 1 asks why the old approach was a bottleneck; Module 2 builds the core mechanism and computes it by hand; Module 3 assembles the full model and traces the data through it; Module 4 weighs the costs and benefits of the new design; Module 5 covers how it was trained, what results it achieved, and which parts mattered most
- What you will be able to do at the end: work a small example by hand, follow the sizes of the data through one block of the model, and explain the trade-offs the authors made
از پردازش ترتیبی تا Transformer
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><defs><marker id="arrow" markerWidth="10" markerHeight="10" refX="8" refY="3" orient="auto"><path d="M0,0 L0,6 L9,3 z" class="tm-arrow"/></marker></defs><text x="400" y="38" class="tm-text" font-size="20" text-anchor="middle" direction="rtl">ترجمهٔ «گربه خوابید»</text><text x="400" y="88" class="tm-text2" font-size="18" text-anchor="middle" direction="rtl">مدل بازگشتی</text><rect x="90" y="120" width="150" height="72" rx="12" class="tm-accent"/><rect x="325" y="120" width="150" height="72" rx="12" class="tm-accent"/><rect x="560" y="120" width="150" height="72" rx="12" class="tm-surface"/><text x="165" y="162" class="tm-text" font-size="17" text-anchor="middle" direction="rtl">گربه</text><text x="400" y="162" class="tm-text" font-size="17" text-anchor="middle" direction="rtl">خوابید</text><text x="635" y="162" class="tm-text" font-size="17" text-anchor="middle" direction="rtl">نتیجه</text><path d="M240 156 L319 156" class="tm-line-accent" marker-end="url(#arrow)"/><path d="M475 156 L554 156" class="tm-line" marker-end="url(#arrow)"/><text x="400" y="224" class="tm-muted" font-size="15" text-anchor="middle" direction="rtl">هر گام وابسته به گام قبلی</text><text x="400" y="290" class="tm-text2" font-size="18" text-anchor="middle" direction="rtl">توجه همزمان به بخشهای ورودی</text><rect x="90" y="320" width="150" height="72" rx="12" class="tm-accent"/><rect x="325" y="320" width="150" height="72" rx="12" class="tm-accent"/><rect x="560" y="320" width="150" height="72" rx="12" class="tm-accent"/><text x="165" y="362" class="tm-text" font-size="17" text-anchor="middle" direction="rtl">گربه</text><text x="400" y="362" class="tm-text" font-size="17" text-anchor="middle" direction="rtl">خوابید</text><text x="635" y="362" class="tm-text" font-size="17" text-anchor="middle" direction="rtl">ترجمه</text></svg>
```
خواندن عمیق یک مقاله
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><rect class="tm-accent" x="285" y="35" width="230" height="58" rx="12"/><text class="tm-text" x="400" y="70" font-size="18" text-anchor="middle" direction="rtl">یک مقاله</text><rect class="tm-surface" x="55" y="165" width="190" height="58" rx="12"/><text class="tm-text" x="150" y="200" font-size="16" text-anchor="middle" direction="rtl">مسئله</text><rect class="tm-surface" x="305" y="165" width="190" height="58" rx="12"/><text class="tm-text" x="400" y="200" font-size="16" text-anchor="middle" direction="rtl">راهحل</text><rect class="tm-surface" x="555" y="165" width="190" height="58" rx="12"/><text class="tm-text" x="650" y="200" font-size="16" text-anchor="middle" direction="rtl">آموزش و نتیجهها</text><path class="tm-line" d="M245 194 H305"/><path class="tm-line" d="M495 194 H555"/><text class="tm-muted" x="400" y="285" font-size="16" text-anchor="middle" direction="rtl">آهسته و وفادار به ترتیب مقاله</text><rect class="tm-soft" x="95" y="330" width="250" height="52" rx="10"/><text class="tm-text2" x="220" y="362" font-size="16" text-anchor="middle" direction="rtl">شکل معماری مقاله</text><rect class="tm-soft" x="455" y="330" width="250" height="52" rx="10"/><text class="tm-text2" x="580" y="362" font-size="16" text-anchor="middle" direction="rtl">جدول آزمایشهای مقاله</text><rect class="tm-accent tm-enter" x="285" y="405" width="230" height="34" rx="9"/><text class="tm-text tm-enter" x="400" y="428" font-size="15" text-anchor="middle" direction="rtl">عمیقخوانی، نه مرور تاریخ مدلها</text></svg>
```
پیوندهای اصلی معماری
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><defs><marker id="arrow" markerWidth="10" markerHeight="10" refX="8" refY="3" orient="auto"><path d="M0,0 L0,6 L9,3 z" class="tm-arrow"/></marker></defs><rect x="58" y="82" width="150" height="58" rx="12" class="tm-soft"/><text x="133" y="117" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">گربه</text><rect x="58" y="206" width="150" height="58" rx="12" class="tm-soft"/><text x="133" y="241" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">خوابید</text><rect x="320" y="145" width="150" height="62" rx="12" class="tm-accent"/><text x="395" y="183" class="tm-text" font-size="18" text-anchor="middle">attention</text><path d="M208 111 C260 111 267 162 320 168" fill="none" class="tm-line-accent" marker-end="url(#arrow)"/><path d="M208 235 C260 235 267 188 320 181" fill="none" class="tm-line-accent" marker-end="url(#arrow)"/><text x="395" y="253" class="tm-muted" font-size="15" text-anchor="middle" direction="rtl">رابطهٔ واژهها</text><rect x="320" y="285" width="150" height="54" rx="10" class="tm-gold"/><text x="395" y="318" class="tm-text" font-size="16" text-anchor="middle" direction="rtl">اطلاعات جایگاه</text><path d="M395 285 L395 210" fill="none" class="tm-line" marker-end="url(#arrow)"/><rect x="560" y="75" width="170" height="64" rx="12" class="tm-green tm-enter"/><text x="645" y="114" class="tm-text" font-size="17" text-anchor="middle" direction="rtl">بلوک پردازشی</text><rect x="560" y="165" width="170" height="64" rx="12" class="tm-green"/><text x="645" y="204" class="tm-text" font-size="17" text-anchor="middle" direction="rtl">بلوک پردازشی</text><path d="M645 139 L645 165" fill="none" class="tm-line" marker-end="url(#arrow)"/><path d="M470 176 L560 107" fill="none" class="tm-line" marker-end="url(#arrow)"/><rect x="560" y="285" width="170" height="60" rx="12" class="tm-soft"/><text x="645" y="321" class="tm-text" font-size="17" text-anchor="middle" direction="rtl">ترجمه</text><path d="M645 229 L645 285" fill="none" class="tm-line" marker-end="url(#arrow)"/></svg>
```
سه مرحله نخست مسیر دوره
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><defs><marker id="arrow" markerWidth="10" markerHeight="10" refX="8" refY="5" orient="auto"><path class="tm-arrow" d="M0,0 L10,5 L0,10 z"/></marker></defs><rect class="tm-surface" x="35" y="90" width="210" height="110" rx="14"/><text class="tm-muted" font-size="20" text-anchor="middle" x="140" y="130" direction="rtl">۱. دلیل نیاز</text><text class="tm-muted" font-size="15" text-anchor="middle" x="140" y="164" direction="rtl">گلوگاه پردازش</text><line class="tm-line" x1="250" y1="145" x2="320" y2="145" marker-end="url(#arrow)"/><rect class="tm-surface" x="330" y="90" width="210" height="110" rx="14"/><text class="tm-muted" font-size="20" text-anchor="middle" x="435" y="130" direction="rtl">۲. سازوکار</text><text class="tm-muted" font-size="15" text-anchor="middle" x="435" y="164">attention</text><line class="tm-line" x1="545" y1="145" x2="615" y2="145" marker-end="url(#arrow)"/><rect class="tm-accent" x="625" y="90" width="150" height="110" rx="14"/><text class="tm-text" font-size="20" text-anchor="middle" x="700" y="130" direction="rtl">۳. مدل کامل</text><text class="tm-text2" font-size="15" text-anchor="middle" x="700" y="164" direction="rtl">قطعهها و داده</text><text class="tm-text2" font-size="16" text-anchor="middle" x="400" y="285" direction="rtl">از مسئله، به سازوکار، تا معماری</text></svg>
```
دو مرحله پایانی دوره
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><rect class="tm-surface" x="55" y="85" width="310" height="260" rx="18"/><text class="tm-text" x="210" y="125" font-size="22" text-anchor="middle" direction="rtl">مرحله چهارم</text><rect class="tm-accent" x="95" y="160" width="105" height="65" rx="10"/><text class="tm-text" x="147" y="200" font-size="17" text-anchor="middle" direction="rtl">فایده</text><rect class="tm-gold" x="220" y="160" width="105" height="65" rx="10"/><text class="tm-text" x="272" y="200" font-size="17" text-anchor="middle" direction="rtl">هزینه</text><text class="tm-muted" x="210" y="280" font-size="17" text-anchor="middle" direction="rtl">محاسبه همزمان ↔ جملههای طولانی</text><rect class="tm-surface" x="435" y="85" width="310" height="260" rx="18"/><text class="tm-text" x="590" y="125" font-size="22" text-anchor="middle" direction="rtl">مرحله پنجم</text><rect class="tm-accent" x="465" y="160" width="75" height="65" rx="10"/><text class="tm-text" x="502" y="200" font-size="15" text-anchor="middle" direction="rtl">آموزش</text><rect class="tm-accent" x="550" y="160" width="75" height="65" rx="10"/><text class="tm-text" x="587" y="200" font-size="15" text-anchor="middle" direction="rtl">نتیجهها</text><rect class="tm-accent" x="635" y="160" width="85" height="65" rx="10"/><text class="tm-text" x="677" y="200" font-size="15" text-anchor="middle" direction="rtl">مقایسه مدلها</text><text class="tm-muted" x="590" y="280" font-size="17" text-anchor="middle" direction="rtl">شواهد مقاله</text><rect class="tm-soft" x="250" y="380" width="300" height="45" rx="12"/><text class="tm-text2" x="400" y="409" font-size="17" text-anchor="middle" direction="rtl">پنج سؤال به ترتیب ← مثال attention با دست</text></svg>
```
بازگشت به دوره