توجه و ترنسفورمر
هوش مصنوعی برای همه: هوش مصنوعی چطور باهوش شد؟
- Attention as assigning different relevance weights to available sequence representations.
- Queries, keys, and values through a small relevance-matching diagram without matrix calculations.
- Self-attention, multiple attention heads, and position information.
- The Transformer architecture introduced in 2017 and its combination of attention and feed-forward layers.
- Causal masking for next-token prediction.
- Why parallel processing of training sequences helped scaling compared with recurrent processing.
- Why generating a sequence still proceeds step by step and why long contexts remain computationally costly.
از زنجیره تا سهمهای متفاوت
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><defs><marker id="arrow" markerWidth="10" markerHeight="10" refX="8" refY="3" orient="auto"><path d="M0,0 L0,6 L9,3 z" class="tm-arrow"/></marker></defs><text x="400" y="48" class="tm-text" font-size="22" text-anchor="middle" direction="rtl">attention: محاسبهٔ یادگرفتهشده</text><rect x="78" y="110" width="145" height="64" rx="12" class="tm-soft"/><rect x="328" y="110" width="145" height="64" rx="12" class="tm-gold"/><rect x="578" y="110" width="145" height="64" rx="12" class="tm-accent"/><text x="150" y="150" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">ربات</text><text x="400" y="150" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">توپ</text><text x="650" y="150" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">برداشت</text><path d="M223 142 Q400 42 578 142" class="tm-line" marker-end="url(#arrow)"/><path d="M473 142 Q530 110 578 142" class="tm-line-accent" marker-end="url(#arrow)"/><text x="400" y="208" class="tm-text-accent" font-size="17" text-anchor="middle" direction="rtl">سهم بیشتر</text><rect x="328" y="260" width="145" height="58" rx="12" class="tm-surface"/><text x="400" y="296" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">را</text><path d="M473 289 Q540 270 578 170" class="tm-line" marker-end="url(#arrow)"/><text x="400" y="365" class="tm-muted" font-size="16" text-anchor="middle" direction="rtl">نه آگاهی؛ محاسبهٔ سهم اطلاعات</text></svg>
```
از query تا ترکیب value
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><defs><marker id="arrow" markerWidth="10" markerHeight="10" refX="8" refY="3" orient="auto"><path d="M0,0 L0,6 L9,3 z" class="tm-arrow"/></marker></defs><text x="400" y="34" class="tm-text" font-size="20" text-anchor="middle" direction="rtl">ترکیب وزندار valueها</text><rect x="48" y="74" width="130" height="58" rx="10" class="tm-accent"/><text x="113" y="109" class="tm-text" font-size="17" text-anchor="middle" direction="rtl">query: برداشت</text><rect x="240" y="64" width="120" height="42" rx="8" class="tm-soft"/><text x="300" y="91" class="tm-text" font-size="16" text-anchor="middle" direction="rtl">key: ربات</text><rect x="240" y="116" width="120" height="42" rx="8" class="tm-soft"/><text x="300" y="143" class="tm-text" font-size="16" text-anchor="middle" direction="rtl">key: توپ</text><rect x="240" y="168" width="120" height="42" rx="8" class="tm-soft"/><text x="300" y="195" class="tm-text" font-size="16" text-anchor="middle" direction="rtl">key: را</text><rect x="240" y="220" width="120" height="42" rx="8" class="tm-soft"/><text x="300" y="247" class="tm-text" font-size="16" text-anchor="middle" direction="rtl">key: برداشت</text><rect x="405" y="64" width="110" height="42" rx="8" class="tm-gold"/><text x="460" y="91" class="tm-text" font-size="16" text-anchor="middle" direction="rtl">ربات: ۱/۴</text><rect x="405" y="116" width="110" height="42" rx="8" class="tm-accent"/><text x="460" y="143" class="tm-text" font-size="16" text-anchor="middle" direction="rtl">توپ: ۱/۲</text><rect x="405" y="168" width="110" height="42" rx="8" class="tm-soft"/><text x="460" y="195" class="tm-text" font-size="16" text-anchor="middle" direction="rtl">را: ۱/۸</text><rect x="405" y="220" width="110" height="42" rx="8" class="tm-soft"/><text x="460" y="247" class="tm-text" font-size="16" text-anchor="middle" direction="rtl">برداشت: ۱/۸</text><path d="M515 84 C550 84 550 110 582 138" class="tm-line-accent" marker-end="url(#arrow)"/><path d="M515 137 C550 137 550 137 582 145" class="tm-line-accent" marker-end="url(#arrow)"/><path d="M515 189 C550 189 550 163 582 152" class="tm-line" marker-end="url(#arrow)"/><path d="M515 241 C550 241 550 176 582 159" class="tm-line" marker-end="url(#arrow)"/><rect x="590" y="117" width="170" height="72" rx="12" class="tm-green"/><text x="675" y="147" class="tm-text" font-size="16" text-anchor="middle" direction="rtl">نمایش ترکیبی</text><text x="675" y="170" class="tm-text2" font-size="14" text-anchor="middle" direction="rtl">نه یک کلمهٔ برنده</text><text x="400" y="314" class="tm-muted" font-size="14" text-anchor="middle" direction="rtl">مقایسهٔ query با keyها ← ساختن وزنها ← ترکیب valueها</text></svg>
```
اجزای معماری Transformer
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><defs><marker id="arr" markerWidth="10" markerHeight="10" refX="8" refY="3" orient="auto"><path class="tm-arrow" d="M0,0 L0,6 L9,3 z"/></marker></defs><rect class="tm-soft" x="30" y="55" width="145" height="58" rx="10"/><text class="tm-text" font-size="18" text-anchor="middle" x="102" y="90" direction="rtl">نمایش دنباله</text><rect class="tm-accent" x="215" y="55" width="145" height="58" rx="10"/><text class="tm-text" font-size="18" text-anchor="middle" x="287" y="90">self-attention</text><rect class="tm-soft" x="400" y="55" width="145" height="58" rx="10"/><text class="tm-text" font-size="18" text-anchor="middle" x="472" y="90">feed-forward</text><rect class="tm-soft" x="585" y="55" width="175" height="58" rx="10"/><text class="tm-text" font-size="18" text-anchor="middle" x="672" y="90" direction="rtl">نمایش تازه</text><path class="tm-line" marker-end="url(#arr)" d="M175 84 L210 84"/><path class="tm-line" marker-end="url(#arr)" d="M360 84 L395 84"/><path class="tm-line" marker-end="url(#arr)" d="M545 84 L580 84"/><rect class="tm-accent" x="215" y="155" width="145" height="45" rx="9"/><text class="tm-text" font-size="16" text-anchor="middle" x="287" y="183" direction="rtl">head ۱</text><rect class="tm-gold" x="215" y="215" width="145" height="45" rx="9"/><text class="tm-text" font-size="16" text-anchor="middle" x="287" y="243" direction="rtl">head ۲</text><path class="tm-line" marker-end="url(#arr)" d="M287 113 L287 150"/><path class="tm-line" marker-end="url(#arr)" d="M287 113 L287 210"/><text class="tm-text2" font-size="17" text-anchor="middle" x="420" y="184" direction="rtl">ترکیب headها</text><path class="tm-line" marker-end="url(#arr)" d="M360 177 L420 177"/><path class="tm-line" marker-end="url(#arr)" d="M360 237 L420 190"/><text class="tm-muted" font-size="15" text-anchor="middle" x="400" y="320" direction="rtl">اطلاعات موقعیت وارد نمایش هر token میشود</text><rect class="tm-gold" x="55" y="355" width="140" height="48" rx="9"/><text class="tm-text" font-size="17" text-anchor="middle" x="125" y="385" direction="rtl">ربات</text><rect class="tm-gold" x="255" y="355" width="140" height="48" rx="9"/><text class="tm-text" font-size="17" text-anchor="middle" x="325" y="385" direction="rtl">توپ</text><rect class="tm-accent" x="455" y="355" width="140" height="48" rx="9"/><text class="tm-text" font-size="17" text-anchor="middle" x="525" y="385" direction="rtl">برداشت</text><path class="tm-line-accent" marker-end="url(#arr)" d="M195 379 C280 300 420 300 480 355"/><text class="tm-text-accent" font-size="15" text-anchor="middle" x="335" y="308" direction="rtl">ارتباط مستقیم در یک مرحله</text></svg>
```
محدودیت آینده در causal masking
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><text x="400" y="38" text-anchor="middle" class="tm-text" font-size="20" direction="rtl">ردیف جایگاه ← اطلاعات در دسترس</text><text x="260" y="90" text-anchor="middle" class="tm-muted" font-size="15" direction="rtl">ربات</text><text x="440" y="90" text-anchor="middle" class="tm-muted" font-size="15" direction="rtl">توپ</text><text x="580" y="90" text-anchor="middle" class="tm-muted" font-size="15" direction="rtl">را</text><text x="710" y="90" text-anchor="middle" class="tm-muted" font-size="15" direction="rtl">برداشت</text><rect x="25" y="110" width="130" height="62" rx="10" class="tm-soft"/><text x="90" y="148" text-anchor="middle" class="tm-text" font-size="17" direction="rtl">ربات</text><rect x="25" y="190" width="130" height="62" rx="10" class="tm-soft"/><text x="90" y="228" text-anchor="middle" class="tm-text" font-size="17" direction="rtl">توپ</text><rect x="25" y="270" width="130" height="62" rx="10" class="tm-soft"/><text x="90" y="308" text-anchor="middle" class="tm-text" font-size="17" direction="rtl">را</text><rect x="25" y="350" width="130" height="62" rx="10" class="tm-soft"/><text x="90" y="388" text-anchor="middle" class="tm-text" font-size="17" direction="rtl">برداشت</text><rect x="220" y="110" width="80" height="62" rx="10" class="tm-green"/><rect x="400" y="110" width="80" height="62" rx="10" class="tm-taupe"/><rect x="540" y="110" width="80" height="62" rx="10" class="tm-taupe"/><rect x="670" y="110" width="80" height="62" rx="10" class="tm-taupe"/><rect x="220" y="190" width="80" height="62" rx="10" class="tm-green"/><rect x="400" y="190" width="80" height="62" rx="10" class="tm-green"/><rect x="540" y="190" width="80" height="62" rx="10" class="tm-taupe"/><rect x="670" y="190" width="80" height="62" rx="10" class="tm-accent"/><rect x="220" y="270" width="80" height="62" rx="10" class="tm-green"/><rect x="400" y="270" width="80" height="62" rx="10" class="tm-green"/><rect x="540" y="270" width="80" height="62" rx="10" class="tm-green"/><rect x="670" y="270" width="80" height="62" rx="10" class="tm-taupe"/><rect x="220" y="350" width="80" height="62" rx="10" class="tm-green"/><rect x="400" y="350" width="80" height="62" rx="10" class="tm-green"/><rect x="540" y="350" width="80" height="62" rx="10" class="tm-green"/><rect x="670" y="350" width="80" height="62" rx="10" class="tm-green"/><text x="710" y="228" text-anchor="middle" class="tm-text-accent" font-size="14" direction="rtl">ممنوع</text><text x="440" y="228" text-anchor="middle" class="tm-text" font-size="15" direction="rtl">مجاز</text><text x="580" y="308" text-anchor="middle" class="tm-muted" font-size="14" direction="rtl">ممنوع</text><text x="710" y="308" text-anchor="middle" class="tm-muted" font-size="14" direction="rtl">ممنوع</text><text x="580" y="388" text-anchor="middle" class="tm-text" font-size="14" direction="rtl">مجاز</text><text x="710" y="388" text-anchor="middle" class="tm-text" font-size="14" direction="rtl">مجاز</text><path d="M440 220 C520 200 600 200 670 220" fill="none" class="tm-line-accent" marker-end="url(#a)"/><text x="550" y="190" text-anchor="middle" class="tm-text-accent" font-size="14" direction="rtl">اتصال ممنوع</text><path d="M710 370 C650 330 500 330 440 230" fill="none" class="tm-line" marker-end="url(#a)"/><text x="580" y="340" text-anchor="middle" class="tm-text-green" font-size="14" direction="rtl">گذشته: مجاز</text></svg>
```
آموزش موازی، تولید مرحلهای
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><defs><marker id="arrow" markerWidth="10" markerHeight="10" refX="8" refY="3" orient="auto"><path d="M0,0 L0,6 L9,3 z" class="tm-arrow"/></marker></defs><text x="400" y="38" text-anchor="middle" class="tm-text" font-size="20" direction="rtl">آموزش و تولید</text><text x="60" y="118" class="tm-text2" font-size="17" direction="rtl">تولید</text><rect x="170" y="82" width="125" height="56" rx="10" class="tm-accent"/><rect x="335" y="82" width="125" height="56" rx="10" class="tm-accent"/><rect x="500" y="82" width="125" height="56" rx="10" class="tm-accent"/><text x="232" y="116" text-anchor="middle" class="tm-text" font-size="16" direction="rtl">ربات</text><text x="397" y="116" text-anchor="middle" class="tm-text" font-size="16" direction="rtl">توپ</text><text x="562" y="116" text-anchor="middle" class="tm-text" font-size="16" direction="rtl">را</text><text x="60" y="232" class="tm-text2" font-size="17" direction="rtl">آموزش</text><rect x="170" y="194" width="125" height="56" rx="10" class="tm-accent"/><rect x="335" y="194" width="125" height="56" rx="10" class="tm-accent"/><rect x="500" y="194" width="125" height="56" rx="10" class="tm-accent"/><text x="232" y="228" text-anchor="middle" class="tm-text" font-size="16" direction="rtl">ربات ← توپ</text><text x="397" y="228" text-anchor="middle" class="tm-text" font-size="16" direction="rtl">توپ ← را</text><text x="562" y="228" text-anchor="middle" class="tm-text" font-size="16" direction="rtl">را ← برداشت</text><text x="400" y="318" text-anchor="middle" class="tm-muted" font-size="16" direction="rtl">mask: آینده نشت نمیکند</text><text x="400" y="365" text-anchor="middle" class="tm-text-gold" font-size="17" direction="rtl">طول × ۲ ⟶ ارتباطها تقریباً × ۴</text><text x="400" y="397" text-anchor="middle" class="tm-muted" font-size="15" direction="rtl">هزینهٔ attention در متن بلند</text></svg>
```
بازگشت به دوره