مدلسازی زبان پیش از ترنسفورمر
هوش مصنوعی برای همه: هوش مصنوعی چطور باهوش شد؟
- Language modeling as predicting plausible continuations of a sequence.
- Count-based models that estimate the next word from a short preceding window.
- Sparse observations and the limitations of fixed short windows.
- Recurrent neural networks as updating a compact state while reading a sequence.
- Sequence-to-sequence systems for tasks such as translating simple technical instructions.
- Difficulties retaining distant details and processing long sequences efficiently.
- Why useful language patterns can be learned without a handwritten rule for every sentence.
ادامهٔ محتمل از متن
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><rect class="tm-soft" x="55" y="155" width="360" height="90" rx="14"/><text class="tm-text" x="235" y="190" font-size="20" text-anchor="middle" direction="rtl">برای روشنکردن دستگاه،</text><text class="tm-text" x="235" y="222" font-size="20" text-anchor="middle" direction="rtl">دکمه را فشار</text><rect class="tm-green" x="455" y="105" width="230" height="70" rx="14"/><text class="tm-text-green" x="570" y="148" font-size="22" text-anchor="middle" direction="rtl">بده</text><rect class="tm-taupe" x="455" y="230" width="230" height="70" rx="14"/><text class="tm-text2" x="570" y="273" font-size="22" text-anchor="middle" direction="rtl">موز</text><text class="tm-muted" x="570" y="335" font-size="16" text-anchor="middle" direction="rtl">ادامههای ممکن</text><rect class="tm-accent" x="55" y="350" width="630" height="58" rx="12"/><text class="tm-text-accent" x="370" y="386" font-size="18" text-anchor="middle" direction="rtl">بخش قبلی متن ← تخمین احتمال ادامه</text></svg>
```
شمارش در پنجرهٔ کوتاه
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><defs><marker id="arrow" markerWidth="10" markerHeight="10" refX="8" refY="3" orient="auto"><path d="M0,0 L0,6 L9,3 z" class="tm-arrow"/></marker></defs><rect x="45" y="55" width="190" height="80" rx="12" class="tm-soft"/><text x="140" y="88" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">پنجره</text><text x="140" y="116" class="tm-text-accent" font-size="20" text-anchor="middle" direction="rtl">«دکمه را»</text><path d="M235 95 H310" class="tm-line" marker-end="url(#arrow)"/><rect x="315" y="55" width="180" height="80" rx="12" class="tm-surface"/><text x="405" y="88" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">متن آموزشی</text><text x="405" y="116" class="tm-muted" font-size="16" text-anchor="middle" direction="rtl">ادامهها</text><path d="M495 95 H560" class="tm-line" marker-end="url(#arrow)"/><rect x="565" y="55" width="190" height="80" rx="12" class="tm-accent"/><text x="660" y="88" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">محتملترین</text><text x="660" y="116" class="tm-text" font-size="20" text-anchor="middle" direction="rtl">«فشار»</text><text x="400" y="190" class="tm-text2" font-size="17" text-anchor="middle" direction="rtl">ادامههای دیدهشده</text><rect x="210" y="215" width="150" height="65" rx="10" class="tm-accent"/><text x="285" y="255" class="tm-text" font-size="20" text-anchor="middle" direction="rtl">فشار: ۶</text><rect x="375" y="215" width="150" height="65" rx="10" class="tm-soft"/><text x="450" y="255" class="tm-text" font-size="20" text-anchor="middle" direction="rtl">نگه: ۳</text><rect x="540" y="215" width="150" height="65" rx="10" class="tm-soft"/><text x="615" y="255" class="tm-text" font-size="20" text-anchor="middle" direction="rtl">رها: ۱</text><text x="400" y="340" class="tm-muted" font-size="17" text-anchor="middle" direction="rtl">احتمال هر گزینه = تعداد آن ÷ مجموع تعدادها</text><rect x="45" y="380" width="210" height="45" rx="10" class="tm-gold tm-pulse"/><text x="150" y="409" class="tm-text" font-size="16" text-anchor="middle" direction="rtl">پنجرهٔ بینمونه</text><path d="M255 402 H330" class="tm-line-accent" marker-end="url(#arrow)"/><rect x="335" y="380" width="420" height="45" rx="10" class="tm-soft"/><text x="545" y="409" class="tm-muted" font-size="16" text-anchor="middle" direction="rtl">ندیدنِ ترکیب، ناممکنبودنش را ثابت نمیکند</text></svg>
```
حالت درونی RNN
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><defs><marker id="a" markerWidth="10" markerHeight="10" refX="8" refY="5" orient="auto"><path d="M0,0 L10,5 L0,10 z" class="tm-arrow"/></marker></defs><rect x="35" y="165" width="120" height="62" rx="12" class="tm-soft"/><text x="95" y="202" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">حالت آغازین</text><rect x="210" y="165" width="135" height="62" rx="12" class="tm-accent"/><text x="277" y="202" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">بهروزرسانی</text><line x1="155" y1="196" x2="210" y2="196" class="tm-line" marker-end="url(#a)"/><rect x="180" y="65" width="100" height="50" rx="10" class="tm-gold"/><text x="230" y="96" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">دستگاه</text><line x1="230" y1="115" x2="260" y2="165" class="tm-line-accent" marker-end="url(#a)"/><rect x="400" y="165" width="120" height="62" rx="12" class="tm-green"/><text x="460" y="202" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">حالت تازه</text><line x1="345" y1="196" x2="400" y2="196" class="tm-line" marker-end="url(#a)"/><text x="400" y="250" class="tm-muted" font-size="15" text-anchor="middle" direction="rtl">خلاصهٔ آنچه تا اینجا خوانده</text><rect x="360" y="65" width="100" height="50" rx="10" class="tm-gold"/><text x="410" y="96" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">آماده</text><line x1="410" y1="115" x2="300" y2="165" class="tm-line" marker-end="url(#a)"/><rect x="545" y="165" width="135" height="62" rx="12" class="tm-accent"/><text x="612" y="202" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">بهروزرسانی</text><line x1="520" y1="196" x2="545" y2="196" class="tm-line" marker-end="url(#a)"/><rect x="535" y="65" width="70" height="50" rx="10" class="tm-gold"/><text x="570" y="96" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">است</text><line x1="570" y1="115" x2="590" y2="165" class="tm-line-accent" marker-end="url(#a)"/><rect x="700" y="165" width="85" height="62" rx="12" class="tm-green"/><text x="742" y="202" class="tm-text" font-size="16" text-anchor="middle" direction="rtl">حالت نهایی</text><line x1="680" y1="196" x2="700" y2="196" class="tm-line" marker-end="url(#a)"/><rect x="700" y="300" width="85" height="55" rx="10" class="tm-soft"/><text x="742" y="333" class="tm-text" font-size="16" text-anchor="middle" direction="rtl">پیشبینی</text><line x1="742" y1="227" x2="742" y2="300" class="tm-line-accent tm-flow" marker-end="url(#a)"/><text x="670" y="390" class="tm-muted" font-size="15" text-anchor="middle" direction="rtl">احتمال کلمهٔ بعدی</text></svg>
```
خواندن ورودی، ساختن خروجی
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><defs><marker id="arrow" markerWidth="10" markerHeight="10" refX="8" refY="3" orient="auto"><path d="M0,0 L0,6 L9,3 z" class="tm-arrow"/></marker></defs><text x="400" y="38" text-anchor="middle" font-size="22" class="tm-text">sequence-to-sequence</text><text x="400" y="94" text-anchor="middle" font-size="17" class="tm-muted" direction="rtl">دنبالهٔ ورودی</text><rect x="105" y="118" width="130" height="54" rx="10" class="tm-soft"/><text x="170" y="151" text-anchor="middle" font-size="18" class="tm-text" direction="rtl">دکمه</text><rect x="260" y="118" width="70" height="54" rx="10" class="tm-soft"/><text x="295" y="151" text-anchor="middle" font-size="18" class="tm-text" direction="rtl">را</text><rect x="355" y="118" width="90" height="54" rx="10" class="tm-soft"/><text x="400" y="151" text-anchor="middle" font-size="18" class="tm-text" direction="rtl">فشار</text><rect x="470" y="118" width="75" height="54" rx="10" class="tm-soft"/><text x="507" y="151" text-anchor="middle" font-size="18" class="tm-text" direction="rtl">بده</text><rect x="290" y="213" width="220" height="68" rx="14" class="tm-accent"/><text x="400" y="242" text-anchor="middle" font-size="18" class="tm-text">encoder</text><text x="400" y="265" text-anchor="middle" font-size="15" class="tm-text" direction="rtl">حالت نهایی</text><path d="M170 172 L320 213 M295 172 L360 213 M400 172 L400 213 M507 172 L480 213" fill="none" class="tm-line" stroke-width="2" marker-end="url(#arrow)"/><rect x="320" y="323" width="160" height="58" rx="12" class="tm-accent"/><text x="400" y="358" text-anchor="middle" font-size="18" class="tm-text">decoder</text><path d="M400 281 L400 323" fill="none" class="tm-line-accent" stroke-width="3" marker-end="url(#arrow)"/><text x="400" y="414" text-anchor="middle" font-size="16" class="tm-muted" direction="rtl">خروجی، قدمبهقدم</text><rect x="95" y="414" width="105" height="28" rx="8" class="tm-green"/><text x="147" y="434" text-anchor="middle" font-size="15" class="tm-text" direction="rtl">کلید</text><rect x="220" y="414" width="65" height="28" rx="8" class="tm-soft"/><text x="252" y="434" text-anchor="middle" font-size="15" class="tm-text" direction="rtl">را</text><rect x="305" y="414" width="105" height="28" rx="8" class="tm-soft"/><text x="357" y="434" text-anchor="middle" font-size="15" class="tm-text" direction="rtl">بفشار</text><text x="455" y="434" text-anchor="middle" font-size="15" class="tm-muted" direction="rtl">پایان خروجی</text></svg>
```
حافظه و ترتیب در RNN
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><defs><marker id="a" markerWidth="10" markerHeight="10" refX="8" refY="3" orient="auto"><path d="M0,0 L0,6 L9,3 z" class="tm-arrow"/></marker></defs><text x="400" y="38" class="tm-text" font-size="22" text-anchor="middle" direction="rtl">حافظه محدود؛ محاسبهٔ پیاپی</text><rect x="55" y="82" width="150" height="62" rx="12" class="tm-accent"/><text x="130" y="120" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">فیلتر آبی</text><rect x="245" y="82" width="150" height="62" rx="12" class="tm-soft"/><text x="320" y="120" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">تمیز کن</text><rect x="435" y="82" width="150" height="62" rx="12" class="tm-soft"/><text x="510" y="120" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">قاب را بررسی کن</text><rect x="625" y="82" width="125" height="62" rx="12" class="tm-soft"/><text x="687" y="120" class="tm-text" font-size="18" text-anchor="middle" direction="rtl">همان فیلتر</text><path d="M205 113 H242 M395 113 H432 M585 113 H622" class="tm-line" marker-end="url(#a)"/><rect x="62" y="190" width="300" height="72" rx="12" class="tm-surface"/><text x="212" y="220" class="tm-text" font-size="17" text-anchor="middle" direction="rtl">پنجرهٔ پایان</text><text x="212" y="245" class="tm-muted" font-size="16" text-anchor="middle" direction="rtl">اطلاعات آغاز در پنجره نیست</text><rect x="438" y="190" width="300" height="72" rx="12" class="tm-soft"/><text x="588" y="220" class="tm-text" font-size="17" text-anchor="middle" direction="rtl">حالتهای RNN</text><text x="588" y="245" class="tm-text-accent" font-size="18" text-anchor="middle" direction="rtl">اثر رنگ؛ حفظ نامطمئن</text><path d="M130 145 C160 300 500 305 588 263" class="tm-line-accent" marker-end="url(#a)"/><text x="400" y="342" class="tm-muted" font-size="16" text-anchor="middle" direction="rtl">جزئیات دور کمرنگ میشود؛ آموزش دشوار</text><rect x="245" y="370" width="310" height="48" rx="12" class="tm-gold"/><text x="400" y="400" class="tm-text" font-size="17" text-anchor="middle" direction="rtl">sequence-to-sequence: یک حالت نهایی</text><text x="400" y="438" class="tm-text" font-size="17" text-anchor="middle" direction="rtl">حالت قبلی ← حالت بعدی ← دنبالهٔ بلند، زمان بیشتر</text></svg>
```
بازگشت به دوره