بازیابی و تولید تقویتشده با بازیابی
هوش مصنوعی برای همه: هوش مصنوعی چطور باهوش شد؟
- Why a model's parameters are a poor substitute for a current, inspectable document collection.
- Keyword retrieval and semantic retrieval using document embeddings.
- Splitting documents into chunks, indexing them, retrieving candidates, and optionally reranking them.
- Retrieval-augmented generation, or RAG: placing retrieved material into the model's context.
- A continuing example that answers questions from equipment manuals.
- Retrieval failure versus generation failure.
- Source attribution, irrelevant chunks, incomplete evidence, and why RAG reduces some errors without eliminating hallucination.
پارامترها سند نیستند
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><defs><marker id="arrow" markerWidth="10" markerHeight="10" refX="8" refY="5" orient="auto"><path class="tm-arrow" d="M0,0 L10,5 L0,10 z"/></marker></defs><rect class="tm-soft" x="65" y="95" width="280" height="245" rx="18"/><text class="tm-text" x="205" y="135" font-size="20" text-anchor="middle" direction="rtl">مدل زبانی</text><rect class="tm-surface" x="105" y="175" width="200" height="105" rx="12"/><text class="tm-text2" x="205" y="220" font-size="16" text-anchor="middle" direction="rtl">پاسخ بر پایهٔ شاهد</text><text class="tm-muted" x="205" y="250" font-size="14" text-anchor="middle" direction="rtl">نه فقط احتمال</text><rect class="tm-gold" x="455" y="95" width="280" height="245" rx="18"/><text class="tm-text" x="595" y="135" font-size="20" text-anchor="middle" direction="rtl">مجموعهٔ دفترچهها</text><rect class="tm-surface" x="490" y="170" width="210" height="42" rx="8"/><text class="tm-text2" x="595" y="197" font-size="15" text-anchor="middle" direction="rtl">متن</text><rect class="tm-surface" x="490" y="225" width="210" height="42" rx="8"/><text class="tm-text2" x="595" y="252" font-size="15" text-anchor="middle" direction="rtl">نام مدل و نسخه</text><rect class="tm-surface" x="490" y="280" width="210" height="42" rx="8"/><text class="tm-text2" x="595" y="307" font-size="15" text-anchor="middle" direction="rtl">بخش مربوط</text><path class="tm-line-accent" d="M345 215 H445" marker-end="url(#arrow)"/><text class="tm-text-accent" x="395" y="195" font-size="14" text-anchor="middle" direction="rtl">شاهد</text><text class="tm-muted" x="400" y="385" font-size="16" text-anchor="middle" direction="rtl">مسیر قابل بررسی؛ نه تضمین درستی</text></svg>
```
دو راه برای یافتن نامزدها
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><defs><marker id="arrow" markerWidth="10" markerHeight="10" refX="8" refY="3" orient="auto"><path d="M0,0 L0,6 L9,3 z" class="tm-arrow"/></marker></defs><rect x="26" y="28" width="748" height="62" rx="12" class="tm-soft"/><text x="400" y="67" text-anchor="middle" class="tm-text" font-size="18" direction="rtl">پرسش: خشک کردن قطعه</text><rect x="48" y="142" width="300" height="106" rx="14" class="tm-accent"/><text x="198" y="174" text-anchor="middle" class="tm-text" font-size="18">keyword retrieval</text><text x="198" y="212" text-anchor="middle" class="tm-text2" font-size="16" direction="rtl">واژهٔ دقیق و کد قطعه</text><rect x="452" y="142" width="300" height="106" rx="14" class="tm-accent"/><text x="602" y="174" text-anchor="middle" class="tm-text" font-size="18">semantic retrieval</text><text x="602" y="212" text-anchor="middle" class="tm-text2" font-size="16" direction="rtl">عبارتهای هممعنا</text><path d="M198 248 L198 300" class="tm-line-accent" marker-end="url(#arrow)"/><path d="M602 248 L602 300" class="tm-line-accent" marker-end="url(#arrow)"/><rect x="48" y="310" width="300" height="88" rx="14" class="tm-green"/><text x="198" y="345" text-anchor="middle" class="tm-text" font-size="16" direction="rtl">هویت دستگاه</text><text x="198" y="373" text-anchor="middle" class="tm-text2" font-size="16" direction="rtl">حفظ میشود</text><rect x="452" y="310" width="300" height="88" rx="14" class="tm-green"/><text x="602" y="345" text-anchor="middle" class="tm-text" font-size="16" direction="rtl">بیانهای متفاوت</text><text x="602" y="373" text-anchor="middle" class="tm-text2" font-size="16" direction="rtl">پیدا میشوند</text><rect x="238" y="410" width="324" height="34" rx="10" class="tm-gold"/><text x="400" y="433" text-anchor="middle" class="tm-text" font-size="15" direction="rtl">نامزد بررسی ≠ جواب تأییدشده</text></svg>
```
مسیر کامل RAG
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><defs><marker id="a" markerWidth="10" markerHeight="10" refX="8" refY="5" orient="auto"><path d="M0,0 L10,5 L0,10 z" class="tm-arrow"/></marker></defs><rect x="35" y="55" width="145" height="72" rx="12" class="tm-surface"/><text x="107" y="85" text-anchor="middle" font-size="17" class="tm-text" direction="rtl">دفترچه</text><text x="107" y="108" text-anchor="middle" font-size="14" class="tm-muted" direction="rtl">مدل، نسخه، نشانی</text><path d="M180 91 H225" class="tm-line" marker-end="url(#a)"/><rect x="230" y="42" width="120" height="58" rx="10" class="tm-accent"/><text x="290" y="77" text-anchor="middle" font-size="16" class="tm-text">chunk</text><rect x="230" y="112" width="120" height="58" rx="10" class="tm-soft"/><text x="290" y="147" text-anchor="middle" font-size="16" class="tm-text">chunk</text><text x="290" y="195" text-anchor="middle" font-size="13" class="tm-muted" direction="rtl">شرط + دستور کنار هم</text><path d="M350 106 H395" class="tm-line" marker-end="url(#a)"/><rect x="400" y="65" width="110" height="80" rx="12" class="tm-surface"/><text x="455" y="110" text-anchor="middle" font-size="17" class="tm-text">index</text><text x="455" y="132" text-anchor="middle" font-size="13" class="tm-muted" direction="rtl">واژه یا embedding</text><path d="M510 105 H555" class="tm-line" marker-end="url(#a)"/><rect x="560" y="65" width="100" height="80" rx="12" class="tm-soft"/><text x="610" y="100" text-anchor="middle" font-size="16" class="tm-text" direction="rtl">نامزدها</text><text x="610" y="124" text-anchor="middle" font-size="13" class="tm-muted">reranking</text><rect x="35" y="285" width="145" height="66" rx="12" class="tm-gold"/><text x="107" y="325" text-anchor="middle" font-size="17" class="tm-text" direction="rtl">پرسش</text><path d="M180 318 H590 V155" class="tm-line-accent" marker-end="url(#a)"/><rect x="675" y="65" width="100" height="80" rx="12" class="tm-green"/><text x="725" y="100" text-anchor="middle" font-size="16" class="tm-text">context</text><text x="725" y="124" text-anchor="middle" font-size="13" class="tm-muted" direction="rtl">پرسش + شاهد</text><path d="M660 105 H670" class="tm-line" marker-end="url(#a)"/><text x="400" y="375" text-anchor="middle" font-size="16" class="tm-text" direction="rtl">مدل پاسخ میسازد؛ پارامترها تغییر نمیکنند</text></svg>
```
انتخاب شاهد کامل
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><defs><marker id="arrow" markerWidth="10" markerHeight="10" refX="8" refY="5" orient="auto"><path d="M0,0 L10,5 L0,10 z" class="tm-arrow"/></marker></defs><rect x="30" y="35" width="175" height="70" rx="12" class="tm-accent"/><text x="117" y="65" text-anchor="middle" class="tm-text" font-size="17" direction="rtl">پرسش: مدل پایه</text><text x="117" y="88" text-anchor="middle" class="tm-text" font-size="16" direction="rtl">دما + مدت</text><rect x="315" y="25" width="190" height="72" rx="12" class="tm-green"/><text x="410" y="55" text-anchor="middle" class="tm-text" font-size="17" direction="rtl">الف</text><text x="410" y="78" text-anchor="middle" class="tm-text2" font-size="15" direction="rtl">مدل پایه، دما</text><rect x="315" y="120" width="190" height="72" rx="12" class="tm-green tm-pulse"/><text x="410" y="150" text-anchor="middle" class="tm-text" font-size="17" direction="rtl">ب</text><text x="410" y="173" text-anchor="middle" class="tm-text2" font-size="15" direction="rtl">مدل پایه، مدت</text><rect x="315" y="245" width="190" height="72" rx="12" class="tm-taupe"/><text x="410" y="275" text-anchor="middle" class="tm-text" font-size="17" direction="rtl">پ</text><text x="410" y="298" text-anchor="middle" class="tm-text2" font-size="15" direction="rtl">مدل بزرگ، دما</text><rect x="315" y="340" width="190" height="72" rx="12" class="tm-taupe"/><text x="410" y="370" text-anchor="middle" class="tm-text" font-size="17" direction="rtl">ت</text><text x="410" y="393" text-anchor="middle" class="tm-text2" font-size="15" direction="rtl">مدل پایه، تمیزکاری</text><path d="M205 70 C250 70 260 60 315 60" fill="none" class="tm-line" marker-end="url(#arrow)"/><path d="M505 60 C570 60 565 70 630 70" fill="none" class="tm-line-accent" marker-end="url(#arrow)"/><rect x="630" y="35" width="145" height="72" rx="12" class="tm-green"/><text x="702" y="65" text-anchor="middle" class="tm-text" font-size="16" direction="rtl">شاهد دما</text><text x="702" y="88" text-anchor="middle" class="tm-text" font-size="16" direction="rtl">۴۰ درجه</text><path d="M505 156 C570 156 565 180 630 180" fill="none" class="tm-line-accent tm-flow" marker-end="url(#arrow)"/><rect x="630" y="145" width="145" height="72" rx="12" class="tm-green tm-enter"/><text x="702" y="175" text-anchor="middle" class="tm-text" font-size="16" direction="rtl">شاهد مدت</text><text x="702" y="198" text-anchor="middle" class="tm-text" font-size="16" direction="rtl">۲۰ دقیقه</text></svg>
```
تشخیص دو منشأ خطا
```svg
<svg viewBox="0 0 800 450" xmlns="http://www.w3.org/2000/svg"><defs><marker id="a" markerWidth="10" markerHeight="10" refX="8" refY="5" orient="auto"><path d="M0,0 L10,5 L0,10 z" class="tm-arrow"/></marker></defs><rect x="40" y="45" width="325" height="350" rx="16" class="tm-soft"/><text x="202" y="78" text-anchor="middle" font-size="20" class="tm-text" direction="rtl">حالت اول: شاهد ناقص</text><rect x="75" y="110" width="255" height="58" rx="10" class="tm-surface"/><text x="202" y="145" text-anchor="middle" font-size="17" class="tm-text" direction="rtl">پرسش: دما و مدت</text><rect x="75" y="198" width="112" height="58" rx="10" class="tm-accent"/><text x="131" y="233" text-anchor="middle" font-size="17" class="tm-text" direction="rtl">قطعهٔ الف</text><text x="131" y="275" text-anchor="middle" font-size="16" class="tm-text-accent" direction="rtl">چهل درجه</text><rect x="204" y="198" width="126" height="58" rx="10" class="tm-surface"/><text x="267" y="233" text-anchor="middle" font-size="17" class="tm-muted" direction="rtl">شاهد مدت: ندارد</text><rect x="75" y="310" width="255" height="54" rx="10" class="tm-surface"/><text x="202" y="343" text-anchor="middle" font-size="16" class="tm-text" direction="rtl">جواب ناقص؛ کمبود retrieval</text><rect x="435" y="45" width="325" height="350" rx="16" class="tm-soft"/><text x="597" y="78" text-anchor="middle" font-size="20" class="tm-text" direction="rtl">حالت دوم: شاهد رسیده</text><rect x="470" y="110" width="255" height="58" rx="10" class="tm-surface"/><text x="597" y="145" text-anchor="middle" font-size="17" class="tm-text">context</text><rect x="470" y="198" width="112" height="58" rx="10" class="tm-surface"/><text x="526" y="233" text-anchor="middle" font-size="17" class="tm-text" direction="rtl">قطعهٔ الف</text><text x="526" y="275" text-anchor="middle" font-size="15" class="tm-text2" direction="rtl">چهل درجه</text><rect x="597" y="198" width="128" height="58" rx="10" class="tm-green"/><text x="661" y="233" text-anchor="middle" font-size="17" class="tm-text" direction="rtl">قطعهٔ ب</text><text x="661" y="275" text-anchor="middle" font-size="15" class="tm-text-green" direction="rtl">بیست دقیقه</text><rect x="470" y="310" width="255" height="54" rx="10" class="tm-accent"/><text x="597" y="343" text-anchor="middle" font-size="16" class="tm-text" direction="rtl">ادعا: یک ساعت؛ خطای generation</text></svg>
```
بازگشت به دوره