In-context learning (ICL) is a capability of large language models to infer a task from instructions and examples provided only within the prompt—without updating model weights. By conditioning on instructions plus a few labeled demonstrations or format patterns, the model produces task-aligned outputs in the same session, enabling zero/few-shot classification, extraction, and reasoning.
What is In-Context Learning (ICL)?
ICL treats the prompt as a temporary training set and attention as the learner. Demonstrations create a pattern to imitate: input→output pairs, schemas (JSON/XML), reasoning styles, or tool-argument formats. The model aligns next-token predictions to the induced pattern, generalizing to new inputs. Reliability depends on example quality, coverage, ordering, delimiters, and negative counterexamples. ICL composes with retrieval (to fetch topical exemplars at runtime), function calling (to constrain outputs), and self-checking steps for validation.
Why it matters and where it’s used
ICL transforms a general LM into a task specialist on the fly—no fine-tuning, infrastructure, or data export required. It accelerates prototyping, supports privacy-sensitive or air-gapped settings, and reduces ops overhead. Teams use ICL for ad-hoc analytics, structured extraction, code transforms, and domain formatting, and as a building block in agents that learn conventions from context.
Examples
- Few-shot classification: provide 5–10 labeled examples, then classify new items.
- Structured extraction: show a target JSON schema with filled examples, then extract for new passages.
- Reasoning: demonstrate step-by-step solutions, then prompt a similar problem.
- Tool schemas: provide function signatures and valid argument shapes; ask the model to emit only arguments.
FAQs
- How many examples are needed? Start with 3–8; more can help but compete with context budget.
- Does order matter? Yes—place diverse, high-quality examples closest to the query; keep formatting consistent.
- Does ICL replace fine-tuning? Not always. Fine-tuning wins on scale, latency, and robustness; ICL excels for rapid, low-ops adaptation.
- How to improve reliability? Use retrieval to fetch exemplars, add counterexamples, enforce schemas, and add self-checks/verifiers.
- What about long context? Larger windows enable more/better exemplars; use compression and reranking to fit limits.