AIAdvanced#Machine Learning#Risk
Model Collapse
M
Definition
Degradation that can occur when generative models are repeatedly trained on their own or other model-generated outputs, especially when the training process loses rare but important parts of the original data distribution.
Source reviewed
- Nature: AI models collapse when trained on recursively generated data opens in a new tab
Supports: Reports degradation and loss of distribution tails from indiscriminate recursive use of model-generated data under the studied conditions.
Concept Diagram
Key Takeaways
1
Recursive use of generated data can lose rare parts of an original distribution under studied conditions
2
Synthetic data is not automatically harmful or safe; provenance, filtering, mixture, and evaluation matter
3
Test rare and held-out real cases rather than relying only on aggregate scores
4
Keep source records and labels so generated summaries can be traced and checked
Practical Example
A research team keeps the original filings and market records separate from model-generated summaries, labels every synthetic sample, and evaluates an updated model on rare filings and held-out records. If coverage drops, it changes the data mixture or filters rather than claiming synthetic data is universally harmful.
Finished learning?
Related Terms
AI
LLM ReasoningUsing a language model to summarize, classify, or compare supplied information. Its output is probabilistic text, not verified market analysis, financial advice, or authority to execute a trade.
AI
Fine-TuningAdditional training of an existing model on a selected dataset for a defined task. It may change model behavior on evaluated examples; it does not prove domain accuracy, reduce factual errors by itself, or create a reliable trading system.
AI
RLHF (Reinforcement Learning from Human Feedback)A family of training methods that uses human demonstrations or preferences to train a reward signal and optimize a model toward that measured preference. It can improve behavior on evaluated tasks, but it does not establish truth, safety, or domain competence.
AI
Model DistillationA training technique in which a student model learns from outputs or internal signals of a teacher model or ensemble. It can reduce deployment cost or latency, but does not guarantee that the student matches the teacher or is suitable for a task.
Related Deep Dives
Master the hub of AI
Explore all our strategic guides about AI to take your operations to the next level.
View all articles