What is a Foundation Model?
A foundation model is a large-scale artificial intelligence model trained on a vast quantity of broad, unlabelled data. Instead of being trained to do one highly specific task (like predicting the price of a house), it learns general patterns and structures. It acts as a "foundation" that can be adapted, prompted, or fine-tuned to perform hundreds of different downstream tasks.
How does it work?
Foundation models are typically trained using self-supervised learning, where the model learns by hiding parts of the training data from itself and trying to predict the missing pieces. Because this process doesn't require humans to manually label the data, developers can train these models on almost the entire public internet.
What is it commonly confused with?
Foundation models are often conflated with Large Language Models (LLMs). While most famous foundation models are LLMs, a foundation model can be trained on entirely different modalities, such as raw visual data for computer vision or protein sequences for biological research.
Why does it matter?
Before foundation models, every AI application required building and training a custom model from scratch. Now, companies can take an existing foundation model and slightly adapt it for their specific use case, drastically reducing the time and cost required to deploy AI.