# How do I avoid training a model from scratch?

> How to reuse existing models instead of pre-training your own: fine-tune, adapt with LoRA, merge fine-tuned models or retrieve knowledge.

Source: https://greenai.guide/how-do-i/avoid-training-a-model-from-scratch/  
Updated: 2026-11-19

## In short

Search for a pre-trained model close to your task, then adapt it: fine-tune it, train a small adapter such as LoRA, or add knowledge through retrieval instead of retraining. Where you already have several models fine-tuned from the same base, averaging their weights can improve accuracy at no extra inference cost. Training from scratch is rarely necessary outside foundation-model research.

## Steps

1. Search public model hubs for pre-trained models in your domain and language.
2. Test the closest candidates on your task before deciding anything is missing.
3. Fine-tune or train a LoRA adapter on your own data.
4. Use retrieval for knowledge that changes instead of retraining.
5. Reuse and merge models fine-tuned from the same base where you have them.
6. Publish your fine-tuned model and its training emissions so others can reuse it.

**Reuse first.** Somebody has almost always trained a model close to what you need, and adapting it costs a small
fraction of training your own.

## Why does it matter?

Pre-training is the most energy-intensive stage of a model's life. The Green Software Foundation's patterns
recommend leveraging pre-trained models and transfer learning rather than training from scratch. A 2026 UK green AI
sandpit of mathematicians, computer scientists and industry also noted that a lot of useful work sits in little-known
repositories, so a careful search is the first step of any project.

## What are the options?

- **Fine-tuning** a pre-trained model on your data. With LoRA, Hu and colleagues reduced trainable parameters by
  10,000 times compared with fully fine-tuning GPT-3 175B. See
  [how to fine-tune without huge compute](/how-do-i/fine-tune-a-model-with-less-compute/).
- **Retrieval** for knowledge that changes: see [retrieval or fine-tuning](/how-do-i/choose-between-rag-and-fine-tuning/).
- **Model merging.** Wortsman and colleagues showed that averaging the weights of several models fine-tuned from the
  same base, a "model soup", can improve accuracy without increasing inference time.
- **Distillation** into a small specialist: see [how to distil a model](/how-do-i/distil-a-large-model-into-a-small-one/).

## Common mistakes

- **Assuming your domain is too specialist.** Test general and domain models first; they often get close.
- **Ignoring licences**, which can rule out some reuse.
- **Not sharing back.** Publishing your fine-tuned model, with training emissions in the model card metadata as
  Hugging Face supports, saves others repeating your work.

For the research angle, read [maths for efficient AI](/articles/maths-for-efficient-ai/).

## Sources

- [Green Software Patterns: leverage pre-trained models and transfer learning](https://github.com/Green-Software-Foundation/patterns/blob/main/docs/development/pre-trained-transfer-learning.md), Green Software Foundation (2025)
- [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time](https://arxiv.org/abs/2203.05482), Wortsman et al., ICML (2022)
- [LoRA: low-rank adaptation of large language models](https://arxiv.org/abs/2106.09685), Hu et al. (2021)
- [Displaying carbon emissions for your model](https://huggingface.co/docs/hub/model-cards-co2), Hugging Face Hub documentation (2026)
