# How do I prune a neural network?

> How to prune a trained neural network to remove weights that contribute little, test what accuracy survives, and check the hardware really saves.

Source: https://greenai.guide/how-do-i/prune-a-neural-network/  
Updated: 2026-11-19

## In short

Prune a trained network by removing the weights, neurons or layers that contribute least, then fine-tune briefly so it recovers, and test it against your quality bar. Pruning cut AlexNet's parameters ninefold with no loss of accuracy, and very large language models have been pruned to at least 50% sparsity in one shot. Prefer structured pruning if you want ordinary hardware to run faster.

## Steps

1. Measure the accuracy, latency and energy of the unpruned model first.
2. Choose structured pruning (whole neurons, heads or layers) unless your hardware exploits sparse weights.
3. Remove a modest share of weights by importance, such as smallest magnitude.
4. Fine-tune briefly to let the network recover.
5. Repeat in steps, testing each time, and stop when quality drops below your bar.

**Large networks usually carry far more weights than their task needs, so many can be removed after training.**
Pruning makes the model smaller and, on the right hardware, cheaper to run.

## What does the evidence show?

Han and colleagues showed in 2015 that pruning reduced the parameters of the AlexNet image model by a factor of nine
and of VGG-16 by a factor of thirteen, with no loss of accuracy. Frankle and Carbin's lottery ticket hypothesis
found that large networks contain much smaller sub-networks that can match the full network's accuracy. For
language models, Frantar and Alistarh's SparseGPT pruned very large models to at least 50% sparsity in one shot,
without retraining, with little increase in perplexity.

## Structured or unstructured?

- **Unstructured pruning** removes individual weights. It can remove the most, but leaves scattered zeros that
  ordinary GPUs and CPUs often can't skip.
- **Structured pruning** removes whole neurons, attention heads, channels or layers. It removes less, but the
  smaller model runs faster on any hardware.

## Common mistakes

- **Counting zeros instead of measuring energy.** Theoretical savings often exceed measured ones. A 2026 UK green
  AI sandpit of mathematicians, computer scientists and industry noted that sparsity remains under-used partly
  because mainstream chips struggle to exploit it.
- **Pruning too much at once.** Go in steps and fine-tune between them.
- **Testing on benchmarks only.** Check your own cases, including rare ones, which pruning can hurt first.

Hoefler and colleagues' 2021 survey of more than 300 papers is the standard guide. For the theory, read
[maths for efficient AI](/articles/maths-for-efficient-ai/).

## Sources

- [Learning both weights and connections for efficient neural networks](https://arxiv.org/abs/1506.02626), Han, Pool, Tran and Dally, NeurIPS (2015)
- [SparseGPT: massive language models can be accurately pruned in one-shot](https://arxiv.org/abs/2301.00774), Frantar and Alistarh, ICML (2023)
- [Sparsity in deep learning: pruning and growth for efficient inference and training in neural networks](https://jmlr.org/papers/v22/21-0366.html), Hoefler, Alistarh, Ben-Nun, Dryden and Peste, Journal of Machine Learning Research (2021)
- [The lottery ticket hypothesis: finding sparse, trainable neural networks](https://arxiv.org/abs/1803.03635), Frankle and Carbin, ICLR (2019)
