In short
Store training and feature data in a columnar format so jobs read only the columns and partitions they use, and convert raw text to a binary format once instead of re-parsing it on every run or epoch. Measure a typical job before and after to check the conversion pays for itself. Data handling is easy to overlook next to the model, but it runs on every experiment.
Steps
- Find the pipelines that read text files such as CSV or JSON in full on every run.
- Convert them once to a columnar or binary format.
- Partition data by fields you filter on, such as date or source.
- Preprocess and tokenise once, and reuse the result across runs and epochs.
- Remove duplicate and unused data before training.
- Time and measure a typical job before and after the change.
Make each job read only what it uses, and parse it only once. Training data often lives in text files that are read and parsed in full every time, even when the model needs a handful of columns.
Why does it matter?
Reading and parsing data costs CPU time, memory and storage traffic on every run, every epoch and every experiment. In research and development, where the same dataset is processed hundreds of times, that adds up. The Green Software Foundation's pattern on data storage formats for AI recommends columnar formats, so jobs read only the columns and partitions they need, and converting data to a binary format once rather than re-parsing text.
What should I change?
- Columnar formats such as Parquet store each column separately, so a job using five columns out of fifty reads roughly a tenth of the data.
- Partitioning by date, source or label lets a job skip whole files it doesn't need.
- Pre-tokenised datasets save repeating the same text processing on every epoch.
- Deduplication shrinks the dataset and the training run with it.
Common mistakes
- Converting data that is read once. The conversion has a cost too; it pays off for data read repeatedly.
- Keeping every intermediate copy. Old preprocessed versions fill storage that has its own footprint.
- Not measuring. Wrap a typical job in a tracker such as CodeCarbon, or at least time it, before and after.
For model-side savings, see how to reduce the energy of training a model.
Sources
- Green Software Patterns: optimise data storage formats for AI training and inference, Green Software Foundation (2025)
- CodeCarbon, CodeCarbon (2026)
Related
How do I reduce the energy of training a model?
The main levers for cutting training energy: start pre-trained, right-size, use mixed precision, stop early, pick clean places and times, measure.
How do I make vector search and RAG more efficient?
Ways to cut the compute of retrieval-augmented generation: approximate indexes, fewer and shorter passages, incremental indexing and smaller models.
Efficient AI training and inference
Techniques for developers and researchers to cut AI energy: pruning, sparsity, quantisation, distillation, KV cache, batching, mixed precision and scheduling.
How do I measure the energy AI uses on my own computer or GPU?
Measure local AI energy with hardware counters, GPU telemetry or a plug-in meter, sample over the whole job, and subtract the idle baseline.