NILMBench2026
A reproducible benchmark for energy disaggregation
We evaluate 16 NILM models across 3 datasets and 2 resolutions — on accuracy, efficiency, and generalization. Cross-building and cross-dataset results show a substantial generalization gap.
Schematic of the NILM task — appliance MAEs are NILMFormer's intra-building scores on UK-DALE. Real predictions below.
Benchmark scope and motivation
Non-Intrusive Load Monitoring (NILM) decomposes a household's aggregate power signal into appliance-level estimates. Despite many architectural advances, progress is hindered by the lack of a reproducible benchmark that evaluates models under varying compute budgets and at the resolutions that matter for real-time feedback (1 min) and utility-scale planning (15 min).
NILMBench2026 systematically evaluates sixteen models across regression accuracy, event detection, computational cost, and generalization on UK-DALE, REDD and REFIT. Two insights emerge: (1) model performance is highly context-dependent, varying with appliance type and temporal resolution; and (2) more critically, most models fail to generalize across buildings — especially when test appliances exhibit power characteristics unseen during training.
We also update the NILMTK model implementations: legacy models are re-implemented in PyTorch behind a unified API, and the evaluated environments are specified with Docker and uv. These artifacts support reproduction and extension of the reported experiments.
Evaluation beyond in-domain accuracy
Foundational toolkits standardized data parsers and baselines. NILMBench2026 additionally evaluates computational efficiency, 15-minute resolution, and cross-domain generalization under a recorded software environment.
| Feature | NILMTK '14 | Contrib '19 | NILMBench2026 |
|---|---|---|---|
| Deployability | ✗ | ✗ | Docker + uv |
| Models | 2 | 9 | 16 |
| Resolutions | variable | 1-min | 1-min & 15-min |
| Efficiency (FLOPs / time) | ✗ | ✗ | ✓ |
| Cross-building generalization | ✗ | ✓ | ✓ |
| Cross-dataset generalization | ✗ | ✗ | ✓ |
| Software stack | Python 2.7 | TensorFlow 1.x | PyTorch + uv |
Reproduce every result in three commands
NILMBench2026 is the benchmark runner around NILMTK
data and nilmtk-contrib models.
It freezes real-data protocols, provenance and leaderboard generation behind one CLI. Use uv
for development or build the pinned CPU/CUDA container family. Pick a tab.
# Python 3.11 benchmark environment uv venv --python 3.11 source .venv/bin/activate uv pip install "nilmbench[benchmark] @ git+https://github.com/nilmtk/nilmbench.git" nilmbench list
# Until the first public image release, build the pinned CUDA runner locally git clone https://github.com/nilmtk/nilmtk-contrib.git git clone https://github.com/nilmtk/nilmbench.git cd nilmbench docker buildx build --load --build-context contrib=../nilmtk-contrib \ -f docker/Dockerfile.cuda -t nilmbench:cuda . docker run --rm --gpus all nilmbench:cuda doctor
from nilmtk.api import API
from nilmtk.disaggregate import Mean
from nilmtk_contrib.torch import (
Seq2PointTorch, Seq2Seq, RNN, WindowGRU, DAE,
NILMFormer, TCN, ConvLSTM, MSDC, Reformer, # +5 modern architectures
)
experiment = {
'power': {'mains': ['active'], 'appliance': ['active']},
'sample_rate': 60, # 60s real-time | 900s utility-scale
'appliances': ['fridge', 'washing machine', 'microwave', 'kettle'],
'methods': {
'NILMFormer': NILMFormer({'n_epochs': 50, 'batch_size': 256}),
'Seq2Point': Seq2PointTorch({'n_epochs': 50, 'batch_size': 256}),
'TCN': TCN({'n_epochs': 50, 'batch_size': 256}),
'Mean': Mean({}),
},
'train': {'datasets': {'UKDALE': {'path': 'ukdale.h5',
'buildings': {1: {'start_time': '2013-04-01', 'end_time': '2013-05-01'}}}}},
'test': {'datasets': {'UKDALE': {'path': 'ukdale.h5',
'buildings': {4: {'start_time': '2013-05-01', 'end_time': '2013-05-08'}}}},
'metrics': ['mae', 'f1score']},
}
results = API(experiment) # trains, tests & scores every model — cross-building (T2)# One provenance-recorded real-data smoke nilmbench run --task corrected-t1-redd --model Seq2Point \ --appliance fridge --seed 42 --epochs 3 --max-samples 8192 \ --sequence-length 299 --device cuda --results results/candidates # Generate the site artifacts only from immutable result bundles nilmbench leaderboard --results results/published \ --output leaderboard.json --csv leaderboard.csv
Unified PyTorch interface
Legacy Keras and TensorFlow models are re-implemented in PyTorch under a common training and inference interface.
Pinned environments
Docker images and uv lock files record the software environment used for each evaluation.
Common model contract
All architectures conform to the NILMTK Experiment API and can be selected through the experiment configuration.
Five additional architectures
TCN, ConvLSTM, MSDC, Reformer, and NILMFormer extend the evaluated temporal-modeling families.
A maintained benchmark for NILM
NILMBench2026 defines shared T1, T2, and T3 evaluation tasks, fixed data windows, and a public result format. New models and metrics use the same experiment API and can be evaluated on held-out buildings and datasets without changing the protocol.
from nilmtk.disaggregate import Disaggregator
class MyNILM(Disaggregator): # implement the interface
def __init__(self, params):
self.MODEL_NAME = 'MyNILM'
self.models = {}
self.sequence_length = params.get('sequence_length', 99)
self.n_epochs = params.get('n_epochs', 50)
def partial_fit(self, train_main, train_appliances, **kw):
... # train one PyTorch model per appliance
def disaggregate_chunk(self, test_main):
... # return {appliance: predicted_power}
return predictions
# Register it — the only line your experiment changes
experiment['methods']['MyNILM'] = MyNILM({'n_epochs': 50})# nilmtk/losses.py — define once, reference by name
def sae(app_gt, app_pred): # Signal Aggregate Error
return abs(app_pred.sum() - app_gt.sum()) / app_gt.sum()
def nep(app_gt, app_pred): # Normalized Error in Power
return (app_gt - app_pred).abs().sum() / app_gt.abs().sum()
# Score every model on it — just name it in the experiment
experiment['test']['metrics'] = ['mae', 'f1score', 'sae', 'nep']
# Same harness, same splits, same pre-processing for everyone —
# so reported gains are architectural, not implementation luck.Subclass Disaggregator
Implement the shared training and inference contract used by the reference models.
Run the fixed protocol
Use the published T1, T2, or T3 split and record the resulting data, runtime, and model provenance.
Open a pull request
Submit the implementation and validated result bundle for review and possible publication.
This page records the fixed BuildSys 2026 study. The living board grows as new provenance-checked result bundles are published, without rewriting the paper.
Sixteen architectures, four families
The evaluated suite spans baseline, recurrent, convolutional, attention-based, and hybrid designs behind one PyTorch API.
Recurrent & Hybrid
Fully Convolutional
Transformer-Based
Specialized NILM
Three tasks, increasing realism
Generalization is tested at three levels — from the best case (same home) to the hardest case (a different country). Performance falls off a cliff as the domain shifts.
Intra-Building
Train and test on disjoint time segments from the same home — temporal generalization, the best-case baseline.
Cross-Building
Train and test on different homes within a dataset. The realistic deployment test — and where most models break.
Cross-Dataset
Zero-shot transfer across countries & grids (110/230V). The hardest test — a symmetric, bidirectional collapse.
Three datasets, two countries, two grids
Selected for building diversity, temporal continuity, and open access — the properties needed to validate generalization. Single-building and pay-walled repositories are excluded.
REDD
USAUK-DALE
UKREFIT
UKExcluded: AMPds, iAWE, BLUED, DRED (single-building → cannot test cross-building generalization); PecanStreet (not freely available at full scale).
Explore the benchmark
Cells use the paper's color scale: green indicates better performance and red indicates worse performance. Bold marks the best and underline the second-best value per column. Scroll tables horizontally.
Representative failure cases
Predictions on UK-DALE, REDD, and REFIT illustrate the quantitative findings.
Four findings
No single model wins
The best architecture depends on the appliance's electrical signature: CNNs perform well on sparse, high-power events, whereas Transformers perform well on several multi-state loads.
Generalization is the main hurdle
Most models lose substantial accuracy on unseen buildings and datasets. The decrease from within-building to cross-building evaluation occurs in both transfer directions.
MAE is misleading for sparse events
Always predicting "off" yields a deceptively low MAE while missing every activation. Event metrics like F1 are essential for sparse, bursty appliances.
Efficiency ≠ accuracy
The trade-off is non-monotonic. Architectural inductive bias beats raw compute: a 69K-param TCN rivals the heavyweight NILMFormer on cross-dataset tasks.
Illustrative error difference. On the UK-DALE fridge, moving from RNN (~43 W MAE) to RNN Attn. Cl. (~18 W) corrects roughly 219 kWh / year of energy attribution — a difference that matters for consumer feedback and billing systems.
Research directions
The observed failure modes motivate six directions for improving cross-building and cross-dataset performance.
Domain adaptation
Study source/target alignment for transfer to new grids and appliances with limited labels.
Self-supervised pre-training
Evaluate masked modeling on unlabeled aggregate-power data before supervised fine-tuning.
Multi-task state classification
Combine power regression with probabilistic state and transition modeling for sparse loads.
Adaptive denormalization
Test local statistics and exogenous covariates, including time of day and occupancy, at low resolution.
Generative augmentation
Measure whether synthetic appliance signatures improve performance on held-out domains.
Maintained leaderboards
Publish provenance-checked results with explicit in-domain and out-of-domain cohorts.
Cite NILMBench2026
If you use the benchmark protocol, implementation, or reported results, please cite the paper.
@inproceedings{kuloor2026nilmbench,
title = {NILMBench2026: A Benchmark for Energy Disaggregation},
author = {Kuloor, Aayush and Singh, Anurag and Dhru, Harsh and Batra, Nipun},
booktitle = {Proceedings of the 13th ACM International Conference on Systems for
Energy-Efficient Buildings, Cities, and Transportation (BuildSys '26)},
year = {2026},
doi = {10.1145/3744256.3812587},
publisher = {ACM},
address = {Banff, AB, Canada}
}