NILMBench2026

Disaggregation has innovations — but no reproducible yardstick

The first deployment-aware NILM benchmark

Foundational toolkits standardized data parsers and baselines, but overlooked the constraints that decide real-world adoption. NILMBench2026 adds the first rigorous evaluation of efficiency, utility-scale resolution, and cross-domain generalization — on a modernized, reproducible software stack.

Reproduce every result in three commands

NILMBench2026 is the benchmark runner around NILMTK data and nilmtk-contrib models. It freezes real-data protocols, provenance and leaderboard generation behind one CLI. Use uv for development or build the pinned CPU/CUDA container family. Pick a tab.

# Python 3.11 benchmark environment
uv venv --python 3.11
source .venv/bin/activate
uv pip install "nilmbench[benchmark] @ git+https://github.com/nilmtk/nilmbench.git"
nilmbench list

# Until the first public image release, build the pinned CUDA runner locally
git clone https://github.com/nilmtk/nilmtk-contrib.git
git clone https://github.com/nilmtk/nilmbench.git
cd nilmbench
docker buildx build --load --build-context contrib=../nilmtk-contrib \
  -f docker/Dockerfile.cuda -t nilmbench:cuda .
docker run --rm --gpus all nilmbench:cuda doctor

from nilmtk.api import API
from nilmtk.disaggregate import Mean
from nilmtk_contrib.torch import (
    Seq2PointTorch, Seq2Seq, RNN, WindowGRU, DAE,
    NILMFormer, TCN, ConvLSTM, MSDC, Reformer,   # +5 modern architectures
)

experiment = {
  'power':       {'mains': ['active'], 'appliance': ['active']},
  'sample_rate': 60,                          # 60s real-time  |  900s utility-scale
  'appliances':  ['fridge', 'washing machine', 'microwave', 'kettle'],
  'methods': {
      'NILMFormer': NILMFormer({'n_epochs': 50, 'batch_size': 256}),
      'Seq2Point':  Seq2PointTorch({'n_epochs': 50, 'batch_size': 256}),
      'TCN':        TCN({'n_epochs': 50, 'batch_size': 256}),
      'Mean':       Mean({}),
  },
  'train': {'datasets': {'UKDALE': {'path': 'ukdale.h5',
            'buildings': {1: {'start_time': '2013-04-01', 'end_time': '2013-05-01'}}}}},
  'test':  {'datasets': {'UKDALE': {'path': 'ukdale.h5',
            'buildings': {4: {'start_time': '2013-05-01', 'end_time': '2013-05-08'}}}},
            'metrics': ['mae', 'f1score']},
}

results = API(experiment)   # trains, tests & scores every model — cross-building (T2)

# One provenance-recorded real-data smoke
nilmbench run --task corrected-t1-redd --model Seq2Point \
  --appliance fridge --seed 42 --epochs 3 --max-samples 8192 \
  --sequence-length 299 --device cuda --results results/candidates

# Generate the site artifacts only from immutable result bundles
nilmbench leaderboard --results results/published \
  --output leaderboard.json --csv leaderboard.csv

Built to become NILM's ImageNet

Benchmarks move fields. ImageNet moved vision, GLUE moved language, the KDD Cup moved applied ML. NILM has never had a shared, sealed, generalization-first leaderboard — NILMBench2026 is engineered to be exactly that. Add a model in one class, add a metric in one function, and submit to a public board scored on held-out buildings and datasets.

from nilmtk.disaggregate import Disaggregator

class MyNILM(Disaggregator):            # implement the interface
    def __init__(self, params):
        self.MODEL_NAME = 'MyNILM'
        self.models = {}
        self.sequence_length = params.get('sequence_length', 99)
        self.n_epochs = params.get('n_epochs', 50)

    def partial_fit(self, train_main, train_appliances, **kw):
        ...    # train one PyTorch model per appliance

    def disaggregate_chunk(self, test_main):
        ...    # return {appliance: predicted_power}
        return predictions

# Register it — the only line your experiment changes
experiment['methods']['MyNILM'] = MyNILM({'n_epochs': 50})

# nilmtk/losses.py — define once, reference by name
def sae(app_gt, app_pred):              # Signal Aggregate Error
    return abs(app_pred.sum() - app_gt.sum()) / app_gt.sum()

def nep(app_gt, app_pred):              # Normalized Error in Power
    return (app_gt - app_pred).abs().sum() / app_gt.abs().sum()

# Score every model on it — just name it in the experiment
experiment['test']['metrics'] = ['mae', 'f1score', 'sae', 'nep']

# Same harness, same splits, same pre-processing for everyone —
# so reported gains are architectural, not implementation luck.

This page records the fixed BuildSys 2026 study. The living board grows as new provenance-checked result bundles are published, without rewriting the paper.

Sixteen architectures, four families

From foundational baselines to current state-of-the-art, spanning recurrent, convolutional, attention-based, and hybrid designs — integrated behind one unified PyTorch API.

Three tasks, increasing realism

Generalization is tested at three levels — from the best case (same home) to the hardest case (a different country). Performance falls off a cliff as the domain shifts.

Three datasets, two countries, two grids

Selected for building diversity, temporal continuity, and open access — the properties needed to validate generalization. Single-building and pay-walled repositories are excluded.

Excluded: AMPds, iAWE, BLUED, DRED (single-building → cannot test cross-building generalization); PecanStreet (not freely available at full scale).

Explore the benchmark

Cells are heat-mapped exactly as in the paper — green is good, red is bad. Bold = best, underline = second-best per column. Scroll tables horizontally.

What the failures actually look like

Real predictions on real UK-DALE / REDD / REFIT traces — the receipts behind the headline findings.

The task, on real data. A UK-DALE site meter (top) and NILMFormer's appliance estimates against ground truth — a periodic fridge (middle) and a sparse, high-power load (bottom). Within a building, modern models track signatures well.

Generalization is the wall. NILMFormer on a REFIT television. Same-building (bottom) the model tracks the dynamic signature; on an unseen television (top) it collapses into spurious spikes — it memorized one device's electrical fingerprint, not the abstract concept of "television".

MAE is misleading. On the sparse REFIT microwave, NILMFormer, Seq2Seq, RNN and BERT all miss every 1200 W spike — yet score a low MAE by predicting "off". Event metrics (F1) are essential.

Why static splits. Mains traces across REDD buildings show severe gaps and disjoint coverage — test homes like B6 lack data in the training window, ruling out naive rotation-based cross-validation.

Four takeaways for the community

Closing the generalization gap

Based on the failure modes the benchmark surfaces, we chart six directions to move NILM from controlled accuracy to deployable robustness.

Cite NILMBench2026

If the benchmark or modernized toolkit helps your research, please cite the paper.

@inproceedings{kuloor2026nilmbench,
  title     = {NILMBench2026: A Benchmark for Energy Disaggregation},
  author    = {Kuloor, Aayush and Singh, Anurag and Dhru, Harsh and Batra, Nipun},
  booktitle = {Proceedings of the 13th ACM International Conference on Systems for
               Energy-Efficient Buildings, Cities, and Transportation (BuildSys '26)},
  year      = {2026},
  doi       = {10.1145/3744256.3812587},
  publisher = {ACM},
  address   = {Banff, AB, Canada}
}