nilmtk.datastore package

Submodules

nilmtk.datastore.csvdatastore module

class nilmtk.datastore.csvdatastore.CSVDataStore(*args, **kwargs)[source]

Bases: nilmtk.datastore.datastore.DataStore

static append(*args, **kwargs)[source]
Parameters:

key : str

value : pd.DataFrame

Notes

To quote the Pandas documentation for pandas.io.pytables.HDFStore.append: Append does not check if data being appended overlaps with existing data in the table, so be careful.

static close(*args, **kwargs)[source]
static elements_below_key(*args, **kwargs)[source]
Returns:list of strings
static get_timeframe(*args, **kwargs)[source]
Returns:nilmtk.TimeFrame of entire table after intersecting with self.window.
static load(*args, **kwargs)[source]
Parameters:

key : string, the location of a table within the DataStore.

cols : list of Measurements, optional

e.g. [(‘power’, ‘active’), (‘power’, ‘reactive’), (‘voltage’)] if not provided then will return all columns from the table.

sections : TimeFrameGroup; or list of nilmtk.TimeFrame objects;

or a pd.PeriodIndex, optional. Defines the time sections to load. If self.window is enabled then each section will be intersected with self.window.

n_look_ahead_rows : int, optional, defaults to 0

If >0 then each returned DataFrame will have a look_ahead property which will be a DataFrame of length n_look_ahead_rows of the data immediately in front of the data in the main DataFrame.

chunksize : int, optional
Returns:

generator of DataFrame objects

Each DataFrame is has extra attributes:

  • timeframe : TimeFrame of section intersected with self.window

  • look_ahead : pd.DataFrame:

    with n_look_ahead_rows rows. The first row will be for section.end. look_ahead stores data which appears on disk immediately after section.end; i.e. it ignores the next section.start.

Returns an empty DataFrame if no data is available for the specified section (or if the section.intersection(self.window) is empty).
Raises:

KeyError if `key` is not in store.
static load_metadata(*args, **kwargs)[source]
Parameters:

key : string, optional

if ‘/’ then load metadata for the whole dataset.
Returns:

metadata : dict
static open(*args, **kwargs)[source]
static put(*args, **kwargs)[source]
Parameters:

key : str

value : pd.DataFrame
static remove(*args, **kwargs)[source]
Parameters:

key : str

value : pd.DataFrame
static save_metadata(*args, **kwargs)[source]
Parameters:

key : string

metadata : dict

nilmtk.datastore.datastore module

class nilmtk.datastore.datastore.DataStore[source]

Bases: object

Provides a common interface to all physical data stores. Supports hierarchical stores.

The DataStore class lives in the bottom layer of NILMTK. It loads a single chunk at a time from physical location and returns a DataFrame.

  • Deals with: retrieving data from disk / network / direct from a meter
  • Optimised for: handling large amounts of data
  • Services it provides: delivering a generator of pd.DataFrames of data given a specific time span and columns
  • Totally agnostic about what the data ‘means’. It could be voltage, current, temperature, PIR readings etc.
  • could have subclasses for NILMTK HDF5, NILMTK CSV, Xively, Current Cost meters etc.
  • One DataStore per HDF5 file or folder or CSV files or Xively feed etc.

Attributes

window (nilmtk.TimeFrame) Defines the timeframe we are interested in.
append(key, value)[source]
Parameters:

key : str

value : pd.DataFrame

Notes

To quote the Pandas documentation for pandas.io.pytables.HDFStore.append: Append does not check if data being appended overlaps with existing data in the table, so be careful.

close()[source]
elements_below_key(key='/')[source]
Returns:list of strings
get_timeframe(key)[source]
Returns:nilmtk.TimeFrame of entire table after intersecting with self.window.
load(key, cols=None, sections=None, n_look_ahead_rows=0, chunksize=268435456)[source]
Parameters:

key : string, the location of a table within the DataStore.

cols : list of Measurements, optional

e.g. [(‘power’, ‘active’), (‘power’, ‘reactive’), (‘voltage’)] if not provided then will return all columns from the table.

sections : TimeFrameGroup; or list of nilmtk.TimeFrame objects;

or a pd.PeriodIndex, optional. Defines the time sections to load. If self.window is enabled then each section will be intersected with self.window.

n_look_ahead_rows : int, optional, defaults to 0

If >0 then each returned DataFrame will have a look_ahead property which will be a DataFrame of length n_look_ahead_rows of the data immediately in front of the data in the main DataFrame.

chunksize : int, optional
Returns:

generator of DataFrame objects

Each DataFrame is has extra attributes:

  • timeframe : TimeFrame of section intersected with self.window

  • look_ahead : pd.DataFrame:

    with n_look_ahead_rows rows. The first row will be for section.end. look_ahead stores data which appears on disk immediately after section.end; i.e. it ignores the next section.start.

Returns an empty DataFrame if no data is available for the specified section (or if the section.intersection(self.window) is empty).
Raises:

KeyError if `key` is not in store.
load_metadata(key='/')[source]
Parameters:

key : string, optional

if ‘/’ then load metadata for the whole dataset.
Returns:

metadata : dict
open()[source]
put(key, value)[source]
Parameters:

key : str

value : pd.DataFrame
remove(key, value)[source]
Parameters:

key : str

value : pd.DataFrame
save_metadata(key, metadata)[source]
Parameters:

key : string

metadata : dict
window[source]
nilmtk.datastore.datastore.convert_datastore(input_store, output_store)[source]
Parameters:

input_store : nilmtk.DataStore

output_store : nilmtk.DataStore
nilmtk.datastore.datastore.join_key(*args)[source]

Examples

>>> join_key('building1', 'elec', 'meter1')
'/building1/elec/meter1'

>>> join_key('/')
'/'

>>> join_key('')
'/'
nilmtk.datastore.datastore.write_yaml_to_file(metadata_filename, metadata)[source]

nilmtk.datastore.hdfdatastore module

class nilmtk.datastore.hdfdatastore.HDFDataStore(*args, **kwargs)[source]

Bases: nilmtk.datastore.datastore.DataStore

static append(*args, **kwargs)[source]
Parameters:

key : str

value : pd.DataFrame

Notes

To quote the Pandas documentation for pandas.io.pytables.HDFStore.append: Append does not check if data being appended overlaps with existing data in the table, so be careful.

static close(*args, **kwargs)[source]
static elements_below_key(*args, **kwargs)[source]
Returns:list of strings
static get_timeframe(*args, **kwargs)[source]
Returns:nilmtk.TimeFrame of entire table after intersecting with self.window.
static load(*args, **kwargs)[source]
Parameters:

key : string, the location of a table within the DataStore.

cols : list of Measurements, optional

e.g. [(‘power’, ‘active’), (‘power’, ‘reactive’), (‘voltage’)] if not provided then will return all columns from the table.

sections : TimeFrameGroup; or list of nilmtk.TimeFrame objects;

or a pd.PeriodIndex, optional. Defines the time sections to load. If self.window is enabled then each section will be intersected with self.window.

n_look_ahead_rows : int, optional, defaults to 0

If >0 then each returned DataFrame will have a look_ahead property which will be a DataFrame of length n_look_ahead_rows of the data immediately in front of the data in the main DataFrame.

chunksize : int, optional
Returns:

generator of DataFrame objects

Each DataFrame is has extra attributes:

  • timeframe : TimeFrame of section intersected with self.window

  • look_ahead : pd.DataFrame:

    with n_look_ahead_rows rows. The first row will be for section.end. look_ahead stores data which appears on disk immediately after section.end; i.e. it ignores the next section.start.

Returns an empty DataFrame if no data is available for the specified section (or if the section.intersection(self.window) is empty).
Raises:

KeyError if `key` is not in store.
static load_metadata(*args, **kwargs)[source]
Parameters:

key : string, optional

if ‘/’ then load metadata for the whole dataset.
Returns:

metadata : dict
static open(*args, **kwargs)[source]
static put(*args, **kwargs)[source]
Parameters:

key : str

value : pd.DataFrame
static remove(*args, **kwargs)[source]
Parameters:

key : str

value : pd.DataFrame
static save_metadata(*args, **kwargs)[source]
Parameters:

key : string

metadata : dict

nilmtk.datastore.key module

class nilmtk.datastore.key.Key(string=None, building=None, meter=None)[source]

Bases: object

A location of data or metadata within NILMTK.

Attributes

building (int)
meter (int)
utility (str)

Module contents