Framework

Pecos contains the following modules

Module

Description

monitoring

Contains the PerformanceMonitoring class and individual quality control test functions that are used to run analysis

metrics

Contains metrics that describe the quality control analysis or compute quantities that might be of use in the analysis

io

Contains functions to load data, send email alerts, write results to files, and generate html reports

graphics

Contains functions to generate scatter, time series, and heatmap plots for reports

utils

Contains helper functions, including functions to convert time series indices from seconds to datetime

In addition to the modules listed above, Pecos also includes a pv module that contains metrics specific to photovoltaic analysis.

Object-oriented approach

Pecos includes a PerformanceMonitoring class which is the base class used to define the quality control analysis. This class stores:

  • Raw data

  • Translation dictionary (maps raw data column names to common names)

  • Time filter (excludes specific timestamps from analysis)

The class can be used to call quality control tests, including:

  • Check timestamps for missing, duplicate, and non-monotonic indexes

  • Check for missing data

  • Check for corupt data

  • Check for data outside expected range

  • Check for stagnant of abrupt changes in the data

  • Check for outliers

The class can be used to return:

  • Cleaned data (data that failed a test is replaced by NaN)

  • Boolean mask (indicates if data failed a test)

  • Summary of the quality control test results

The object-oriented approach is convenient when running a series of quality control tests and can make use of the translation dictionary and time filter across all tests. The cleaned data, boolean mask, and test results summary reflect results from all quality control tests.

When using the object-oriented approach, a PerformanceMonitoring object is created and methods are called using that object. The cleaned data, mask, and tests results can then be extracted from the PerformanceMonitoring object. These properties are updated each time a quality control test is run.

>>> pm = pecos.monitoring.PerformanceMonitoring()
>>> pm.add_dataframe(data)
>>> pm.check_range([-3,3])
>>> cleaned_data = pm.cleaned_data
>>> mask = pm.mask
>>> test_results = pm.test_results

Functional approach

The same quality control tests can also be run using individual functions. These functions generate a PerformanceMonitoring object under the hood and return:

  • Cleaned data

  • Boolean mask

  • Summary of the quality control test results

The functional approach is a convenient way to quickly get results from a single quality control tests.

When using the functional approach, data is passed to the quality control test function. All other augments match the object-oriented approach. The cleaned data, mask, and tests results can then be extracted from a resulting dictionary.

>>> results = pecos.monitoring.check_range(data, [-3,3])
>>> cleaned_data = results['cleaned_data']
>>> mask = results['mask']
>>> test_results = results['test_results']

Note, examples in the documentation use the object-oriented approach.