pecos.io module

The io module contains functions to read/send data and write results to files/html reports.

pecos.io.read_campbell_scientific(filename, index_col='TIMESTAMP', encoding=None)

Read Campbell Scientific CSV file.

Parameters

• filename (string) – File name

• index_col (string, optional) – Index column name, default = ‘TIMESTAMP’

• encoding (string, optional) – Character encoding (i.e. utf-16)

Returns

pandas DataFrame – Data

pecos.io.send_email(subject, body, recipient, sender, attachment=None, host='localhost', username=None, password=None)

Send email using Python smtplib and email packages.

Parameters

• subject (string) – Subject text

• body (string) – Email body, in HTML or plain format

• recipient (list of string) – Recipient email address or addresses

• sender (string) – Sender email address

• attachment (string, optional) – Name of file to attach

• host (string, optional) – Name of email host (or host:port), default = ‘localhost’

• username (string, optional) – Email username for authentication

• password (string, optional) – Email password for authentication

pecos.io.write_metrics(metrics, filename='metrics.csv')

Write metrics file.

Parameters

• metrics (pandas DataFrame) – Data to add to the metrics file

• filename (string, optional) – File name. If the full path is not provided, the file is saved into the current working directory. By default, the file is named ‘metrics.csv’

Returns

string – filename

pecos.io.write_test_results(test_results, filename='test_results.csv')

Write test results file.

Parameters

• test_results (pandas DataFrame) – Summary of the quality control test results (pm.test_results)

• filename (string, optional) – File name. If the full path is not provided, the file is saved into the current working directory. By default, the file is named ‘test_results.csv’

Returns

string – filename

pecos.io.write_monitoring_report(data, test_results, test_results_graphics=None, custom_graphics=None, metrics=None, title='Pecos Monitoring Report', config=None, logo=False, im_width_test_results=1, im_width_custom=1, im_width_logo=0.1, encode=False, file_format='html', filename='monitoring_report.html')

Generate a monitoring report. The monitoring report is used to report quality control test results for a single system. The report includes custom graphics, performance metrics, and test results.

Parameters

• data (pandas DataFrame) – Data, indexed by time (pm.data)

• test_results (pandas DataFrame) – Summary of the quality control test results (pm.test_results)

• test_results_graphics (list of strings or None, optional) – Graphics files, with full path. These graphics highlight data points that failed a quality control test, created using pecos.graphics.plot_test_results(). If None, test results graphics are not included in the report.

• custom_graphics (list of strings or None, optional) – Custom files, with full path. Created by the user. If None, custom graphics are not included in the report.

• metrics (pandas Series or DataFrame, optional) – Performance metrics to add as a table to the monitoring report

• title (string, optional) – Monitoring report title, default = ‘Pecos Monitoring Report’

• config (dictionary or None, optional) – Configuration options, to be printed at the end of the report. If None, configuration options are not included in the report.

• logo (string, optional) – Graphic to be added to the report header

• im_width_test_results (float, optional) – Image width as a fraction of page size, for test results graphics, default = 1

• im_width_custom (float, optional) – Image width as a fraction of page size, for custom graphics, default = 1

• im_width_logo (float, optional) – Image width as a fraction of page size, for the logo, default = 0.1

• encode (boolean, optional) – Encode graphics in the html, default = False

• filename (string, optional) – File name. If the full path is not provided, the file is saved into the current working directory. By default, the file is named ‘monitoring_report.html’

Returns

string – filename

pecos.io.write_dashboard(column_names, row_names, content, title='Pecos Dashboard', footnote='', logo=False, im_width=250, datatables=False, encode=False, filename='dashboard.html')

Generate a dashboard. The dashboard is used to compare results across multiple systems. Each cell in the dashboard includes custom system graphics and metrics.

Parameters

• column_names (list of strings) – Column names listed in the order they should appear in the dashboard, i.e. [‘location1’, ‘location2’]

• row_names (list of strings) – Row names listed in the order they should appear in the dashboard, i.e. [‘system1’, ‘system2’]

• content (dictionary) –

  Dashboard content for each cell.

  Dictionary keys are tuples indicating the row name and column name, i.e. (‘row name’, ‘column name’), where ‘row name’ is in the list row_names and ‘column name’ is in the list column_names.

  For each key, another dictionary is defined that contains the content to be included in each cell of the dashboard. Each cell can contain text, graphics, a table, and an html link. These are defined using the following keys:

  • text (string) = text at the top of each cell

  • graphics (list of strings) = a list of graphics file names. Each file name includes the full path

  • table (string) = a table in html format, for example a table of performance metrics. DataFrames can be converted to an html string using df.to_html() or df.transpose().to_html(). Values in the table can be color coded using pandas Styler class.

  • link (dict) = a dictionary where keys define the name of the link and values define the html link (with full path)

  For example:

  content = {('row name', 'column name'): {
      'text': 'text at the top',
      'graphic': ['C:\\pecos\\results\\custom_graphic.png'],
      'table': df.to_html(),
      'link': {'Link to monitoring report': 'C:\\pecos\\results\\monitoring_report.html'}}
  

• title (string, optional) – Dashboard title, default = ‘Pecos Dashboard’

• footnote (string, optional) – Text to be added to the end of the report

• logo (string, optional) – Graphic to be added to the report header

• im_width (float, optional) – Image width in the HTML report, default = 250

• datatables (boolean, optional) – Use datatables.net to format the dashboard, default = False. See https://datatables.net/ for more information.

• encode (boolean, optional) – Encode graphics in the html, default = False

• filename (string, optional) – File name. If the full path is not provided, the file is saved into the current working directory. By default, the file is named ‘dashboard.html’

Returns

string – filename

pecos.io.device_to_client(config)

Read channels on modbus device, scale and calibrate the values, and store the data in a MySQL database. The inputs are provided by a configuration dictionary that describe general information for data acquisition and the devices.

Parameters

config (dictionary) – Configuration options, see Data acquisition