Telemetry

Models

class enhydris.telemetry.models.Telemetry

Information about how to automatically fetch data.

Telemetry has a OneToOneField to Station. Every minute Celery Beat runs a task that iterates through the records of Telemetry; for each record, if the time for fetching data has arrived, it fetches the data by calling the fetch() method.

Telemetry objects have the following attributes, properties and methods:

station: OneToOneField

The Station to which the record refers.

type: CharField

The type of the API, such as Adcon AddUPI or Metrica MeteoView2. See Telemetry API types for more information.

data_timezone: CharField

The time zone of the data, like Europe/Athens or Etc/GMT. Enhydris converts the timestamps from this time zone to UTC in order to store them.

To avoid ambiguity, it is recommended that stations do not switch to DST. However, if data_timezone is a time zone that switches to DST, Enhydris will handle it accordingly. It assumes that the time change occurs exactly when it is supposed to occur, not a few hours earlier or later. For the switch towards DST, things are simple. For the switch from DST to winter time, things are more complicated, because there’s an hour that appears twice. If the ambiguous hour occurs twice, Enhydris will usually do the correct thing; it will consider that the second occurence is after the switch and the first is before the switch. If according to the system’s clock the switch hasn’t occurred yet, any references to the ambiguous hour are considered to have occurred before the switch.

fetch_interval_minutes: PositiveSmallIntegerField

This can be, e.g., 60 to fetch data every 60 minutes, 1440 to fetch data once a day, etc.

fetch_offset_minutes: PositiveSmallIntegerField

If fetch_interval_minutes is 10 and fetch_offset_minutes is 2, then data will be fetched at :02, :12, :22, etc. If fetch_interval_minutes is 1440 and fetch_offset_minutes is 125, then data will be fetched every day at 02:05 in the morning. Generally fetch_offset_minutes counts from midnight UTC.

device_locator: string

The address of the remote API. Depending on the API, this may be the IP address, host name, or URL of the data collection device. Some APIs don’t have it at all, as the API is served by a given location regardless of which station it is (e.g. TheThingsNetwork). In such cases the attribute is left blank.

username: string

password: string

The username and password with which Enhydris will log on to the remote API. The password might actually be an API key, and the username might be an email, or it could be absent.

remote_station_id: string

If the API supports a single station (for that user), this should be blank. Some APIs provide access to many different stations; in that case, this is the id with which the station can be identified on the API (i.e. the remote_station_id on the API corresponds to the station of Enhydris).

additional_config: JSONField

If the specific telemetry type needs any additional configuration (e.g. serial interface parameters), it is stored in this attribute.

property is_due: Boolean

True if according to fetch_interval_minutes, fetch_offset_minutes, and the current system time it’s time to fetch data.

fetch() → None

Connects to the API, fetches the data, and inserts them to TimeseriesRecord.

class enhydris.telemetry.models.Sensor

Each record in that model represents a sensor in the API, and also holds the time series group to which the sensor corresponds, i.e. the time series group to which the data from the sensor are to be uploaded. If a sensor is to be ignored, then no row must exist in this table.

telemetry: ForeignKey

A foreign key to Telemetry.

sensor_id: string

An id with which the sensor can be identified in the API.

timeseries_group_id: string

A foreign key to TimeseriesGroup.

Telemetry API types

Each API type is one Python file in the enhydris/telemetry/types directory. The Python file must contain a TelemetryAPIClient class with all required functionality to retrieve data from the API.

When it starts, Enhydris scans the enhydris/telemetry/types directory and imports all Python files it contains. The result of this scanning goes to enhydris.telemetry.drivers.

enhydris.telemetry.drivers

A dictionary that contains all TelemetryAPIClient classes imported from the enhydris/telemetry/types directory. Each dictionary item maps the telemetry type’s slug (the base name of the Python file) to the TelemetryAPIClient class.

class TelemetryAPIClient(telemetry)

Should inherit from enhydris.telemetry.TelemetryAPIClientBase. The base class __init__() method initializes the object with a Telemetry object, which becomes the telemetry attribute.

TelemetryAPIClient classes may define the following attributes, methods and properties (some of them must be defined):

name: string

The name of the API, such as Adcon AddUPI or Metrica MeteoView2. This is what is stored in enhydris.telemetry.models.Telemetry.type.

device_locator_label: string

device_locator_help_text: string

The label and help that appears in the wizard for device_locator when the user is configuring telemetry; if absent, “Device URL” is used for the label and nothing is shown as help.

username_label: string

The label that appears in the wizard for username when the user is configuring telemetry; if absent, “Username” is used. For example, it can be “Email”.

password_label: string

The string that appears in the wizard for password when the user is configuring telemetry; if absent, “Password” is used. For example, it can be “API key”.

hide_device_locator: boolean

The default is False. Set it to True if that particular driver shouldn’t show the device locator (i.e. the URL or hostname or IP address of the device) in the connection data form. This is useful for APIs that are served from a well-known location for all stations, such as TheThingsNetwork.

hide_username: boolean

The default is False. Set it to True if that particular driver shouldn’t show the username in the connection data form (e.g. if the API just requires an API token).

hide_data_timezone: boolean

The default is False. Set it to True if that particular driver shouldn’t show the data timezone field in the data form. This is useful for APIs that are known to always provide timestamps in a given time zone.

If True, the timestamps in the return value of get_measurements() must be in UTC.

sensor_prompt: str

The default is something along the lines of “To which Enhydris time series does sensor X correspond?”. It is shown in the (usually) last step of the form. A different one can be specified here if the default doesn’t make much sense (e.g. if the remote system is a database that has time series that aren’t “sensors”).

ignore_sensor_prompt: str

To the question posed by sensor_prompt there’s a dropdown with possible answers. One of these answers is, by default, “Ignore this sensor”. A different phrasing can be specified here if the default doesn’t make much sense (e.g. if the remote system is a database that has time series that aren’t “sensors”).

forms: list

The forms that appear in the wizard. The base class provides a default that is suitable for most telemetry types, but it can be overridden (see influxdb.py for an example that overrides forms).

connect() → None

disconnect() → None

connect() initiates connection to the API and logs on. It should raise TelemetryError if something goes wrong. In some cases nothing needs to be done for connection (e.g. in the case of an HTTP API the key to which is a token that is passed to all requests).

disconnect() performs any required cleanup. In many cases no such cleanup is required. In some cases it is needed to logout, or a connection established by connect() might need to be closed.

Leave connect() and disconnect() unspecified if nothing needs to be done for connection or disconnection; the inherited methods do nothing.

get_stations() → dict

Retrieves and returns pairs of station ids and station names from the API. When the telemetry configuration wizard is shown to the user, at some point the user is asked which of the stations offered by the API corresponds to the Enhydris station; the stations offered by the API is what is returned by this method. If the API offers a single station, this method can be omitted (the base method returns None).

The station id is what is stored in remote_station_id; the station name is what is shown to the user in the wizard.

The method must raise TelemetryError if something goes wrong.

get_sensors() → dict

Retrieves and returns pairs of sensor ids and sensor names from the API. When the telemetry configuration wizard is shown to the user, at some point the user is asked which Enhydris time series group corresponds to each sensor of the API; the sensors available on the API is what is returned by this method.

The sensor id is what is stored in sensor_id; the sensor name is what is shown to the user in the wizard.

The method must raise TelemetryError if something goes wrong.

get_measurements(sensor_id, enhydris_timeseries_end_date) → StringIO

Reads data records for the sensor specified, starting with the first record whose timestamp is greater than enhydris_timeseries_end_date, and returns them in text format.

enhydris_timeseries_end_date is either None (meaning get all measurements since the beginning) or an aware datetime.

In order to avoid loading the server too much, this should not return more than a reasonable number of records, such as half a year or 20000 records. In the initial uploading of a backlog of records, it will thus take a few successive data fetches to bring the Enhydris time series up to date, but this is usually not a problem.

Enhydris can’t currently handle more than one records with timestamps within the same minute. However it’s OK for this method to return such records; the caller will ignore all except for the first one.

The method must raise TelemetryError if something goes wrong.

Exceptions

class enhydris.telemetry.TelemetryError

Telemetry API clients raise this exception if something goes wrong when connecting to the API. It derives from OSError.