Backend Services

The following bundles are available in OpenEMS Backend.

This list is not complete.

1. Metadata File

OpenEMS Backend implementation for Metadata.

Allows you to configure multiple edges by a single JSON file. Using this bundle enables you to easily set up an OpenEMS Backend for testing purposes.

1.1. Example configuration file

{
	edges: {
		edge0: {
			comment: "Edge #0",
			apikey: "edge0",
			setuppassword: "abcdefgh"
		},
		edge1: {
			comment: "Edge #1",
			apikey: "edge1",
			setuppassword: "1234567"
		}
	}
}

Source Code

2. Metadata Odoo

Metadata OpenEMS Backend implementation for Odoo.

Enables you to manage multiple edges with one backend.

  • This bundle works with Odoo 15. For a seamless integration of Odoo and OpenEMS the open-source Odoo-OpenEMS module is needed.

Source Code

3. Timedata Aggregated InfluxDB

OpenEMS Backend implementation for aggregated InfluxDB access.

Bundle is needed for highly optimized environments with thousands of connected edges only. If you do not need that functionality you can skip this chapter.

A large number of OpenEMS edges connected to a single backend leads to a performance bottleneck. The bottleneck is mainly due to the access to the database. This Aggregated InfluxDB bundle in combination with the Timedata Influx bundle can mitigate this bottleneck. This happens by writing all data as usual via Timedata Influx into an Influx database (DB 1). Furthermore, all "aggregated data" (see edge datatypes) are written via the Aggregated InfluxDB bundle to another database (DB 2) located on another server. Thus DB 1 holds all data in high resolution, while DB 2 only holds very few data (5-minute averages for power values or only daily values for energy). Furthermore, a retention policy of e.g. 90 days is applied to DB 2. Thus DB 1 provides relatively slow access to a huge amount of data and DB 2 provides fast and responsive access to a relatively small database.

Within OpenEMS backend access to both databases is managed by the TimedataManager. The TimedataManager writes edge relevant data to all Timedata providers, whereas it reads data only from the first Timedata provider, which can deliver. Assuming correct configuration, TimedataManager first returns historical data from DB 2 (fast and responsive). And only in a very few cases it gets the data from DB 1 (probably slower).

The Timedata.AggregatedInfluxDB includes a class AllowedChannels.java. This implementation includes a hardcoded list of channels:

  • ALLOWED_AVERAGE_CHANNELS are used to calculate the 5min average values (e.g. average power values).

  • ALLOWED_CUMULATED_CHANNELS are used to calculate daily values (e.g. energy values).

This list must be adopted to a concrete usecase. It strongly depends on

  • your strategy to select component-IDs.

  • the components you are using within OpenEMS.

If you detect some widgets within your OpenEMS-UI which have empty values, it may have to do with an incorrect hardcoded list.

3.1. Configuration Setup

If you expect your backend to handle thousands of connected edges, the following configuration may provide a good start setup:

Create database and set retention policy:

Before starting the OpenEMS backend and after initially setting up the influx servers, you need to create the databases influx0 and aggregated0 and some retention policies:

##### Influx Server 1 #####
curl -i -XPOST http://127.0.0.1:8082/query --data-urlencode "q=CREATE DATABASE influx0"

##### Influx Server 2 #####
curl -i -XPOST http://127.0.0.1:8081/query --data-urlencode "q=CREATE DATABASE aggregated0"

curl -i -XPOST http://127.0.0.1:8081/query --data-urlencode "q=CREATE RETENTION POLICY rp_max ON aggregated0 DURATION 90d REPLICATION 1"
curl -i -XPOST http://127.0.0.1:8081/query --data-urlencode "q=CREATE RETENTION POLICY rp_avg ON aggregated0 DURATION 90d REPLICATION 1"

OpenEMS backend configuration:

  • bundle Timedata.AggregatedInfluxDB

    • ComponentID: timedata0

    • QueryLanguage: INFLUX_QL

    • URL: http://127.0.0.1:8081

    • Apikey: influxuser:influxpassword

    • Bucket: aggregated0

    • Retention policy for avg values: rp_avg

    • Retention policy for max values: rp_max

    • Measurement Avg: avg

    • Measurement for max values: Europe/Berlin=max

  • bundle Timedata.InfluxDB

    • ComponentID: timedata1

    • QueryLanguage: INFLUX_QL

    • URL: http://127.0.0.1:8082

    • Org: -

    • Apikey: influxuser:influxpassword

    • Bucket: influx0/autogen

    • Measurement: data

  • bundle Core Timedata-Manager

    • Timedata-IDs: [timedata0, timedata1]

  • Note the different Bucket namings, one with retention policy autogen and one without.

  • Make sure that your influx databases are located on two different servers or that you can easily move one database to another server later.

  • Be sure that the sequence within the Core Timedata-Manager is configured correctly - the component-ID of the Timedata.AggregatedInfluxDB comes first.

  • Influx Database must have at least Version 1.8.10.

  • Take care of your edge to backend connections on the edge side. Be sure to understand and select the right PersistencePriority for the different datatypes.

  • Remember the hardcoded list in AllowedChannels.java

  • Note that you can shift the load of your databases by choosing different retention policies.

  • Be sure you know exactly how Influx handles reading and writing data when using different retention policies on one database.

Source Code

4. Timedata Dummy

OpenEMS Backend dummy implementation for a timeseries database.

Using this bundle enables you to easily set up an OpenEMS Backend for testing purposes.

Source Code

5. Timedata InfluxDB

OpenEMS Backend implementation for InfluxDB.

InfluxDB is an open-source time series database (TSDB). It is optimized for storage and retrieval of time series data.

  • Influx Database must have at least Version 1.8.10.

  • The Timedata InfluxDB bundle is the preferred database to use with OpenEMS.

  • A great OpenSource tool to visualize InfluxDB data is Grafana. It helps you getting a deep insight into the data collected with OpenEMS.

  • Before connecting OpenEMS Backend to your influx database, create the database manually:

curl -i -XPOST http://127.0.0.1:8082/query --data-urlencode "q=CREATE DATABASE influx0"

Source Code

6. Timedata TimescaleDB

OpenEMS Backend implementation for TimescaleDB.

6.1. Setup TimescaleDB on Debian

See https://docs.timescale.com/install/latest/self-hosted/installation-debian/ for reference.

apt install gnupg postgresql-common apt-transport-https lsb-release wget

/usr/share/postgresql-common/pgdg/apt.postgresql.org.sh

curl -s https://packagecloud.io/install/repositories/timescale/timescaledb/script.deb.sh | bash

apt install timescaledb-2-postgresql-14 timescaledb-tools

timescaledb-tune --quiet --yes

# Open port 5432
vim /etc/postgresql/14/main/postgresql.conf
    listen_addresses = '*'
    data_directory = '/opt/postgresql'
vim /etc/postgresql/14/main/pg_hba.conf
    host    all             all              0.0.0.0/0                       md5

# Firewall
ufw allow to any port 5432

# Apply configuration changes
systemctl restart postgresql

cd /tmp
su postgres -c psql

CREATE DATABASE data;

CREATE USER openems WITH ENCRYPTED PASSWORD 'password';

GRANT ALL PRIVILEGES ON DATABASE data TO openems;

# Use 'data' db
\c data

CREATE EXTENSION IF NOT EXISTS timescaledb;
CREATE EXTENSION IF NOT EXISTS tablefunc;

exit

Source Code