Hiveeyes¶
About¶
The Hiveeyes project conceives a data collection platform for bee hive monitoring voluntarily operated by the beekeeper community.
Together with Mosquitto, InfluxDB, Grafana, mqttwarn and BERadio,
Kotori powers the Hiveeyes system
on swarm.hiveeyes.org as a data collection hub for a Berlin-based
beekeeper collective.
Environment¶
Let’s have a look at the environment:
Arduino is a popular embedded computing platform used intensively here.
Telemetry data is transmitted from sensor nodes over RFM69/RFM95 radio links.
Telemetry data is forwarded and distributed over a wide area multi-tenancy communication bus based on MQTT.
Sensor nodes¶
The Hiveeyes project for collaborative beehive monitoring uses two different kinds of sensor nodes.
Hiveeyes One sensor nodes transmit data to a gateway using RF, the gateway receives and decodes telemetry data from Bencode format, then forwards it to the MQTT broker in JSON. This is specified in BERadio specification and implemented in Python using the serial-to-mqtt forwarder beradio-python.
Open Hive sensor nodes based on the Adafruit HUZZAH ESP8266 directly send telemetry data to the MQTT broker in JSON format.
System overview¶
Radio link¶
BERadio handles the radio link communication based on RFM69 and RFM95. It receives data messages in Bencode format over radio, decodes them and forwards them to a serial interface to MQTT encoded with JSON.
![// Hiveeyes radio link data acquisition
digraph hiveeyes_radio_link {
// Options
rankdir=LR;
ranksep=0.5;
// Style
//graph [splines=ortho];
node [pin=true, shape="box", fontname="Verdana"];
edge [fontname="Verdana"];
// Graph nodes represent hardware node units
"sensor-1" [label="Sensor node 1"];
"sensor-2" [label="Sensor node 2"];
"sensor-N" [label="Sensor node N"];
"transceiver" [label="Protocol transceiver node"];
"gateway" [label="Gateway node"];
"soc" [label="SoC machine", style=dashed];
// Graph edges represent radio families and
// protocols spoken between node units.
"sensor-1" -> "transceiver" [label="RFM69"];
"sensor-2" -> "transceiver";
"sensor-N" -> "transceiver";
"transceiver" -> "gateway" [label="RFM95\n(LoRa)"];
"gateway" -> "soc" [label="UART", style=dashed];
// Draw a dotted line between sensor-2
// and sensor-N, but retain node positions.
"sensor-2" -> "sensor-N" [dir=none, style=dotted];
{rank=same; "sensor-1"; "sensor-2"; "sensor-N" };
}](../_images/graphviz-48525423de480c692476eb948d441fba3c58a946.png)
MQTT- and HTTP-based data acquisition, storage and visualization¶
Kotori receives telemetry data from MQTT topic subscriptions. For details about the addressing scheme and topology, see Hiveeyes One topology.
The Open Hive Box uses a GPRSbee modem to do HTTP.
Store measurements to the timeseries database.
Automatically create default Grafana panels for instant telemetry data visualization.
Detect events and anomalies on the telemetry data and emit appropriate signals.
![// Hiveeyes WAN and backend
digraph hiveeyes_wan_mqtt {
// Options
rankdir=LR;
ranksep=0.5;
// Style
//graph [splines=ortho];
node [pin=true, shape="box", fontname="Verdana"];
edge [fontname="Verdana"];
/* MQTT */
// Graph nodes represent system components
"soc" [label="SoC machine"];
"mosquitto" [label="Mosquitto"];
"kotori" [label="Kotori"];
{rank=same; "influxdb"; "grafana"; "mqttwarn" };
"influxdb" [label="InfluxDB"];
"grafana" [label="Grafana"];
"mqttwarn" [label="mqttwarn", style=dashed];
// Graph edges represent communication paths
"soc" -> "mosquitto" [label="MQTT"];
"mosquitto" -> "kotori" [label="MQTT"];
"kotori" -> "influxdb";
"kotori" -> "grafana";
"kotori" -> "mqttwarn";
"influxdb" -> "grafana";
/* HTTP */
// Graph nodes represent system components
"avr" [label="AVR MCU"];
"gprsbee" [label="GPRSbee"];
// Graph edges represent communication paths
"avr" -> "gprsbee" [label="UART"];
"gprsbee" -> "kotori" [label="HTTP"];
}](../_images/graphviz-55d9f489b6fcf6613951fb4247afe71de5f17096.png)
Data export¶
There are different ways to get data out of Kotori, see Data export.
![// Hiveeyes data export interface
digraph hiveeyes_export {
// Options
//rankdir=LR;
ranksep=0.5;
// Style
//graph [splines=ortho];
node [pin=true, shape="plaintext", fontname="Verdana"];
edge [fontname="Verdana"];
// Graph nodes represent system components
"kotori" [label="Kotori", shape="box"];
"csv" [label="CSV"];
"json" [label="JSON"];
"xlsx" [label="Excel XLSX"];
"matplotlib" [label="matplotlib"];
"ggplot" [label="ggplot"];
"dygraphs" [label="dygraphs"];
// Graph edges represent communication paths
"kotori" -> "csv";
"kotori" -> "json";
"kotori" -> "xlsx";
"kotori" -> "matplotlib";
"kotori" -> "ggplot";
"kotori" -> "dygraphs";
}](../_images/graphviz-de7979cd2b77cea6b3ff223a6eccc9f77f503589.png)
Firmware builder¶
Using the Firmware builder, beekeepers can upload customized firmwares to their sensor and telemetry nodes derived from a “golden master” Generic firmware. Firmware images can be easily downloaded using HTTP.
![// Hiveeyes Firmware Builder
digraph hiveeyes_firmware {
// Options
rankdir=LR;
ranksep=0.5;
// Style
//graph [splines=ortho];
node [pin=true, shape="box", fontname="Verdana"];
edge [fontname="Verdana"];
// Graph nodes represent system components
"kotori" [label="Kotori"];
"firmware" [label="Firmware HEX"];
"avr" [label="AVR MCU\nATmega328p"];
// Graph edges represent communication paths
"kotori" -> "firmware" [label="HTTP"];
"firmware" -> "avr" [label="avrdude"];
}](../_images/graphviz-747403f1f74f3a012c20bee39244b7484715462f.png)
Domain-specific features¶
Hiveeyes Daily
Schwarmalarm
Elektronische Stockkarte
Results¶
Intro¶
Data currently is measured at intervals of about 15 minutes.
In the following graphs the weight of the hive is shown with the light blue line. The temperature sensor inside the hive is shown in dark blue and the sensor at the air hole, which is in the sun most times, is yellow.
Spring 2016¶
Let’s have a look at the data. During the cold period at the end of April the beehive loses absolute weight because the bees have to live on their reserves. In contrast, the following two weeks were amazingly warm in May, so the worker bees are getting extremely busy.
From April 29 until May 13, the weight has increased by almost 14 Kg. On some days the beehive gained 2 Kg weigth due to pollen and nectar collecting.
It was the time of the fruit blossoms, so the cherry, pear and apple trees were in bloom in the nearby allotments.
The beehive gained a lot of weight on the 2nd of May. After six o’clock in the morning the temperature begins to rise, at 7:15 the first sun rays shine on the entrance hole. The first scouts leave the beehive at 7:45 and at 8:15 everyone is in the air. The beehive suddenly loses 120g since about 1200 bees are leaving to harvest.
At 9:20, the ratio between the arriving and departing bees reverses and the weight increases rapidly.
Throughout the day, bees are flying in and out, collecting nectar and pollen, explore and report. Even as the beehive is in the shadow from 16:00 and the temperature starts sinking, the bee colony continues to collect until 20:00.
1800g have been carried home. It was a good warm and important day, after the long period of winter and the cold snap at the end of April.
Schwarmalarm May 2016¶
This is a weight-loss event from The “Beutenkarl” scale recorded on 2016-05-20 between 10:11 and 10:26 hours CEST after a bee colony started swarming at the Hiveeyes Labs Beehive in Berlin Wedding, see also 2016-05-20 Schwarmalarm.
Details¶
Interfaces¶
Entrypoints to the platform running on swarm.hiveeyes.org as of 2016-01-29:
MQTT:
mqtt://swarm.hiveeyes.orgHTTP:
https://swarm.hiveeyes.org/apiGrafana:
https://swarm.hiveeyes.org/grafana/
Platform operations¶
This section is about running the whole platform on your own hardware. Please be aware this is a work in progress. We are happy to receive valuable feedback for improving things gradually.
Install the platform¶
The most convenient way is by using Debian packages for all infrastructure services and Kotori, see Setup on Debian and Ubuntu. After that, the service should have been automatically started by systemd so the system is ready to serve requests.
InfluxDB¶
For working directly with the InfluxDB API, please have a look at the InfluxDB.
Platform development¶
Want to dig even deeper? Read on my dear.
Setup¶
When developing on Kotori or for ad-hoc installations, you should follow the instructions for installing Kotori as Python package.
Run Kotori¶
In ad-hoc installations, or when turning off the systemd service, you might want to start Kotori interactively in the foreground:
/opt/kotori/bin/kotori --config /etc/kotori/kotori.ini --debug
Hacking¶
For getting your development sandbox up and running, please have a look at Setup development sandbox.
Note
The source code of the Hiveeyes project is available at Hiveeyes at GitHub.
Under the hood¶
Please also have a look at Hiveeyes One topology and Wishlist.