Hiveeyes


About

The Hiveeyes project conceives a data collection platform for bee hive monitoring voluntarily operated by the beekeeper community.

Together with Mosquitto, InfluxDB, Grafana, mqttwarn and BERadio, Kotori powers the Hiveeyes system on swarm.hiveeyes.org as a data collection hub for a Berlin-based beekeeper collective.

Environment

Let’s have a look at the environment:

  • Arduino is a popular embedded computing platform used intensively here.
  • Telemetry data is transmitted from sensor nodes over RFM69/RFM95 radio links.
  • Telemetry data is forwarded and distributed over a wide area multi-tenancy communication bus based on MQTT.

System overview

MQTT- and HTTP-based data acquisition, storage and visualization

  • Kotori receives telemetry data from MQTT topic subscriptions. For details about the addressing scheme and topology, see Hiveeyes One topology.
  • The Open Hive Box uses a GPRSbee modem to do Data acquisition over HTTP.
  • Store measurements to the timeseries database.
  • Automatically create default Grafana panels for instant telemetry data visualization.
  • Detect events and anomalies on the telemetry data and emit appropriate signals.
// Hiveeyes WAN and backend
digraph hiveeyes_wan_mqtt {

    // Options
    rankdir=LR;
    ranksep=0.5;

    // Style
    //graph [splines=ortho];
    node [pin=true, shape="box", fontname="Verdana"];
    edge [fontname="Verdana"];


    /* MQTT */

    // Graph nodes represent system components
    "soc"           [label="SoC machine"];
    "mosquitto"     [label="Mosquitto"];
    "kotori"        [label="Kotori"];
    {rank=same; "influxdb"; "grafana"; "mqttwarn" };
    "influxdb"      [label="InfluxDB"];
    "grafana"       [label="Grafana"];
    "mqttwarn"      [label="mqttwarn", style=dashed];

    // Graph edges represent communication paths
    "soc"           -> "mosquitto"      [label="MQTT"];
    "mosquitto"     -> "kotori"         [label="MQTT"];
    "kotori"        -> "influxdb";
    "kotori"        -> "grafana";
    "kotori"        -> "mqttwarn";
    "influxdb"      -> "grafana";


    /* HTTP */

    // Graph nodes represent system components
    "avr"           [label="AVR MCU"];
    "gprsbee"       [label="GPRSbee"];

    // Graph edges represent communication paths
    "avr"           -> "gprsbee"        [label="UART"];
    "gprsbee"       -> "kotori"         [label="HTTP"];

}

Data export

There are different ways to get data out of Kotori, see Data export.

// Hiveeyes data export interface
digraph hiveeyes_export {

    // Options
    //rankdir=LR;
    ranksep=0.5;

    // Style
    //graph [splines=ortho];
    node [pin=true, shape="plaintext", fontname="Verdana"];
    edge [fontname="Verdana"];

    // Graph nodes represent system components
    "kotori"        [label="Kotori", shape="box"];
    "csv"           [label="CSV"];
    "json"          [label="JSON"];
    "xlsx"          [label="Excel XLSX"];
    "matplotlib"    [label="matplotlib"];
    "ggplot"        [label="ggplot"];
    "dygraphs"      [label="dygraphs"];

    // Graph edges represent communication paths
    "kotori"        -> "csv";
    "kotori"        -> "json";
    "kotori"        -> "xlsx";
    "kotori"        -> "matplotlib";
    "kotori"        -> "ggplot";
    "kotori"        -> "dygraphs";

}

Firmware builder

Using the Firmware builder, beekeepers can upload customized firmwares to their sensor and telemetry nodes derived from a “golden master” Generic firmware. Firmware images can be easily downloaded using HTTP.

// Hiveeyes Firmware Builder
digraph hiveeyes_firmware {

    // Options
    rankdir=LR;
    ranksep=0.5;

    // Style
    //graph [splines=ortho];
    node [pin=true, shape="box", fontname="Verdana"];
    edge [fontname="Verdana"];

    // Graph nodes represent system components
    "kotori"        [label="Kotori"];
    "firmware"      [label="Firmware HEX"];
    "avr"           [label="AVR MCU\nATmega328p"];

    // Graph edges represent communication paths
    "kotori"        -> "firmware"       [label="HTTP"];
    "firmware"      -> "avr"            [label="avrdude"];

}

Domain-specific features

  • Hiveeyes Daily
  • Schwarmalarm
  • Elektronische Stockkarte

Results

Intro

Data currently is measured at intervals of about 15 minutes.

In the following graphs the weight of the hive is shown with the light blue line. The temperature sensor inside the hive is shown in dark blue and the sensor at the air hole, which is in the sun most times, is yellow.

Spring 2016

Let’s have a look at the data. During the cold period at the end of April the beehive loses absolute weight because the bees have to live on their reserves. In contrast, the following two weeks were amazingly warm in May, so the worker bees are getting extremely busy.

From April 29 until May 13, the weight has increased by almost 14 Kg. On some days the beehive gained 2 Kg weigth due to pollen and nectar collecting.

It was the time of the fruit blossoms, so the cherry, pear and apple trees were in bloom in the nearby allotments.


The beehive gained a lot of weight on the 2nd of May. After six o’clock in the morning the temperature begins to rise, at 7:15 the first sun rays shine on the entrance hole. The first scouts leave the beehive at 7:45 and at 8:15 everyone is in the air. The beehive suddenly loses 120g since about 1200 bees are leaving to harvest.

At 9:20, the ratio between the arriving and departing bees reverses and the weight increases rapidly.

Throughout the day, bees are flying in and out, collecting nectar and pollen, explore and report. Even as the beehive is in the shadow from 16:00 and the temperature starts sinking, the bee colony continues to collect until 20:00.

1800g have been carried home. It was a good warm and important day, after the long period of winter and the cold snap at the end of April.

Schwarmalarm May 2016

This is a weight-loss event from The “Beutenkarl” scale recorded on 2016-05-20 between 10:11 and 10:26 hours CEST after a bee colony started swarming at the Hiveeyes Labs Beehive in Berlin Wedding, see also 2016-05-20 Schwarmalarm.

Details

Interfaces

Entrypoints to the platform running on swarm.hiveeyes.org as of 2016-01-29:

  • Mosquitto:

    uri:      mqtt://swarm.hiveeyes.org
    
  • Grafana:

    uri:      https://swarm.hiveeyes.org/grafana/
    username: hiveeyes
    password: Efocmunk
    

Platform operations

This section is about running the whole platform on your own hardware. Please be aware this is a work in progress. We are happy to receive valuable feedback for improving things gradually.

Install the platform

The most convenient way is by using Debian packages for all infrastructure services and Kotori, see Setup on Debian. After that, the service should have been automatically started by systemd so the system is ready to serve requests.

InfluxDB

For working directly with the InfluxDB API, please have a look at the InfluxDB Handbook.

Platform development

Want to dig even deeper? Read on my dear.

Setup

When developing on Kotori or for ad-hoc installations, you should follow the instructions for installing Kotori as Python package.

Run Kotori

In ad-hoc installations, or when turning off the systemd service, you might want to start Kotori interactively in the foreground:

/opt/kotori/bin/kotori --config /etc/kotori/kotori.ini --debug

Hacking

For getting your development sandbox up and running, please have a look at Hacking on Kotori.

Note

The source code of the Hiveeyes project is available at Hiveeyes at GitHub.

Under the hood

Please also have a look at Hiveeyes One topology and Wishlist.