Forwarding HTTP to InfluxDB

Intro

For enabling Data export via HTTP, just forward GET requests to InfluxDB and let the transformation machinery handle all the rest.

Data export means access to raw data in different output formats as well as data plots.

Setup

This can be achieved by configuring a generic HTTP-to-InfluxDB forwarder application:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
; ------------------------------------------
; Family:   Protocol forwarder
; About:    Versatile protocol forwarder components for bridging the gap between
;           different data sinks, bus systems and serialization formats.
; ------------------------------------------
; Name:     http-to-influx forwarder
; Date:     June 2016
; About:    Suitable for data export from InfluxDB via HTTP GET requests.
; Channel:  Transport: HTTP
; Format:   CSV, JSON, HDF5, NetCDF, matplotlib, ggplot,
;           HTML, DataTables, dygraphs, Bokeh, Vega
; See also: https://getkotori.org/docs/applications/forwarders/http-to-influx.html
;           https://getkotori.org/docs/handbook/export/
; ------------------------------------------
; Description:
;
;         - Listen to HTTP GET requests
;         - Transform inbound parameters to InfluxDB query expression
;         - Query InfluxDB
;         - Respond with raw data or data plots
;
; Manual:   Please specify forwarding source and target parameters in uri format.
;
;           In the example below, the quadruple (``realm / network / gateway / node``)
;           will be transformed into::
;
;              database = {realm}_{network}
;              series   = {gateway}_{node}
;
;           which will be used for querying InfluxDB. All identifiers are sanitized before.
;
;           Results from this query will get routed through a powerful
;           data export machinery. The output format is determined
;           by the ``suffix`` parameter.
;
; Example:  When requesting http://localhost:24642/api/mqttkit-1/testdrive/area-42/node-1/data.txt?from=2016-06-23&to=2016-06-27,
;           the query expression against database "mqttkit_1_testdrive" will be::
;
;               SELECT * FROM area_42_node_1 WHERE time >= '2016-06-23 00:00:00.000' AND time <= '2016-06-27 23:59:59.999';
;
; ------------------------------------------

[mqttkit-1.influx-data-export]
enable          = true

type            = application
application     = kotori.io.protocol.forwarder:boot

realm           = mqttkit-1
source          = http:/api/{realm:mqttkit-1}/{network:.*}/{gateway:.*}/{node:.*}/{slot:(data|event)}.{suffix} [GET]
target          = influxdb:/{database}?measurement={measurement}
transform       = kotori.daq.intercom.strategies:WanBusStrategy.topology_to_storage,
                  kotori.io.protocol.influx:QueryTransformer.transform


; ------------------------------------------------
;   Notes about "source" and "target" parameters
; ------------------------------------------------

; Note that the "netloc" part (i.e. for specifying hostname/port)
; in directives "source" and "target" are omitted from these uris.
;
; Kotori currently will only listen to the default http port
; and query the default InfluxDB, both specified in the main
; configuration file, usually "/etc/kotori/kotori.ini".
;
; However, this might change in the future to enable spinning
; up HTTP listeners on arbitrary ports at runtime and to allow
; querying data from different InfluxDB instances.