WeeRT uses InfluxDB to store both LOOP and archive data. The data model is discussed <https://github.com/tkeffer/weert-js#weert-and-influxdb> in the README.
The measurement wxpackets is just the raw LOOP packets, one per row. Much like WeeWX, a timestamp and a unit system must be declared for each row, but, otherwise, there are no constraints. Partial packets cause no problems. The measurement wxrecords is LOOP packets aggregated over a regular time interval, typically 5 minutes. At first, I used InfluxDB's "subsampling" ability to create these records. It sort of worked, but it timestamped the newly created subsampled records with the *beginning* of the time interval, which required all sorts of data gymnastics to fix. I finally gave up and had WeeRT do its own aggregations. Regarding MQTT, I prefer a JSON object with a set of observations over single observations. Fewer packets, smaller InfluxDB storage requirements, and the packets are much more efficient (a timestamp and a unit system covers a set of observations, instead of just one observation). The only downside is that you can't precisely subscribe to a single observation, so in very narrowcast situations you transmit more data to subscribers than you might have to. OTOH, the reverse is also true for subscribers like the database, which want all the observations. I am not a big fan of exposing any database on the internet. There should always be some trusted app between it and the wild west out there. That could be an MQTT broker. I chose not to do that with WeeRT because it would be yet another application that has to be running to make the whole thing work. -tk On Fri, Sep 7, 2018 at 1:57 PM mwall <[email protected]> wrote: > vince and i have been having a discussion about how to get data from weewx > into grafana. i figured there are others who could benefit from this > discussion, and i'm sure there are some grafana/influx/mqtt users who could > contribute. > > vince's application is to compare data from three different types of > weather station hardware. > > my applications include tide monitoring (and prediction), tank level > monitoring (freshwater supplies), weather station comparisons (cc3000, > vantage, and others), and power monitoring (outback mate, victron vedirect, > sunny webbox, and enphase envoy) > > grafana is a wonderful tool for plotting data. it will talk to many > different database back-ends, but it is particularly nice to use with > influx. there is now a windrose plugin to grafana, so you can make > weather-specific dashboards. and of course there are all of the other > graphic analysis tools. > > influxdb is a nice way to store near-realtime data (e.g., loop data from > weewx) as well as weewx archive data. however, it is not obvious how one > should structure the data within influxdb. the general guideline is that > an influx measurement is analogous to an SQL table, a tag is analogous to > an indexed column, and a field is analogous to an unindex column. but that > still leaves many ways to structure the data. > > you can use the weewx-mqtt extension to easily feed loop and/or archive > data into an mqtt broker. then you can use a subscriber such as telegraf > to put the data into a timeseries data store such as influxdb. > > or you can use the weewx-influx extension to feed loop and/or archive data > directly into influxdb. > > but what is the best way to structure the mqtt messages? one observation > per message? or a json object in each message with a set of observations? > > and what is the best way to structure data in influxdb? > > if vince does not mind, i'd like to post some of our conversation here > then continue it on this thread > > m >
