On Friday, September 7, 2018 at 1:57:51 PM UTC-7, mwall wrote:
>
> if vince does not mind, i'd like to post some of our conversation here
> then continue it on this thread
>
>
Absolutely !
Some thoughts:
- generating JSON-formatted MQTT topics as input is to me the ideal
format
- it's easy to generate and subscribe to the one(s) you want
- it's easy to use/ignore the fields that you choose
- having one 'listener' process to subscribe to topics and get them to
influxdb is to me the ideal architecture
- that means only that listener needs to actually talk to influxdb
- that makes it easier to set up and easier to secure
- albeit at the risk of single point of failure of course
- minimizing the number of moving parts to receive/store/display the
data makes automation much simpler
- you could even cook up a few Docker containers to deploy the
server-side stuff !
- having the dashboard front-end visible, yet the
collector/listener/database back end hidden seems wisest security-wise
Re: formats and influxdb:
- I like the idea of every source of data being 'able' to have its own
database
- that means you could have different retention periods as needed.
More on that in a bit.....
- For a weather station, I'd do something ala the weewx/loop topic its
MQTT extension can publish
- for the VP2 weewx setup here, my experience is the MQTT extension
publishing weather/loop is the ideal for me
- for the WeatherFlow station here, I cooked up MQTT topics lining up
with what the hardware sends and how often:
- "obs/air" = temp/humidity/lightning - once every 60 secs
- "obs/sky" = wind/rain - once every 60 secs
- "status/hub" = inside hub status like RSSI values - once every
10 secs
- "status/air" and "status/sky" - outside device status like RSSI
and battery - once every 10 secs
- "obs/rapid_wind" = more frequent wind measurements - once every
3 secs
- For my here-and-there instrumentation I have a little nodeMCU setup
that sends temperature every 5 secs currently
- That is JSON-formatted ala { "sensorID": "nodeMCU_<chipid>",
"degF": 72.4 }
- - the intent is to have the same code on multiple like setups
around the house, with the sensor ID being the hardware id of the nodeMCU
card
- - that lets me use the same arduino code on each nodeMCU as long as
I know where I put which card around the house :-)
Re: how many databases, my experience is that grafana is good at tearing
apart which db stuff came from. That lets you do things like say "give me
outTemp from station1's db" as opposed to "give me outTemp as measured by
nodeMCU card-12345" and use common terminology everywhere. I also like
the idea of saving things with different retention periods, similar
notionally to the difference in how weewx keeps archive_day_rain (etc.)
summary tables vs. the bigger main archive table itself with the main
source data.
For a WeatherFlow station, I probably only care about obs/air and obs/sky
for weather, but I could see keeping summary data of status/air and
status/sky and status/hub type data. I personally haven't found a good
reason to watch the every-3-second rapid_wind needle moving on a display,
but that would be fun too. In that case I'd just have a couple gauges with
the current speed+direction and not even bother with a graph. Again to me
a separate short-retention-time db for that seems to make sense.
For a WeatherFlow station, I 'could' generate a weewx(ish) loop structure
with all the observations, and just fill in the data I have every time a
UDP message is received. In other words, have one big honkin' JSON data
structure and fill in the blanks as the data is available (it would take 2
messages minimum to get the weather data, 3 more for device status, 1 more
if you wanted rapid_wind too).
But there's no right way or wrong way I'm thinking, but generally the fewer
moving parts the better, and the fewer technologies to do job-xyz the
better...usually...
