Vince Skahan <[email protected]> writes: > Currently drivers for gear supporting backfill are monolithic. They do > vendor-specific backfill then talk to the hardware for data, again > vendor-specific. Combining the two tasks into a monolithic driver can get > very complicated.
Sure, but is the backfill *concept* really hard? What I mean is consider that you have a python module that deals with the details, and you have calls like open to return an object, and then a get_record_indexes call to return a sequence of timestamps, and then get_record, that returns a dictionary. Then it's just doing that and matching up with the db and adding. It's not a lot of code, and it could be a common routine if the per-vendor modules are all subclasses. I assert that the huge amount of code in an online/backfill driver is not because of online/backfill but because it has code for two different interfaces each of which is complicated. > Ecowitt is a good example. Its backfill is pretty odd under the hood. The > driver part is even more complicated as they flood the market with more and > more sensors and their rather poorly documented APIs change over time. Sure, but that's all 'ecowitt is messy', rather than making it net easier to split along those lines. > Given weewx's requirement for 'some' driver and its support for optional > services, can the typically monolithic 'backfill + drive' driver can be > componetized ala: > > - (driver) component-X that does backfill via the mechanism the user > wants, if any > - (service) component-Y that does the periodic talking to the hardware > for sensor readings That would be weird. The same values (e.g. outTemp) would be sometimes coming from driver, and sometimes from service, and if you aren't missing anything, there's be no backfill archive record. It's really the same device and the same data, with just catchup on boot, like Davis. > Using ecowitt as an example: > > - you might have a gw3000 driver which backfills from SD card > - or you might have a driver that backfills from API query of the > ecowitt server > - or you might want a null driver that does nothing at all (no backfill) That's a great argument for having a backfill class definition and three implementations (in separate .py files) and then config to say which one you want, with the code that calls it being simple. Basically I'm saying that traditional module design tames the complexity, or at least encapsulates it and prevents extra spaghetti connections. > And then of course add a service to talk to the hardware or external API or > whatever for sensor data > > - you might be able to talk to hardware via the original gw1000 driver > - but other gear might require a different ecowitt driver to talk to the > hardware > - or you could query the vendor server for sensor data > - or you could get data via MQTTSubscribe > - and so on Aside from MQTTSubscribe, same answer - multiple modules that implement the "fetch online data" class. For MQTTSubscribe, there should be a module that gets all the data from MQTT and keeps it available in a pythonic (dict surely) way. Then the MQTTSubscribe service would be... a service that just uses this and injects database columns that aren't part of the "station". And a driver would have this as one of the choices. > Basically be able to split the two pieces (backfill + sensor data) into > two components. Componentize. Be able to break up monolithic things. > think ala-carte. Splitting the code is great, into modules that can be composed and used in various ways. But I do not believe that extending that split into driver/service is helpful. Looking at Home Assistant, which has a vast number of integrations (weewx equivalent of service, sort of, because HA has no concept of a single thing which is the main point), there's an event bus, entities, etc. which correspond to weewx's database and what should be assembled into LOOP and ARCHIVE. Then there are a vast number of integrations that bridge to any device one can. The core doesn't know anything about those devices, and when writing an integration you can ignore the rest. And then further the details of talking to that device are abstracted into an async python module that is not at all about HA. Pretending async/not divide did not exist (counterfactual but for the discussion), suppose one wrote an ecowitt online module that you can use to query the local API and return a dict. Then one could use that same code to write a HA integration that would basically call open and then 'get me the list of variables' and create entities, and then once/minute call read and use the dict to update the entities. Then the weather values would natively appear in HA. All of this with only writing the code to talk HTTP API and produce a dict of values once. And not implementing HTTP, but using the requests module probably (as the WLL driver does). -- You received this message because you are subscribed to the Google Groups "weewx-development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/weewx-development/rmipl79umra.fsf%40s1.lexort.com.
