Just one question please :-). Suppose the read of the arduino could
possibly take a relatively long time, and you want to have a timeout after
which it gives up and saves None/NULL for the indoor data.
What is the max timeout that would be reasonable relative to the archive
interval ? What happens if the delay goes past the end of the main archive
interval the event you are handling was in?
I think what is happening is I usually get very fast reads from the arduino
but once in a while I get a really slow one. Could be something
independent like me reading the database while weewx is trying to write it?
On Wednesday, April 26, 2023 at 11:24:36 PM UTC-7 gjr80 wrote:
> Way more complex than I think is needed. I would see you using the SDR
> driver as the your driver and nothing else. The example cited in the
> Customisation Guide does augment archive records rather than loop packets,
> in your case I would expect you are better served by augmenting loop
> packets. You don't necessarily need to augment every loop packet (if your
> loop packets are coming in every 7 seconds from SDR) but you need to make
> sure you get a good few in each archive period. You should have no need to
> deal with queues or accumulators in your service, the queue is a hangover
> of the threaded SDR driver and the WeeWX engine should handle everything to
> do with accumulators. At then end of the day you want to do two things (1)
> feed WeeWX with a loop packet stream - the SDR driver does that and (2)
> augment loop packets with Arduino data - your service does that. If you do
> these two things WeeWX will take care of generating archive records,
> reports etc.
>
> For the Arduino, can't you just read the Arduino data direct from the
> serial port and decode/parse/process it in your service? (there seems to be
> plenty of examples around the traps on this).
>
> The sort of service I had in mind is something like (untested of course):
>
> import weewx
> import weewx.units
>
> class ArduinoService(weewx.engine.StdService):
> """Data service to augment loop packets with data from an Arduino.
>
> We will augment loop packets with obs obtained from the Arduino. The
> WeeWX
> engine will take care of accumulating this data and generating archive
> records.
> """
>
> def __init__(self, engine, config_dict):
>
> # initialisie my parent class
> super(ArduinoService, self).__init__(engine, config_dict)
>
> # include whatever initialisation you need to speak to the Arduino
> < some python code to initialise any properties etc>
> # we will augment loop packets so bind ourself to any new loop
> packet
> # events
> self.bind(weewx.NEW_LOOP_PACKET, self.new_loop_packet)
>
> def new_loop_packet(self, event):
> """Augment a loop packet with data obtained from the Arduino."""
>
> # obtain a dict of data from the Arduino
> _ard_data_dict = self.get_arduino_data()
>
> # do we have any data
> if _ard_data_dict is not None:
> # convert the Arduino data to the same unit system as used in
> the
> # loop packet, first obtain a suitable Converter
> converter =
> weewx.units.StdUnitConverters[event.packet['usUnits']]
> # now convert the Arduino data to the same unit system as used
> in
> # the loop packet
> _conv_data = converter.convertDict(_ard_data_dict)
> # and finally update the loop packet with our converted data
> event.packet.update(_conv_data)
>
> def get_arduino_data():
> """Get a data dict from the Arduino.
>
> Obtain any data from the Arduino, parse the data and return as a
> dict
> using a known unit system. If no data could be obtained return
> None.
> """
>
> # some code to obtain the data from the Arduino, this code needs
> to be
> # carefully constructed to not block
> < python code to read data from the Arduino>
>
> if <we have Arduino data>:
> # create an empty dict with a pre-populated field usUnits to
> hold
> # our parsed Arduino data. The usUnits value is arbitrary, but
> must
> # be one of weewx.US (1), weewx.METRIC (16) or weewx.METRICWX
> (17).
> # Choose a conventient value, eg if the Arduino provides data
> in
> # metric use weewx.METRIC (16), if US customary use weewx.US,
> if a
> # mix of both then pick one and we will need to do some manual
> # conversion for the odd fields. In this case we will use
> # weewx.METRIC
> _data_dict = {'usunits': weewx.METRIC}
>
> # populate the dict, assuming here the Arduino is providing
> WeeWX
> # fields inTemp, inHumidity and barometer (could be 'pressure'
> not
> # 'barometer')
> _data_dict['inTemp'] = <some code to provide inside
> temperature in degree C>
> _data_dict['inHumidity'] = <some code to provide inside
> humidity in percent>
> _data_dict['barometer'] = <some code to provide barometer in
> hPa>
> # we have our data dict, now return it
> return _data_dict
> # we could not get any data from the Arduino so return None
> return None
>
> This is fairly bare bones and you might want to put in a few pieces of
> code to provide log output when debug = 1 (or 2 or 3) but hopefully you get
> the idea
>
> Gary
> On Thursday, 27 April 2023 at 15:08:42 UTC+10 william...@att.net wrote:
>
>> here is my service. In this version I took your advice and decided to
>> use the weewx-sdr driver completely unmodified and put all my stuff in a
>> service. Sorry the weewx-sdr driver (the original, not my version) reads
>> its input from AsyncReader (not my modification). That uses a thread.
>> That's what I was referring to. But since I already had my (slightly
>> modified) version of AsyncReader from before I spoke to you, I reused it in
>> the service (see the attachment). It has a queue. It stores lines of
>> input from the arduino over usb and discards any which are not json. This
>> is completely separate from the weewx-sdr and its AsyncReader. Plus this
>> AsyncReader can also be set up as a writer which enables me to have the
>> option of a simple time server on the raspberry pi to set the RTC on the
>> arduino if needed.
>>
>> Anyway I could always get rid of the AsyncReader and just use
>> serial.readline().
>>
>> Perhaps the example in the weewx customization guide is for a one-shot
>> read of the indoors sensors. Instead I made it loop over the lines of
>> input in the queue and use an accumulator to average them.
>>
>> The callback for NEW_ARCHIVE_RECORD for my service would (I think) be
>> called before StdArchive.new_archive_record(). My service only has access
>> to the event and event.record. Does it also have access to
>> StdArchive.old_accumulator ?
>> In StdArchive.new_archive_record(event) it gets the event which was
>> modified by my service so the record includes my data.
>>
>> *So my question is* what about old_accumulator in
>> StdArchive.new_archive_record() ? Is it strictly optional ? That
>> accumulator only has the outside data not the data added by my
>> new_archive_record() callback.
>> Of course my data is already averaged.
>> In StdArchive.new_archive_record() the event.record has more observation
>> types than the old_accumulator. Does this pose a problem ?
>>
>> what do you mean by augmenting? does this involve choosing
>> self.record_generation = hardware ?
>> On Wednesday, April 26, 2023 at 6:12:50 PM UTC-7 gjr80 wrote:
>>
>>> Yes, that section covers it fairly well. The two main things to watch
>>> are not delaying the main WeeWX engine loop and ensuring the data you add
>>> to the loop packet follows the unit system used in the loop packet.
>>> Delaying the main loop is often associated with accessing data via the
>>> internet or some other network. A common approach is to develop the data
>>> service without using its own thread, this makes debugging much easier. If
>>> delay is an issue you can move the service to its own thread later. Unit
>>> consistency is achieved by checking the usUnits field in the loop
>>> packet and then converting your service' data to the correct units. This is
>>> important for obs whose units that vary across unit systems (eg
>>> temperature) but not an issue if the same unit is used across all unit
>>> systems (eg wind direction). A robust, well written data service will not
>>> assume the loop packet always uses the same unit system.
>>>
>>> Gary
>>>
>>> On Thursday, 27 April 2023 at 01:24:52 UTC+10 william...@att.net wrote:
>>>
>>>> Thank you I expect you are referring to the Customization guide
>>>> "Customizing the WeeWX service engine; Adding a second data source"; I
>>>> will
>>>> try doing it that way; Thanks again;
>>>>
>>>> On Tuesday, April 25, 2023 at 10:16:18 PM UTC-7 gjr80 wrote:
>>>>
>>>>> I'm not sure if you are providing a running commentary or seeking
>>>>> help. If the latter then I have no idea where to start. Personally, I
>>>>> think
>>>>> your architecture is way too complex; you appear to be running a highly
>>>>> modified driver that seeks to amalgamate data from two sources. I imagine
>>>>> you will strike all sorts of corner cases depending on what arrives when.
>>>>> I
>>>>> would also question the utility of reading indoor obs every seven
>>>>> seconds,
>>>>> seems way too frequent to me. All told I doubt you are going to find too
>>>>> many folks here to help.
>>>>>
>>>>> Why not run a standard sdr driver to feed WeeWX with loop packets from
>>>>> the Atlas with a simple, non-threaded data service bound to the new loop
>>>>> packet arrival to read the 'indoor data' (I assume this is pressure,
>>>>> temperature and humidity) and augment the loop packet. Far more modular
>>>>> and
>>>>> easier to test/develop (and get help), you will be running a standard sdr
>>>>> driver, and since you are already getting your hands dirty modifying the
>>>>> sdr driver, writing a small data service to handle the Arduino input
>>>>> should
>>>>> be a walk in the park. If the Arduino is serially connected to the RPi
>>>>> you
>>>>> should not have great latency in accessing data, so a suitably short
>>>>> timeout on the serial reads should provide you with you indoor data
>>>>> without
>>>>> blocking the main WeeWX thread. Once proved, the serial read could be
>>>>> moved
>>>>> to a thread if you really had the need.
>>>>>
>>>>> Gary
>>>>>
>>>>
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/7e5a7de6-c535-458f-b81a-e927bd3a2572n%40googlegroups.com.
