The modification to AsyncReader is to add a write mode so weewx acts as a
timeserver. Triggered by sentinel "request_datetime".
The timeserver was not called during the gap at 9:15pm in the previous post
attachment and log. Search the log for "AsyncReader"
def run_read_mode(self):
# fixme I think this should be longer than the loop time for the
arduino
timeout = 12
# select(rlist,wlist,xlist,timeout); wait until ready for reading
(rlist);
ready,_,_ = select.select([self._fd], [], [], timeout)
if not ready:
return
line = self._fd.readline()
# fixme verbose
logdbg("read line = '%s'" % (line))
line = line.rstrip()
# all our files are binary
line = line.decode()
if not line:
return
if self._write_mode and (line == "request_datetime"):
self._read_mode = False
self._write_attempt = 0
return
# garberw begin -----------------------
# we are not processing any lines other than json so just keep only
json;
# this saves a little work until we get to PacketFactory.create;
# to see the debugging info comment this line out;
# PacketFactory.parse_json handles this line;
# PacketFactory.parse_text handles the other lines; we never use
that;
#if not line.startswith("{"):
# return
# garberw end -----------------------
# line = fudge_time()
self._queue.put(line)
def run_write_mode(self):
# fixme check this
timeout = 12
# select(rlist,wlist,xlist,timeout); wait until ready for writing
(wlist);
_,ready,_ = select.select([], [self._fd], [], timeout)
if not ready:
self._write_attempt += 1
if self._write_attempt >= self.WRITE_ATTEMPT_MAX:
logdbg("AsyncReader write timeout")
self._read_mode = True
return
# int(1677178029.12345) seconds since epoch
time_now_epoch = int(time.time())
# '1677178029\n'
time_str_epoch = '{}\n'.format(time_now_epoch)
# b'1677178029\n'
buf = bytes(time_str_epoch, 'ascii')
self._fd.write(buf)
self._fd.flush()
loginf("AsyncReader wrote timestamp={}
buf={}".format(time_now_epoch, buf))
self._read_mode = True
Maybe the problem is that the queue in AsyncReader gets too full before
ProcessManager.get_stdout()
yields all the lines to PacketFactory.create() which parses the json in
them and
pops each line off the list "lines". Sort of like flushing the input
buffer.
ProcessManager.get_stdout() yields the lines immediately if they contain a
timestamp matching
TS = re.compile('^\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d[\s]+')
maybe this would "flush the input buffer sooner" but this doesn't work for
me since I am using unix epoch time as an integer
not formatted like TS.
On Monday, April 24, 2023 at 10:31:21 PM UTC-7 William Garber wrote:
> jjj.log.gz is journalctl -b -u weewx_atlas.service
> typescript is an sql query of the database weewx_atlas.sdb showing a gap
> at 9:15pm.
> could it be garbage collecting caused the gap ?
> On Monday, April 24, 2023 at 10:12:15 PM UTC-7 William Garber wrote:
>
>> I have a weewx-sdr driver receiving data over usb from rtl-sdr from an
>> rtl-sdr radio receiver dongle. The radio receiver gets outdoor data from
>> an Acurite Atlas. The inside weather data comes from an arduino over usb.
>> The weewx server is on a raspberry pi. I modified the weewx-sdr driver
>> AsyncReader (in the beginning of the file) and ProcessManager. There is
>> now an AsyncReader thread for both the rtl433 ttyACM0 connection receiving
>> data over USB and for the arduino reading data over usb. These both share
>> the same queue where they save both indoor and outdoor lines of input data.
>> The driver only accepts data formatted as json and rejects the other
>> lines which are for debugging. There is a lot of extra discarded data.
>>
>> It looks like there simply be so much data that the weewx server skips
>> some of it sometimes. I get some records with nothing but None except for
>> rain.
>>
>> The archive interval is 5 minutes. The Atlas and the arduino both
>> generate data every 7 seconds or so. The arduino was much faster and I got
>> more gaps. When I slowed the arduino down to the speed of the Atlas there
>> were fewer gaps.
>>
>> Maybe I should just increase the archive interval? Or decrease it?
>> I don't know how to change the Atlas logging interval.
>>
>
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/6074a419-a368-4a9e-98eb-a08c266f24c6n%40googlegroups.com.
