Tom, Thanks for the pointer. That is what I am attempting to do. Here is what I THINK is my problem. See 4.d below. 1. A new loop packet is generated. 2. StdEngine fires a NEW_LOOP event 3. MyService handles the NEW_LOOP event. a. It sees that a prior value is less then the current value in the loop packet. b. The prior value is added to the loop packet as min_value. 4. StdArchive handles the NEW_LOOP event. a. It tries to add it to the accumulator. b. The accumulator throws OutOfSpan exception c. StdArchive creates a 'new' accumulator for the next archive period. d. StdArchive adds the loop packet (which has been updated with prior loop packet data) to the accumulator, 5. StdEngine fires the CHECK_LOOP event. 6. StdArchive handles the CHECK_LOOP. a. The dateTime in the loop packet is greater than the end of the archive period. b. StdArchive fires the END_ARCHIVE_PERIOD event. c. The dateTime is not greater than the end of the archive period + the archive delay. 7. MyService handles the END_ARCHIVE_PERIOD. a. Min values are reset to None, etc. 8. Eventually a packet with dateTime >= the end of archive period + archive delay is generated. a. This causes StdArchive (in CHECK_LOOP) to raise the BREAKLOOP exception b. StdEngine handles the exception, raising POST_LOOP; allowing things like reports to run etc. rich
On Sunday 22 September 2024 at 11:00:00 UTC-4 Tom Keffer wrote: > Take a look at the Vantage device driver. In addition to a standard device > driver, it includes a service, VantageService, that participates in the > weewxd event loop. It binds to STARTUP, NEW_LOOP_PACKET, and > END_ARCHIVE_PERIOD. The first two are used to reset wind gust values, the > last to update the archive record with the max wind seen. > > I think this is pretty similar to what you're trying to do. > > -tk > > On Saturday, September 21, 2024 at 5:01:26 PM UTC-7 [email protected] > wrote: > >> >> Goal: Capture the min value and time of lightning strikes. >> >> I started out binding to the PRE_LOOP and NEW_LOOP_PACKET events. When >> the PRE_LOOP event is handled, the min value and time were reset. In the >> handling of the NEW_LOOP_PACKET event, the comparison is done and the loop >> packet is updated as necessary. I 'quickly' realized that PRE_LOOP and >> POST_LOOP events have nothing to do with the archive period. >> >> The END_ARCHIVE_PERIOD event looked promising. Resetting of the value and >> time could be done here. But when I analyze the order of event firing, this >> is what I am seeing. >> 2024-09-21 11:44:59 weewxd[4036220] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 11:44:59 weewxd[4036220] DEBUG user.observationtime: RMB: >> packet dateTime: 1726933499 2024-09-21 11:44:59 >> 2024-09-21 11:44:59 weewxd[4036220] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 11:44:59 weewxd[4036220] DEBUG user.observationtime: RMB: >> packet dateTime: 1726933499 2024-09-21 11:44:59 >> 2024-09-21 11:45:01 weewxd[4036220] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 11:45:01 weewxd[4036220] DEBUG user.observationtime: RMB: >> packet dateTime: 1726933502 2024-09-21 11:45:02 >> 2024-09-21 11:45:01 weewxd[4036220] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 11:45:01 weewxd[4036220] DEBUG user.observationtime: RMB: >> packet dateTime: 1726933502 2024-09-21 11:45:02 >> 2024-09-21 11:45:01 weewxd[4036220] DEBUG user.observationtime: RMBx: In >> end_archive_period >> 2024-09-21 11:45:04 weewxd[4036220] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 11:45:04 weewxd[4036220] DEBUG user.observationtime: RMB: >> packet dateTime: 1726933504 2024-09-21 11:45:04 >> This will result in the first packet (1726933502) of the archive period >> using the data from the previous archive period. >> >> Flipping the order in engine.py to the following: >> for packet in self.console.genLoopPackets(): >> # Allow services to break the loop by throwing >> # an exception: >> self.dispatchEvent(weewx.Event(weewx.CHECK_LOOP, packet=packet)) >> >> # Package the packet as an event, then dispatch it. >> self.dispatchEvent(weewx.Event(weewx.NEW_LOOP_PACKET, >> packet=packet)) >> >> I would have expected to see the last packet in an archive period not >> being processed. But, I was not seeing that. So I added a bit more logging >> to try to understand what was going on. This is what I saw. Sorry, it is a >> bit long, I made a couple of comments with <---. >> >> 2024-09-21 19:39:58 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 19:39:58 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726961999 2024-09-21 19:39:59 >> 2024-09-21 19:39:58 weewxd[235236] DEBUG weewx.engine: RMBx: In >> check_loop (StdArchive) packet dateTime: 1726961999 end_archive_delay_ts: >> 1726962035 1726962000 35 >> 2024-09-21 19:39:58 weewxd[235236] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 19:39:58 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726961999 2024-09-21 19:39:59 >> 2024-09-21 19:39:58 weewxd[235236] DEBUG weewx.engine: RMBx: in >> new_loop_packet (StdArchive) >> 2024-09-21 19:40:01 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 19:40:01 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962001 2024-09-21 19:40:01 >> 2024-09-21 19:40:01 weewxd[235236] DEBUG weewx.engine: RMBx: In >> check_loop (StdArchive) packet dateTime: 1726962001 end_archive_delay_ts: >> 1726962035 1726962000 35 >> 2024-09-21 19:40:01 weewxd[235236] DEBUG user.observationtime: RMBx: In >> end_archive_period <--- END_ARCHIVE_PERIOD event must have fired >> 2024-09-21 19:40:01 weewxd[235236] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 19:40:01 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962001 2024-09-21 19:40:01 <--- And the >> NEW_LOOP_PACKET has fired afterward >> 2024-09-21 19:40:01 weewxd[235236] DEBUG weewx.engine: RMBx: in >> new_loop_packet (StdArchive) >> 2024-09-21 19:40:03 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop <--- processing of even packets continues >> 2024-09-21 19:40:03 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962004 2024-09-21 19:40:04 >> 2024-09-21 19:40:03 weewxd[235236] DEBUG weewx.engine: RMBx: In >> check_loop (StdArchive) packet dateTime: 1726962004 end_archive_delay_ts: >> 1726962035 1726962300 35 >> 2024-09-21 19:40:03 weewxd[235236] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 19:40:03 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962004 2024-09-21 19:40:04 >> 2024-09-21 19:40:03 weewxd[235236] DEBUG weewx.engine: RMBx: in >> new_loop_packet (StdArchive) >> 2024-09-21 19:40:06 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 19:40:06 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962006 2024-09-21 19:40:06 >> 2024-09-21 19:40:06 weewxd[235236] DEBUG weewx.engine: RMBx: In >> check_loop (StdArchive) packet dateTime: 1726962006 end_archive_delay_ts: >> 1726962035 1726962300 35 >> 2024-09-21 19:40:06 weewxd[235236] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 19:40:06 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962006 2024-09-21 19:40:06 >> 2024-09-21 19:40:06 weewxd[235236] DEBUG weewx.engine: RMBx: in >> new_loop_packet (StdArchive) >> 2024-09-21 19:40:08 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 19:40:08 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962009 2024-09-21 19:40:09 >> 2024-09-21 19:40:08 weewxd[235236] DEBUG weewx.engine: RMBx: In >> check_loop (StdArchive) packet dateTime: 1726962009 end_archive_delay_ts: >> 1726962035 1726962300 35 >> 2024-09-21 19:40:08 weewxd[235236] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 19:40:08 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962009 2024-09-21 19:40:09 >> 2024-09-21 19:40:08 weewxd[235236] DEBUG weewx.engine: RMBx: in >> new_loop_packet (StdArchive) >> 2024-09-21 19:40:11 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 19:40:11 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962011 2024-09-21 19:40:11 >> 2024-09-21 19:40:11 weewxd[235236] DEBUG weewx.engine: RMBx: In >> check_loop (StdArchive) packet dateTime: 1726962011 end_archive_delay_ts: >> 1726962035 1726962300 35 >> 2024-09-21 19:40:11 weewxd[235236] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 19:40:11 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962011 2024-09-21 19:40:11 >> 2024-09-21 19:40:11 weewxd[235236] DEBUG weewx.engine: RMBx: in >> new_loop_packet (StdArchive) >> 2024-09-21 19:40:13 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 19:40:13 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962014 2024-09-21 19:40:14 >> 2024-09-21 19:40:13 weewxd[235236] DEBUG weewx.engine: RMBx: In >> check_loop (StdArchive) packet dateTime: 1726962014 end_archive_delay_ts: >> 1726962035 1726962300 35 >> 2024-09-21 19:40:13 weewxd[235236] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 19:40:13 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962014 2024-09-21 19:40:14 >> 2024-09-21 19:40:13 weewxd[235236] DEBUG weewx.engine: RMBx: in >> new_loop_packet (StdArchive) >> 2024-09-21 19:40:16 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 19:40:16 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962016 2024-09-21 19:40:16 >> 2024-09-21 19:40:16 weewxd[235236] DEBUG weewx.engine: RMBx: In >> check_loop (StdArchive) packet dateTime: 1726962016 end_archive_delay_ts: >> 1726962035 1726962300 35 >> 2024-09-21 19:40:16 weewxd[235236] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 19:40:16 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962016 2024-09-21 19:40:16 >> 2024-09-21 19:40:16 weewxd[235236] DEBUG weewx.engine: RMBx: in >> new_loop_packet (StdArchive) >> 2024-09-21 19:40:18 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 19:40:18 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962019 2024-09-21 19:40:19 >> 2024-09-21 19:40:18 weewxd[235236] DEBUG weewx.engine: RMBx: In >> check_loop (StdArchive) packet dateTime: 1726962019 end_archive_delay_ts: >> 1726962035 1726962300 35 >> 2024-09-21 19:40:18 weewxd[235236] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 19:40:18 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962019 2024-09-21 19:40:19 >> 2024-09-21 19:40:18 weewxd[235236] DEBUG weewx.engine: RMBx: in >> new_loop_packet (StdArchive) >> 2024-09-21 19:40:21 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 19:40:21 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962021 2024-09-21 19:40:21 >> 2024-09-21 19:40:21 weewxd[235236] DEBUG weewx.engine: RMBx: In >> check_loop (StdArchive) packet dateTime: 1726962021 end_archive_delay_ts: >> 1726962035 1726962300 35 >> 2024-09-21 19:40:21 weewxd[235236] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 19:40:21 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962021 2024-09-21 19:40:21 >> 2024-09-21 19:40:21 weewxd[235236] DEBUG weewx.engine: RMBx: in >> new_loop_packet (StdArchive) >> 2024-09-21 19:40:23 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 19:40:23 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962024 2024-09-21 19:40:24 >> 2024-09-21 19:40:23 weewxd[235236] DEBUG weewx.engine: RMBx: In >> check_loop (StdArchive) packet dateTime: 1726962024 end_archive_delay_ts: >> 1726962035 1726962300 35 >> 2024-09-21 19:40:23 weewxd[235236] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 19:40:23 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962024 2024-09-21 19:40:24 >> 2024-09-21 19:40:23 weewxd[235236] DEBUG weewx.engine: RMBx: in >> new_loop_packet (StdArchive) >> 2024-09-21 19:40:26 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 19:40:26 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962026 2024-09-21 19:40:26 >> 2024-09-21 19:40:26 weewxd[235236] DEBUG weewx.engine: RMBx: In >> check_loop (StdArchive) packet dateTime: 1726962026 end_archive_delay_ts: >> 1726962035 1726962300 35 >> 2024-09-21 19:40:26 weewxd[235236] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 19:40:26 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962026 2024-09-21 19:40:26 >> 2024-09-21 19:40:26 weewxd[235236] DEBUG weewx.engine: RMBx: in >> new_loop_packet (StdArchive) >> 2024-09-21 19:40:28 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 19:40:28 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962029 2024-09-21 19:40:29 >> 2024-09-21 19:40:28 weewxd[235236] DEBUG weewx.engine: RMBx: In >> check_loop (StdArchive) packet dateTime: 1726962029 end_archive_delay_ts: >> 1726962035 1726962300 35 >> 2024-09-21 19:40:28 weewxd[235236] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 19:40:28 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962029 2024-09-21 19:40:29 >> 2024-09-21 19:40:28 weewxd[235236] DEBUG weewx.engine: RMBx: in >> new_loop_packet (StdArchive) >> 2024-09-21 19:40:31 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 19:40:31 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962031 2024-09-21 19:40:31 >> 2024-09-21 19:40:31 weewxd[235236] DEBUG weewx.engine: RMBx: In >> check_loop (StdArchive) packet dateTime: 1726962031 end_archive_delay_ts: >> 1726962035 1726962300 35 >> 2024-09-21 19:40:31 weewxd[235236] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 19:40:31 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962031 2024-09-21 19:40:31 >> 2024-09-21 19:40:31 weewxd[235236] DEBUG weewx.engine: RMBx: in >> new_loop_packet (StdArchive) >> 2024-09-21 19:40:33 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 19:40:33 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962034 2024-09-21 19:40:34 >> 2024-09-21 19:40:33 weewxd[235236] DEBUG weewx.engine: RMBx: In >> check_loop (StdArchive) packet dateTime: 1726962034 end_archive_delay_ts: >> 1726962035 1726962300 35 >> 2024-09-21 19:40:33 weewxd[235236] DEBUG user.observationtime: RMB: In >> new_loop_packet >> 2024-09-21 19:40:33 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962034 2024-09-21 19:40:34 >> 2024-09-21 19:40:33 weewxd[235236] DEBUG weewx.engine: RMBx: in >> new_loop_packet (StdArchive) >> 2024-09-21 19:40:36 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 19:40:36 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962036 2024-09-21 19:40:36 >> 2024-09-21 19:40:36 weewxd[235236] DEBUG weewx.engine: RMBx: In >> check_loop (StdArchive) packet dateTime: 1726962036 end_archive_delay_ts: >> 1726962035 1726962300 35 >> 2024-09-21 19:40:36 weewxd[235236] DEBUG weewx.engine: RMBx: Raising >> BreakLoop 1726962036 1726962035 <--- due to archive_delay, the loop is >> 'finally' broken and the archive record can be processed. >> 2024-09-21 19:40:36 weewxd[235236] DEBUG weewx.engine: RMBx: handling >> BreakLoop exception >> 2024-09-21 19:40:36 weewxd[235236] DEBUG user.observationtime: RMBx: In >> post_loop >> 2024-09-21 19:40:36 weewxd[235236] DEBUG weewx.engine: RMBx: In >> post_loop (StdArchive) >> 2024-09-21 19:40:36 weewxd[235236] DEBUG user.observationtime: RMBx: In >> new_archive_record <--- processing the new archive record >> 2024-09-21 19:40:56 weewxd[235236] DEBUG user.observationtime: RMBx: In >> pre_loop >> 2024-09-21 19:40:56 weewxd[235236] DEBUG user.observationtime: RMB: In >> check_loop >> 2024-09-21 19:40:56 weewxd[235236] DEBUG user.observationtime: RMB: >> packet dateTime: 1726962039 2024-09-21 19:40:39 >> >> Because of the archive_delay, the last packet goes into the correct >> archive record and the END_ARCHIVE_PERIOD event fires before the >> NEW_LOOP_PACKET event is fired. >> >> I realize that changing the order of events firing can have big impact to >> existing code, but I am curious about possibly making this change? It seems >> to me that this should only impact anyone that wants the END_ARCHIVE_PERIOD >> event to fire before the next packet's NEW_LOOP_PACKET is fired. Not sure >> if that is important to anyone... Or is there a better/more WeeWX way of >> accomplishing what I desire? >> If no changes are made to WeeWX, my current thought is to capture the >> data via the NEW_LOOP_PACKET and perform the processing via the >> NEW_ARCHIVE_RECORD event and only update the archive record with the min >> value and time. This would certainly work for me. >> >> Thanks. rich >> > -- You received this message because you are subscribed to the Google Groups "weewx-development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/53eac2e6-1118-4c03-8aec-c5c58ae091a2n%40googlegroups.com.
