As a followup. When I flipped the order that StdEngine fires the events, such that CHECK_LOOP is before NEW_LOOP this is what I observed. 1. A new loop packet is generated. 2. StdEngine fires the CHECK_LOOP event. 3. StdArchive handles the CHECK_LOOP. a. The dateTime in the loop packet is greater than the end of the archive period. b. StdArchive fires the END_ARCHIVE_PERIOD event. c. The dateTime is not greater than the end of the archive period + the archive delay. 4. MyService handles the END_ARCHIVE_PERIOD. a. Min values are reset to None, etc. 5. StdEngine fires a NEW_LOOP event 6. MyService handles the NEW_LOOP event. a. It sees that no prior value exists b. The current value in the loop packet is added to the loop packet as min_value. 7. StdArchive handles the NEW_LOOP event. a. It tries to add it to the accumulator. b. The accumulator throws OutOfSpan exception c. StdArchive creates a 'new' accumulator for the next archive period. d. StdArchive adds the loop packet to the accumulator,
8. Eventually a packet with dateTime >= the end of archive period + archive delay is generated. a. This causes StdArchive (in CHECK_LOOP) to raise the BREAKLOOP exception b. StdEngine handles the exception, raising POST_LOOP; allowing things like reports to run etc. Again, not saying that changing the order that events are fired should be taken lightly. rich On Sunday 22 September 2024 at 11:28:28 UTC-4 [email protected] wrote: > Tom, > Thanks for the pointer. That is what I am attempting to do. Here is what I > THINK is my problem. See 4.d below. > 1. A new loop packet is generated. > 2. StdEngine fires a NEW_LOOP event > 3. MyService handles the NEW_LOOP event. > a. It sees that a prior value is less then the current value in the > loop packet. > b. The prior value is added to the loop packet as min_value. > 4. StdArchive handles the NEW_LOOP event. > a. It tries to add it to the accumulator. > b. The accumulator throws OutOfSpan exception > c. StdArchive creates a 'new' accumulator for the next archive period. > d. StdArchive adds the loop packet (which has been updated with prior > loop packet data) to the accumulator, > 5. StdEngine fires the CHECK_LOOP event. > 6. StdArchive handles the CHECK_LOOP. > a. The dateTime in the loop packet is greater than the end of the > archive period. > b. StdArchive fires the END_ARCHIVE_PERIOD event. > c. The dateTime is not greater than the end of the archive period + the > archive delay. > 7. MyService handles the END_ARCHIVE_PERIOD. > a. Min values are reset to None, etc. > > > 8. Eventually a packet with dateTime >= the end of archive period + > archive delay is generated. > a. This causes StdArchive (in CHECK_LOOP) to raise the BREAKLOOP > exception > b. StdEngine handles the exception, raising POST_LOOP; allowing things > like reports to run etc. > > rich > > On Sunday 22 September 2024 at 11:00:00 UTC-4 Tom Keffer wrote: > >> Take a look at the Vantage device driver. In addition to a standard >> device driver, it includes a service, VantageService, that participates in >> the weewxd event loop. It binds to STARTUP, NEW_LOOP_PACKET, and >> END_ARCHIVE_PERIOD. The first two are used to reset wind gust values, the >> last to update the archive record with the max wind seen. >> >> I think this is pretty similar to what you're trying to do. >> >> -tk >> >> On Saturday, September 21, 2024 at 5:01:26 PM UTC-7 [email protected] >> wrote: >> >>> >>> Goal: Capture the min value and time of lightning strikes. >>> >>> I started out binding to the PRE_LOOP and NEW_LOOP_PACKET events. When >>> the PRE_LOOP event is handled, the min value and time were reset. In the >>> handling of the NEW_LOOP_PACKET event, the comparison is done and the loop >>> packet is updated as necessary. I 'quickly' realized that PRE_LOOP and >>> POST_LOOP events have nothing to do with the archive period. >>> >>> The END_ARCHIVE_PERIOD event looked promising. Resetting of the value >>> and time could be done here. But when I analyze the order of event firing, >>> this is what I am seeing. >>> 2024-09-21 11:44:59 weewxd[4036220] DEBUG user.observationtime: RMB: >>> In new_loop_packet >>> 2024-09-21 11:44:59 weewxd[4036220] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726933499 2024-09-21 11:44:59 >>> 2024-09-21 11:44:59 weewxd[4036220] DEBUG user.observationtime: RMB: >>> In check_loop >>> 2024-09-21 11:44:59 weewxd[4036220] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726933499 2024-09-21 11:44:59 >>> 2024-09-21 11:45:01 weewxd[4036220] DEBUG user.observationtime: RMB: >>> In new_loop_packet >>> 2024-09-21 11:45:01 weewxd[4036220] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726933502 2024-09-21 11:45:02 >>> 2024-09-21 11:45:01 weewxd[4036220] DEBUG user.observationtime: RMB: >>> In check_loop >>> 2024-09-21 11:45:01 weewxd[4036220] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726933502 2024-09-21 11:45:02 >>> 2024-09-21 11:45:01 weewxd[4036220] DEBUG user.observationtime: RMBx: >>> In end_archive_period >>> 2024-09-21 11:45:04 weewxd[4036220] DEBUG user.observationtime: RMB: >>> In new_loop_packet >>> 2024-09-21 11:45:04 weewxd[4036220] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726933504 2024-09-21 11:45:04 >>> This will result in the first packet (1726933502) of the archive period >>> using the data from the previous archive period. >>> >>> Flipping the order in engine.py to the following: >>> for packet in self.console.genLoopPackets(): >>> # Allow services to break the loop by throwing >>> # an exception: >>> self.dispatchEvent(weewx.Event(weewx.CHECK_LOOP, packet=packet)) >>> >>> # Package the packet as an event, then dispatch it. >>> self.dispatchEvent(weewx.Event(weewx.NEW_LOOP_PACKET, >>> packet=packet)) >>> >>> I would have expected to see the last packet in an archive period not >>> being processed. But, I was not seeing that. So I added a bit more logging >>> to try to understand what was going on. This is what I saw. Sorry, it is a >>> bit long, I made a couple of comments with <---. >>> >>> 2024-09-21 19:39:58 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop >>> 2024-09-21 19:39:58 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726961999 2024-09-21 19:39:59 >>> 2024-09-21 19:39:58 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> check_loop (StdArchive) packet dateTime: 1726961999 end_archive_delay_ts: >>> 1726962035 1726962000 35 >>> 2024-09-21 19:39:58 weewxd[235236] DEBUG user.observationtime: RMB: In >>> new_loop_packet >>> 2024-09-21 19:39:58 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726961999 2024-09-21 19:39:59 >>> 2024-09-21 19:39:58 weewxd[235236] DEBUG weewx.engine: RMBx: in >>> new_loop_packet (StdArchive) >>> 2024-09-21 19:40:01 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop >>> 2024-09-21 19:40:01 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962001 2024-09-21 19:40:01 >>> 2024-09-21 19:40:01 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> check_loop (StdArchive) packet dateTime: 1726962001 end_archive_delay_ts: >>> 1726962035 1726962000 35 >>> 2024-09-21 19:40:01 weewxd[235236] DEBUG user.observationtime: RMBx: In >>> end_archive_period <--- END_ARCHIVE_PERIOD event must have fired >>> 2024-09-21 19:40:01 weewxd[235236] DEBUG user.observationtime: RMB: In >>> new_loop_packet >>> 2024-09-21 19:40:01 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962001 2024-09-21 19:40:01 <--- And the >>> NEW_LOOP_PACKET has fired afterward >>> 2024-09-21 19:40:01 weewxd[235236] DEBUG weewx.engine: RMBx: in >>> new_loop_packet (StdArchive) >>> 2024-09-21 19:40:03 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop <--- processing of even packets continues >>> 2024-09-21 19:40:03 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962004 2024-09-21 19:40:04 >>> 2024-09-21 19:40:03 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> check_loop (StdArchive) packet dateTime: 1726962004 end_archive_delay_ts: >>> 1726962035 1726962300 35 >>> 2024-09-21 19:40:03 weewxd[235236] DEBUG user.observationtime: RMB: In >>> new_loop_packet >>> 2024-09-21 19:40:03 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962004 2024-09-21 19:40:04 >>> 2024-09-21 19:40:03 weewxd[235236] DEBUG weewx.engine: RMBx: in >>> new_loop_packet (StdArchive) >>> 2024-09-21 19:40:06 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop >>> 2024-09-21 19:40:06 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962006 2024-09-21 19:40:06 >>> 2024-09-21 19:40:06 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> check_loop (StdArchive) packet dateTime: 1726962006 end_archive_delay_ts: >>> 1726962035 1726962300 35 >>> 2024-09-21 19:40:06 weewxd[235236] DEBUG user.observationtime: RMB: In >>> new_loop_packet >>> 2024-09-21 19:40:06 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962006 2024-09-21 19:40:06 >>> 2024-09-21 19:40:06 weewxd[235236] DEBUG weewx.engine: RMBx: in >>> new_loop_packet (StdArchive) >>> 2024-09-21 19:40:08 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop >>> 2024-09-21 19:40:08 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962009 2024-09-21 19:40:09 >>> 2024-09-21 19:40:08 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> check_loop (StdArchive) packet dateTime: 1726962009 end_archive_delay_ts: >>> 1726962035 1726962300 35 >>> 2024-09-21 19:40:08 weewxd[235236] DEBUG user.observationtime: RMB: In >>> new_loop_packet >>> 2024-09-21 19:40:08 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962009 2024-09-21 19:40:09 >>> 2024-09-21 19:40:08 weewxd[235236] DEBUG weewx.engine: RMBx: in >>> new_loop_packet (StdArchive) >>> 2024-09-21 19:40:11 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop >>> 2024-09-21 19:40:11 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962011 2024-09-21 19:40:11 >>> 2024-09-21 19:40:11 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> check_loop (StdArchive) packet dateTime: 1726962011 end_archive_delay_ts: >>> 1726962035 1726962300 35 >>> 2024-09-21 19:40:11 weewxd[235236] DEBUG user.observationtime: RMB: In >>> new_loop_packet >>> 2024-09-21 19:40:11 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962011 2024-09-21 19:40:11 >>> 2024-09-21 19:40:11 weewxd[235236] DEBUG weewx.engine: RMBx: in >>> new_loop_packet (StdArchive) >>> 2024-09-21 19:40:13 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop >>> 2024-09-21 19:40:13 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962014 2024-09-21 19:40:14 >>> 2024-09-21 19:40:13 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> check_loop (StdArchive) packet dateTime: 1726962014 end_archive_delay_ts: >>> 1726962035 1726962300 35 >>> 2024-09-21 19:40:13 weewxd[235236] DEBUG user.observationtime: RMB: In >>> new_loop_packet >>> 2024-09-21 19:40:13 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962014 2024-09-21 19:40:14 >>> 2024-09-21 19:40:13 weewxd[235236] DEBUG weewx.engine: RMBx: in >>> new_loop_packet (StdArchive) >>> 2024-09-21 19:40:16 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop >>> 2024-09-21 19:40:16 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962016 2024-09-21 19:40:16 >>> 2024-09-21 19:40:16 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> check_loop (StdArchive) packet dateTime: 1726962016 end_archive_delay_ts: >>> 1726962035 1726962300 35 >>> 2024-09-21 19:40:16 weewxd[235236] DEBUG user.observationtime: RMB: In >>> new_loop_packet >>> 2024-09-21 19:40:16 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962016 2024-09-21 19:40:16 >>> 2024-09-21 19:40:16 weewxd[235236] DEBUG weewx.engine: RMBx: in >>> new_loop_packet (StdArchive) >>> 2024-09-21 19:40:18 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop >>> 2024-09-21 19:40:18 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962019 2024-09-21 19:40:19 >>> 2024-09-21 19:40:18 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> check_loop (StdArchive) packet dateTime: 1726962019 end_archive_delay_ts: >>> 1726962035 1726962300 35 >>> 2024-09-21 19:40:18 weewxd[235236] DEBUG user.observationtime: RMB: In >>> new_loop_packet >>> 2024-09-21 19:40:18 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962019 2024-09-21 19:40:19 >>> 2024-09-21 19:40:18 weewxd[235236] DEBUG weewx.engine: RMBx: in >>> new_loop_packet (StdArchive) >>> 2024-09-21 19:40:21 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop >>> 2024-09-21 19:40:21 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962021 2024-09-21 19:40:21 >>> 2024-09-21 19:40:21 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> check_loop (StdArchive) packet dateTime: 1726962021 end_archive_delay_ts: >>> 1726962035 1726962300 35 >>> 2024-09-21 19:40:21 weewxd[235236] DEBUG user.observationtime: RMB: In >>> new_loop_packet >>> 2024-09-21 19:40:21 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962021 2024-09-21 19:40:21 >>> 2024-09-21 19:40:21 weewxd[235236] DEBUG weewx.engine: RMBx: in >>> new_loop_packet (StdArchive) >>> 2024-09-21 19:40:23 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop >>> 2024-09-21 19:40:23 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962024 2024-09-21 19:40:24 >>> 2024-09-21 19:40:23 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> check_loop (StdArchive) packet dateTime: 1726962024 end_archive_delay_ts: >>> 1726962035 1726962300 35 >>> 2024-09-21 19:40:23 weewxd[235236] DEBUG user.observationtime: RMB: In >>> new_loop_packet >>> 2024-09-21 19:40:23 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962024 2024-09-21 19:40:24 >>> 2024-09-21 19:40:23 weewxd[235236] DEBUG weewx.engine: RMBx: in >>> new_loop_packet (StdArchive) >>> 2024-09-21 19:40:26 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop >>> 2024-09-21 19:40:26 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962026 2024-09-21 19:40:26 >>> 2024-09-21 19:40:26 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> check_loop (StdArchive) packet dateTime: 1726962026 end_archive_delay_ts: >>> 1726962035 1726962300 35 >>> 2024-09-21 19:40:26 weewxd[235236] DEBUG user.observationtime: RMB: In >>> new_loop_packet >>> 2024-09-21 19:40:26 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962026 2024-09-21 19:40:26 >>> 2024-09-21 19:40:26 weewxd[235236] DEBUG weewx.engine: RMBx: in >>> new_loop_packet (StdArchive) >>> 2024-09-21 19:40:28 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop >>> 2024-09-21 19:40:28 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962029 2024-09-21 19:40:29 >>> 2024-09-21 19:40:28 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> check_loop (StdArchive) packet dateTime: 1726962029 end_archive_delay_ts: >>> 1726962035 1726962300 35 >>> 2024-09-21 19:40:28 weewxd[235236] DEBUG user.observationtime: RMB: In >>> new_loop_packet >>> 2024-09-21 19:40:28 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962029 2024-09-21 19:40:29 >>> 2024-09-21 19:40:28 weewxd[235236] DEBUG weewx.engine: RMBx: in >>> new_loop_packet (StdArchive) >>> 2024-09-21 19:40:31 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop >>> 2024-09-21 19:40:31 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962031 2024-09-21 19:40:31 >>> 2024-09-21 19:40:31 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> check_loop (StdArchive) packet dateTime: 1726962031 end_archive_delay_ts: >>> 1726962035 1726962300 35 >>> 2024-09-21 19:40:31 weewxd[235236] DEBUG user.observationtime: RMB: In >>> new_loop_packet >>> 2024-09-21 19:40:31 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962031 2024-09-21 19:40:31 >>> 2024-09-21 19:40:31 weewxd[235236] DEBUG weewx.engine: RMBx: in >>> new_loop_packet (StdArchive) >>> 2024-09-21 19:40:33 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop >>> 2024-09-21 19:40:33 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962034 2024-09-21 19:40:34 >>> 2024-09-21 19:40:33 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> check_loop (StdArchive) packet dateTime: 1726962034 end_archive_delay_ts: >>> 1726962035 1726962300 35 >>> 2024-09-21 19:40:33 weewxd[235236] DEBUG user.observationtime: RMB: In >>> new_loop_packet >>> 2024-09-21 19:40:33 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962034 2024-09-21 19:40:34 >>> 2024-09-21 19:40:33 weewxd[235236] DEBUG weewx.engine: RMBx: in >>> new_loop_packet (StdArchive) >>> 2024-09-21 19:40:36 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop >>> 2024-09-21 19:40:36 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962036 2024-09-21 19:40:36 >>> 2024-09-21 19:40:36 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> check_loop (StdArchive) packet dateTime: 1726962036 end_archive_delay_ts: >>> 1726962035 1726962300 35 >>> 2024-09-21 19:40:36 weewxd[235236] DEBUG weewx.engine: RMBx: Raising >>> BreakLoop 1726962036 1726962035 <--- due to archive_delay, the loop is >>> 'finally' broken and the archive record can be processed. >>> 2024-09-21 19:40:36 weewxd[235236] DEBUG weewx.engine: RMBx: handling >>> BreakLoop exception >>> 2024-09-21 19:40:36 weewxd[235236] DEBUG user.observationtime: RMBx: In >>> post_loop >>> 2024-09-21 19:40:36 weewxd[235236] DEBUG weewx.engine: RMBx: In >>> post_loop (StdArchive) >>> 2024-09-21 19:40:36 weewxd[235236] DEBUG user.observationtime: RMBx: In >>> new_archive_record <--- processing the new archive record >>> 2024-09-21 19:40:56 weewxd[235236] DEBUG user.observationtime: RMBx: In >>> pre_loop >>> 2024-09-21 19:40:56 weewxd[235236] DEBUG user.observationtime: RMB: In >>> check_loop >>> 2024-09-21 19:40:56 weewxd[235236] DEBUG user.observationtime: RMB: >>> packet dateTime: 1726962039 2024-09-21 19:40:39 >>> >>> Because of the archive_delay, the last packet goes into the correct >>> archive record and the END_ARCHIVE_PERIOD event fires before the >>> NEW_LOOP_PACKET event is fired. >>> >>> I realize that changing the order of events firing can have big impact >>> to existing code, but I am curious about possibly making this change? It >>> seems to me that this should only impact anyone that wants the >>> END_ARCHIVE_PERIOD event to fire before the next packet's NEW_LOOP_PACKET >>> is fired. Not sure if that is important to anyone... Or is there a >>> better/more WeeWX way of accomplishing what I desire? >>> If no changes are made to WeeWX, my current thought is to capture the >>> data via the NEW_LOOP_PACKET and perform the processing via the >>> NEW_ARCHIVE_RECORD event and only update the archive record with the min >>> value and time. This would certainly work for me. >>> >>> Thanks. rich >>> >> -- You received this message because you are subscribed to the Google Groups "weewx-development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/c1da359f-b644-43de-b0c4-30218e066dd8n%40googlegroups.com.
