Quite some time ago when I produced a loop based clientraw.txt (for realtime update of the Saratoga dashboard). All was well whilst weeWX and my web server were on the same machine. About 18 months ago I moved weeWX to a new machine and I was left with no means of realtime transfer of clientraw.txt to my web server. I did create a loop based rsync service that rsync'ed clientraw.txt every loop period, the service essentially consisted of putting the the rsync code into its own service with a binding to NEW_LOOP_PACKET to fire the rsync. It was a little crude but worked, although some of its limitations came through when I created a loop based customclientraw.txt (superseded by gauge-data.txt) to drive the SteelSeries Gauges. I had intended to redo the my rsync service such that rsync ran in its own thread monitoring a queue for 'rsync jobs' and recently I had the impetous to finish it off. It seems to work well, any of the services that generate realtime files just place a file name in the queue and the rsync service takes care of it.Whilst running in a thread is a little more complex, performance is much better with little impact during the (substantial) report cycle on my weeWX machine.
Gary On Tuesday, 14 February 2017 03:50:56 UTC+10, Tom Keffer wrote: > > That could work. With this thinking, we could even do away with the report > generator. Instead, image generation and file generation would be separate > weewx services. Each would invoke its own FTP session when done. Because > the set of generated files are disjoint, there would be no duplicated > effort (although there could be multiple simultaneous FTP sessions). > > I believe the module ftpupload is pretty standalone. There's nothing in > there that depends on the report generator framework. > > -tk > > On Mon, Feb 13, 2017 at 9:44 AM, mwall <[email protected] > <javascript:>> wrote: > >> On Monday, February 13, 2017 at 12:19:40 PM UTC-5, Tom Keffer wrote: >>> >>> someone should write an rsync and an ftp service that can be bound to >>>> NEW_LOOP_PACKET. it could use the same code used by the ftp/rsync >>>> reports, >>>> just wrapped in a StdService instead of report generator. >>>> >>> >>> The problem with this approach is that then you need some sort of lock >>> for the FTP service, so it knows when the reporting thread is done. >>> >>> Hence the decision to put the FTP and reporting in the same thread, run >>> by the same generator. >>> >> >> that makes sense. and it works really well for reports. >> >> but there is a need for a general approach to doing transfer/sync outside >> of the StdReport framework. >> >> what if the ftp/rsync were a black box that any service could initiate? >> for example, as soon as it finished its 'work', the crt extension could >> start an ftp/rsync process. or a json or csv extension, as soon as they >> finish writing, could start an ftp/rsync process. >> >> so the notion of a transfer/sync could be a feature that any service >> could easily implement, and there would be a standard >> 'transfer/sync-parameters' stanza analogous to database bindings. >> >> (doing real-time feeds with mqtt, influx, emoncms, etc gets the job done >> but with a totally different approach) >> >> m >> >> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
