Re: [weewx-user] Re: fileparse polling interval

[Bill Burton]

Hello,

On Tue, Apr 21, 2020 at 8:53 AM jf42  wrote:

> My station is a german "MWS 55V", which is read out by a Python program,
> writing the data to a tcp port. Another computer
> collects the data, does a quality check, many other stuff, and writes a
> file. Weewx uses the file parser to get the data.
> The station provides the data every 2 secs, the file resolution is about 5
> secs. We use the data for a lidar instrument
> (light radar) for weather monitoring during atmospheric measurements.
> Thus: the faster the better ;-)
> That's for the background of my question.
>

So it sounds like you are using the WeeWX report generation or archive
records creation to send data to the lidar?

Normally, the archive interval is configured to match reporting
requirements that will work for a day/week/month/year which is why the
default is 300 seconds/5 minutes. Also as some weather stations send data
less frequently, this gives time to collect most if not all the data of
each type. The archive interval is a different requirement than how fast a
WeeWX driver receives data.

Since you are trying to get data to the lidar as fast as possible, have you
considered using MQTT to transfer data? The message based publish-subscribe
model may be more suitable for event based data transfer than polling for
files. For instance:

   - The program that writes the data file after quality control could
   instead publish it to an MQTT topic
   
   .
   - WeeWX would be configured to use the MQTT Subscribe Driver
    and receive these
   messages as loop packets. At the appropriate time, WeeWX would write the
   archive record and generate the reports.
   - You would implement another (Python) program that subscribes to the
   same MQTT topic
   
   and sends that information to the lidar as fast as data is received.

With this approach, the lidar would receive the data almost immediately
after the QC machine publishes it without going through the WeeWX archival
and report generation process. This way, the archive interval could be
configured at the right amount to meet your reporting requirements would
likely be much less frequent then how often data is received. However, if
your requirement is that all data sent to the lidar must be logged, then
you either have to log it elsewhere or use a short archive interval.
However, the MQTT driver can be configured to publish archive data which
should make it available more quickly than other methods that provide the
archive data.
The page, How to setup your own MQTT Broker

 is a nice
overview on setting up MQTT to work with WeeWX even if you don't use the
Belchertown skin. However, the MQTT extension for WeeWX mentioned in that
post is used to *publish* loop packets after WeeWX receives them, it cannot
act as a driver to *subscribe* to an MQTT topic for receiving data.

Hope this helps,
-Bill

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/CAKKfUm9PjshLTTbB_BCwo2m0Vm5cHfkgxLHwPUf1StcbG8ch1w%40mail.gmail.com.

Re: [weewx-user] Re: Registering on weewx map.

[Bill Burton]

Hello Matt,

On Wed, Jun 3, 2020 at 4:10 PM mwall  wrote:

> On Wednesday, June 3, 2020 at 1:55:15 PM UTC-4, William Burton wrote:
>>
>> If the screen capture for your own site fails or if the site's skin has
>> been changed, is there a way to get the registration process to retry the
>> screen capture?
>>
>> Is there a way to test the screen capture outside of the registration
>> process to see if it will work correctly?
>>
>
> hi bill,
>
> i made a wiki page to explain the registry:
>
> https://github.com/weewx/weewx/wiki/registry
>
>
Thanks, this is helpful. Does the description come from [Station], location?

i'm still working on the capture process.  the previous implementation used
> wkhtmltoimage, but that would periodically suck up every bit of memory and
> swap, bringing the capture server to its knees.  using phantomjs has
> alleviated that somewhat, but there are still some problems.  i'll try to
> remember to write up what i learn once its sorted out.
>

I tried a few test captures using phantomjs 2.1.1 and there were a number
of problems due to missing content or sections being rendered incorrectly.

>
> there is no way to force an update of the capture.  it is a somewhat
> stochastic process (intentionally, to avoid overloading the system).
>

Yes, of course.

>
> once the capture process is more reliable we will be able to tune it.  who
> knows, some kind of waybackmachine-like abilities might even be possible!
>

A couple of challenges I can see are some of the skins are sensitive to
width such that a certain minimum around 1300px is needed and the second is
that if the capture is done at a random time, it may not reflect what the
site owner would like for the image due to the time of day or other factors.

-Bill

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/CAKKfUm93QAwEvnsyOtFnc2pCCdeP6ScK%2BeMWOsCNO4G8P5TT6g%40mail.gmail.com.