No idea why I wasn't using Rsync in weewx.conf but it was over 3 years ago. Have changed it to the weewx.conf which didnt work at first so followed the example in USING RYSNC SKIN AS A BACKUP to create the password-less access. Now backing up the public_html and RTGD now work, without the cron jobs. I most have set it up as a non-root password although what threw me was that the cron job work. Thanks for your help John Phil On Monday, May 23, 2022 at 9:50:09 PM UTC+1 [email protected] wrote:
> That rsync command you listed, it is in root’s crontab? > And why aren’t you doing the rsync in weewx? > > On May 23, 2022, at 1:15 PM, [email protected] <[email protected]> > wrote: > > Im using rsync to copy the public_html folder to my server using a cron > job every 10mins ===== rsync -a /home/weewx/public_html/ > [email protected] etc etc > > Then thought I would use rsync in rtgd, from weewx.conf to update the > gauges in real time. > OS is Buster 10 using a Raspberry Pi > > > On Monday, May 23, 2022 at 8:28:18 PM UTC+1 [email protected] wrote: > >> I’m curious, are you using rsync to publish weewx? I’m asking because >> rtgd is calling the same weewx method. >> >> If you are using rsync to publish weewx, please send that section of your >> weewx.conf. There must be some difference. >> >> Also, what OS are you using? I’m worried that it is Windows from your >> paths. If it is, I’m unlikely to be able to debug this. >> >> On May 23, 2022, at 10:56 AM, [email protected] <[email protected]> >> wrote: >> >> Im using LushEr21 for both and ssh LushEr21@server-ip-address logs on to >> the server without a password >> >> >> >> On Monday, May 23, 2022 at 4:39:47 PM UTC+1 [email protected] wrote: >> >>> >>> >>> I’m assuming that <username> in ssh-copy-id -i ~/.ssh/id_rsa.pub >>> <username>@<server-ip-address> - >>> is: >>> LushEr21 >>> Since that what you specified in weewx.conf. >>> >>> Please confirm that the following works (i.e., you are logged in to >>> <server-ip-address> as LushEr21 without having to enter a password or an >>> [empty] passphrase: >>> (As root:) >>> ssh LushEr21@<server-ip-address> >>> >>> On May 23, 2022, at 7:00 AM, [email protected] <[email protected]> >>> wrote: >>> >>> For ssh >>> >>> I used when I was root >>> ssh-keygen -f C:\Users\<username>\.ssh\id_rsa -t rsa -P "" >>> >>> ssh-copy-id -i ~/.ssh/id_rsa.pub <username>@<server-ip-address> -o >>> StrictHostKeyChecking=No >>> Should I have a file /usr/bin/ssh-askpass >>> Sorry to be a pain >>> Phil >>> >>> >>> On Monday, May 23, 2022 at 1:39:10 PM UTC+1 [email protected] wrote: >>> >>>> I don’t recommend extending the timeout. It’s there so that requests >>>> don’t build up. >>>> Are you sure you have ssh set up correctly? In particular, did you set >>>> it up for the same user as weewx is running as? That’s often root. If >>>> weewx is running as root for you, please try to ssh in without a password >>>> when you are logged in as root. >>>> >>>> On May 23, 2022, at 3:13 AM, [email protected] <[email protected]> >>>> wrote: >>>> >>>> Thanks John >>>> I now get >>>> May 23 10:59:03 raspberrypi weewx-weewx[1438] ERROR >>>> weeutil.rsyncupload: [['rsync', '--archive', '--stats', '--timeout=4', '-e >>>> ssh -o ConnectTimeout=1', '/home/weewx/public_html/gauge-data.txt', >>>> '[email protected]:/volume1/web/SheringhamDDNS/gaugedata/gauge-data.txt']] >>>> >>>> reported errors: ssh_askpass: exec(/usr/bin/ssh-askpass): No such file or >>>> directory. [sender] io timeout after 4 seconds -- exiting. rsync error: >>>> timeout in data send/receive (code 30) at io.c(204) [sender=3.1.3] >>>> May 23 10:59:03 raspberrypi weewx-weewx[1438] INFO weeutil.rsyncupload: >>>> rsync executed in 6.03 seconds >>>> I had to extend the Timeout = 4 as it would time out. >>>> I dont appear to have a file usr/bin/ssh_askpass When the passwordless >>>> key was set up I believe it saved the info to ssh-keygen >>>> >>>> rsync -v /home/weewx/public_html/gauge-data.txt >>>> [email protected]:/volume1/web/SheringhamDDNS works OK in a cron job >>>> Phil >>>> On Sunday, May 22, 2022 at 6:24:58 PM UTC+1 [email protected] wrote: >>>> >>>>> OK, you could also replace rtgd.py (in weewx’s bin/user directory) >>>>> with the file at: >>>>> >>>>> https://github.com/chaunceygardiner/weewx-realtime_gauge-data/blob/master/bin/user/rtgd.py >>>>> >>>>> I’ve also sent Gary a PR at: >>>>> https://github.com/gjr80/weewx-realtime_gauge-data/pull/28 >>>>> >>>>> >>>>> On May 22, 2022, at 10:07 AM, John Kline <[email protected]> wrote: >>>>> >>>>> >>>>> >>>>> Never mind. I installed it and found the issue. It’s been broken >>>>> since it was refactored. >>>>> >>>>> Below is the patch to get it working. I’ll also follow up with a URL >>>>> to grab the file from github; as well as a PR to Gary (who may take it if >>>>> he notices it). >>>>> >>>>> diff --git a/bin/user/rtgd.py b/bin/user/rtgd.py >>>>> index 9f631c9..09ddd24 100644 >>>>> --- a/bin/user/rtgd.py >>>>> +++ b/bin/user/rtgd.py >>>>> @@ -1300,7 +1300,7 @@ class HttpPostExport(object): >>>>> # response text from remote URL if post was successful >>>>> self.response = post_config_dict.get('response_text', None) >>>>> >>>>> - def export(self, data): >>>>> + def export(self, data, dateTime): >>>>> """Post the data.""" >>>>> >>>>> self.post_data(data) >>>>> @@ -1419,10 +1419,11 @@ class RsyncExport(object): >>>>> self.rsync_skip_if_older_than = >>>>> to_int(rsync_config_dict.get('rsync_skip_if_older_than', >>>>> >>>>> 4)) >>>>> >>>>> - def export(self, data): >>>>> + def export(self, data, dateTime): >>>>> """Rsync the data.""" >>>>> >>>>> - packet_time = >>>>> datetime.datetime.fromtimestamp(data['dateTime']) >>>>> + log.info("export: data: %r" % data) >>>>> + packet_time = datetime.datetime.fromtimestamp(dateTime) >>>>> self.rsync_data(packet_time) >>>>> >>>>> def rsync_data(self, packet_time): >>>>> @@ -1900,7 +1901,7 @@ class RealtimeGaugeDataThread(threading.Thread): >>>>> self.last_write = time.time() >>>>> # export gauge-data.txt if we have an exporter >>>>> object >>>>> if self.exporter: >>>>> - self.exporter.export(data) >>>>> + self.exporter.export(data, packet['dateTime']) >>>>> # log the generation >>>>> if weewx.debug == 2: >>>>> log.info("gauge-data.txt (%s) generated in >>>>> %.5f seconds" % (cached_packet['dateTime'], >>>>> >>>>> On May 22, 2022, at 6:08 AM, John Kline <[email protected]> wrote: >>>>> >>>>> >>>>> Oh, yes, two brackets. What failures did you get with two brackets? >>>>> >>>>> On May 22, 2022, at 2:59 AM, [email protected] < >>>>> [email protected]> wrote: >>>>> >>>>> Thanks John >>>>> Can I just confirm how many bracket Rsync would have please >>>>> Have tried 1 bracket and got lots of log failures and then tried it >>>>> with 2 brackets as [[StringFormats]] below has 2 brackets , and got a >>>>> different failures. >>>>> Both were critical errors and stopped RTGD from working. >>>>> In both cases the line raspberrypi weewx-weewx[352] INFO user.rtgd: >>>>> gauge-data.txt will not be exported. had gone. >>>>> Phil >>>>> >>>>> On Saturday, May 21, 2022 at 7:01:24 PM UTC+1 [email protected] >>>>> wrote: >>>>> >>>>>> BTW, this is the commit that moved the rsync parms (and, it looks >>>>>> like HTTP post parms) into a separate section: >>>>>> >>>>>> https://github.com/gjr80/weewx-realtime_gauge-data/commit/534032cef8f1bfba45b90cf7ef939c16084a5b2e >>>>>> 0 >>>>>> >>>>>> The README should probably change to match. >>>>>> >>>>>> >>>>>> On May 21, 2022, at 6:48 AM, John Kline <[email protected]> wrote: >>>>>> >>>>>> >>>>>> >>>>>> I believe the rsync_xxx entries need to be under an Rsync section: >>>>>> [RealtimeGaugeData] >>>>>> . >>>>>> . >>>>>> . >>>>>> [Rsync] >>>>>> rsync_server=xxx >>>>>> Rsync_user=LushEr21 >>>>>> etc. >>>>>> >>>>>> If that doesn’t work, I will install the extension and get it working. >>>>>> >>>>>> On May 21, 2022, at 3:18 AM, [email protected] < >>>>>> [email protected]> wrote: >>>>>> >>>>>> [RealtimeGaugeData] >>>>>> date_format = %Y.%m.%d %H:%M >>>>>> rtgd_path = /home/weewx/public_html >>>>>> min_interval = 15 >>>>>> response_text = success >>>>>> rsync_server = xxx.xxx.x.xxx >>>>>> rsync_user = LushEr21 >>>>>> rsync_remote_rtgd_dir = /volume1/web/SheringhamDDNS >>>>>> rsync_compress = False >>>>>> rsync_log_success = True >>>>>> rsync_ssh_options = "-o ConnectTimeout=1" >>>>>> rsync_timeout = 1 >>>>>> rsync_skip_if_older_than = 4 >>>>>> >>>>>> The key/passwordless to the server works with >>>>>> rsync -v /home/weewx/public_html/gauge-data.txt >>>>>> [email protected]:/volume1/web/SheringhamDDNS so the >>>>>> key/passwordless >>>>>> so am assuming the above works >>>>>> Thanks for your help >>>>>> Phil >>>>>> >>>>>> >>>>>> On Thursday, May 19, 2022 at 4:27:56 PM UTC+1 [email protected] >>>>>> wrote: >>>>>> >>>>>>> You need to specify the rsync parameters in the RealtimeGaugeData >>>>>>> section of weewx.conf. Are you doing that? Perhaps send the >>>>>>> RealtimeGaugeData section of weewx.conf so we can have a look. From >>>>>>> your >>>>>>> previous emails, it isn’t clear that you have the parameters in the >>>>>>> correct >>>>>>> section. I can tell you that this worked for me for a long time (in >>>>>>> fact, >>>>>>> I submitted the pull request for this functionality). >>>>>>> >>>>>>> # Fill out the following fields: >>>>>>> # rsync_server : The server to which gauge-data.txt >>>>>>> will be copied. >>>>>>> # rsync_user : The userid on rsync_server with write >>>>>>> # permission to rsync_remote_rtgd_dir. >>>>>>> # rsync_remote_rtgd_dir : The directory on rsync_server where >>>>>>> # gauge-data.txt will be copied. >>>>>>> # rsync_compress : True to compress the file before >>>>>>> sending. >>>>>>> # Default is False. >>>>>>> # rsync_log_success : True to write success with timing >>>>>>> messages to >>>>>>> # the log (for debugging). Default is >>>>>>> False. >>>>>>> # rsync_ssh_options : ssh options Default is '-o >>>>>>> ConnectTimeout=1' >>>>>>> # (When connecting, time out in 1 >>>>>>> second.) >>>>>>> # rsync_timeout : I/O timeout. Default is 1. (When >>>>>>> sending, >>>>>>> # timeout in 1 second.) >>>>>>> # rsync_skip_if_older_than : Don't bother to rsync if greater than >>>>>>> this >>>>>>> # number of seconds. Default is 4. >>>>>>> (Skip this >>>>>>> # and move on to the next if this data >>>>>>> is older >>>>>>> # than 4 seconds. >>>>>>> >>>>>>> On May 19, 2022, at 6:37 AM, [email protected] < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>> Just to update >>>>>>> >>>>>>> Have a cron job working Ok with >>>>>>> rsync -v /home/weewx/public_html/gauge-data.txt >>>>>>> [email protected]:/volume1/web/SheringhamDDNS so the >>>>>>> key/passwordless >>>>>>> is working >>>>>>> Changed >>>>>>> rsync_server = xxx.xxx.x.xxx >>>>>>> rsync_user = LushEr21 >>>>>>> And it still doesnt work >>>>>>> If somebody has tried this and got it to work your ideas on why I >>>>>>> cant get rsync to work with RTGD would be most welcome >>>>>>> Thanks >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Tuesday, May 17, 2022 at 12:00:44 PM UTC+1 [email protected] >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Guys >>>>>>>> Ive tried and failed to get the rsync to work using >>>>>>>> realtimeguagedata >>>>>>>> >>>>>>>> I never get passed gauge-data.txt file will not be exported. >>>>>>>> Here is an extract of the log. >>>>>>>> May 17 11:38:34 raspberrypi weewx-weewx[352] INFO user.rtgd: >>>>>>>> version is 0.5.5 >>>>>>>> May 17 11:38:34 raspberrypi weewx-weewx[352] INFO user.rtgd: >>>>>>>> RealTimeGaugeData scroller text will use a fixed string >>>>>>>> May 17 11:38:34 raspberrypi weewx-tides[363] INFO weewx.engine: >>>>>>>> Using archive interval of 600 seconds (specified in weewx >>>>>>>> configuration) >>>>>>>> May 17 11:38:34 raspberrypi weewx-weewx[352] INFO user.rtgd: >>>>>>>> gauge-data.txt will not be exported. >>>>>>>> May 17 11:38:34 raspberrypi weewx-weewx[352] INFO user.rtgd: >>>>>>>> '/home/weewx/public_html/gauge-data.txt' will be generated. >>>>>>>> min_interval is >>>>>>>> 15 seconds >>>>>>>> >>>>>>>> I have the following in the weewx.conf file >>>>>>>> rsync_server = SynLusher >>>>>>>> rsync_user = guest >>>>>>>> rsync_remote_rtgd_dir = /volume1/web/SheringhamDDNS >>>>>>>> rsync_log_success = True >>>>>>>> >>>>>>>> weewx is version 4.5.1 >>>>>>>> >>>>>>>> The HTTP POST is not being used. >>>>>>>> >>>>>>>> I have set debug = 0, 1, 2 and 3 to try and find where it might be >>>>>>>> failing but its always the same. >>>>>>>> The server is a Synology NAS and I can copy from the rasberry pi to >>>>>>>> the rtgd_dir OK >>>>>>>> Just wondered if its possible to get more logs so give me a clue to >>>>>>>> where is failing >>>>>>>> Thanks >>>>>>>> Phil >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "weewx-user" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/weewx-user/39fff649-3aa4-4a7b-aab5-cd6628080faen%40googlegroups.com >>>>>>> >>>>>>> <https://groups.google.com/d/msgid/weewx-user/39fff649-3aa4-4a7b-aab5-cd6628080faen%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "weewx-user" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/weewx-user/a68f9075-80c3-4980-a3bc-e80f3dfb2ce1n%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/weewx-user/a68f9075-80c3-4980-a3bc-e80f3dfb2ce1n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "weewx-user" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/weewx-user/dbca6111-df73-45be-8d3f-487a90e8bb7en%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/weewx-user/dbca6111-df73-45be-8d3f-487a90e8bb7en%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "weewx-user" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/weewx-user/fd71e6f8-fa8b-4fab-bfe9-e9d9d0abd6cen%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/weewx-user/fd71e6f8-fa8b-4fab-bfe9-e9d9d0abd6cen%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/2db45023-d08b-4691-8609-e43b5ce2fba6n%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/weewx-user/2db45023-d08b-4691-8609-e43b5ce2fba6n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/weewx-user/3af547de-13a5-4e41-9bb0-aadf108b4a52n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/weewx-user/3af547de-13a5-4e41-9bb0-aadf108b4a52n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/0cf5d4d4-b044-420d-8808-306365b69b76n%40googlegroups.com > > <https://groups.google.com/d/msgid/weewx-user/0cf5d4d4-b044-420d-8808-306365b69b76n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/1e6ae945-7ce4-410d-b293-ca6b77987b3an%40googlegroups.com.
