That rsync command you listed, it is in root’s crontab? And why aren’t you doing the rsync in weewx?
> On May 23, 2022, at 1:15 PM, [email protected] <[email protected]> > wrote: > > Im using rsync to copy the public_html folder to my server using a cron job > every 10mins ===== rsync -a /home/weewx/public_html/ [email protected] > etc etc > Then thought I would use rsync in rtgd, from weewx.conf to update the gauges > in real time. > OS is Buster 10 using a Raspberry Pi > > >> On Monday, May 23, 2022 at 8:28:18 PM UTC+1 [email protected] wrote: >> I’m curious, are you using rsync to publish weewx? I’m asking because rtgd >> is calling the same weewx method. >> >> If you are using rsync to publish weewx, please send that section of your >> weewx.conf. There must be some difference. >> >> Also, what OS are you using? I’m worried that it is Windows from your >> paths. If it is, I’m unlikely to be able to debug this. >> >>>> On May 23, 2022, at 10:56 AM, [email protected] <[email protected]> >>>> wrote: >>>> >>> Im using LushEr21 for both and ssh LushEr21@server-ip-address logs on to >>> the server without a password >> >>> >>> >>>> On Monday, May 23, 2022 at 4:39:47 PM UTC+1 [email protected] wrote: >>>> >>>> >>>> I’m assuming that <username> in ssh-copy-id -i ~/.ssh/id_rsa.pub >>>> <username>@<server-ip-address> - is: >>>> LushEr21 >>>> Since that what you specified in weewx.conf. >>>> >>>> Please confirm that the following works (i.e., you are logged in to >>>> <server-ip-address> as LushEr21 without having to enter a password or an >>>> [empty] passphrase: >>>> (As root:) >>>> ssh LushEr21@<server-ip-address> >>>> >>>>>> On May 23, 2022, at 7:00 AM, [email protected] <[email protected]> >>>>>> wrote: >>>>>> >>>>> For ssh >>>> >>>>> I used when I was root >>>>> ssh-keygen -f C:\Users\<username>\.ssh\id_rsa -t rsa -P "" >>>>> ssh-copy-id -i ~/.ssh/id_rsa.pub <username>@<server-ip-address> -o >>>>> StrictHostKeyChecking=No >>>>> Should I have a file /usr/bin/ssh-askpass >>>>> Sorry to be a pain >>>>> Phil >>>>> >>>>> >>>>>> On Monday, May 23, 2022 at 1:39:10 PM UTC+1 [email protected] wrote: >>>>>> I don’t recommend extending the timeout. It’s there so that requests >>>>>> don’t build up. >>>>>> Are you sure you have ssh set up correctly? In particular, did you set >>>>>> it up for the same user as weewx is running as? That’s often root. If >>>>>> weewx is running as root for you, please try to ssh in without a >>>>>> password when you are logged in as root. >>>>>> >>>>>>>> On May 23, 2022, at 3:13 AM, [email protected] >>>>>>>> <[email protected]> wrote: >>>>>>>> >>>>>>> Thanks John >>>>>>> I now get >>>>>>> May 23 10:59:03 raspberrypi weewx-weewx[1438] ERROR >>>>>>> weeutil.rsyncupload: [['rsync', '--archive', '--stats', '--timeout=4', >>>>>>> '-e ssh -o ConnectTimeout=1', '/home/weewx/public_html/gauge-data.txt', >>>>>>> '[email protected]:/volume1/web/SheringhamDDNS/gaugedata/gauge-data.txt']] >>>>>>> reported errors: ssh_askpass: exec(/usr/bin/ssh-askpass): No such file >>>>>>> or directory. [sender] io timeout after 4 seconds -- exiting. rsync >>>>>>> error: timeout in data send/receive (code 30) at io.c(204) >>>>>>> [sender=3.1.3] >>>>>>> May 23 10:59:03 raspberrypi weewx-weewx[1438] INFO weeutil.rsyncupload: >>>>>>> rsync executed in 6.03 seconds >>>>>>> I had to extend the Timeout = 4 as it would time out. >>>>>>> I dont appear to have a file usr/bin/ssh_askpass When the passwordless >>>>>>> key was set up I believe it saved the info to ssh-keygen >>>>>>> >>>>>>> rsync -v /home/weewx/public_html/gauge-data.txt >>>>>>> [email protected]:/volume1/web/SheringhamDDNS works OK in a cron job >>>>>>> Phil >>>>>>>> On Sunday, May 22, 2022 at 6:24:58 PM UTC+1 [email protected] wrote: >>>>>>>> OK, you could also replace rtgd.py (in weewx’s bin/user directory) >>>>>>>> with the file at: >>>>>>>> https://github.com/chaunceygardiner/weewx-realtime_gauge-data/blob/master/bin/user/rtgd.py >>>>>>>> >>>>>>>> I’ve also sent Gary a PR at: >>>>>>>> https://github.com/gjr80/weewx-realtime_gauge-data/pull/28 >>>>>>>> >>>>>>>> >>>>>>>>>> On May 22, 2022, at 10:07 AM, John Kline <[email protected]> wrote: >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> Never mind. I installed it and found the issue. It’s been broken >>>>>>>>> since it was refactored. >>>>>>>>> >>>>>>>>> Below is the patch to get it working. I’ll also follow up with a URL >>>>>>>>> to grab the file from github; as well as a PR to Gary (who may take >>>>>>>>> it if he notices it). >>>>>>>>> >>>>>>>>> diff --git a/bin/user/rtgd.py b/bin/user/rtgd.py >>>>>>>>> index 9f631c9..09ddd24 100644 >>>>>>>>> --- a/bin/user/rtgd.py >>>>>>>>> +++ b/bin/user/rtgd.py >>>>>>>>> @@ -1300,7 +1300,7 @@ class HttpPostExport(object): >>>>>>>>> # response text from remote URL if post was successful >>>>>>>>> self.response = post_config_dict.get('response_text', None) >>>>>>>>> >>>>>>>>> - def export(self, data): >>>>>>>>> + def export(self, data, dateTime): >>>>>>>>> """Post the data.""" >>>>>>>>> >>>>>>>>> self.post_data(data) >>>>>>>>> @@ -1419,10 +1419,11 @@ class RsyncExport(object): >>>>>>>>> self.rsync_skip_if_older_than = >>>>>>>>> to_int(rsync_config_dict.get('rsync_skip_if_older_than', >>>>>>>>> >>>>>>>>> 4)) >>>>>>>>> >>>>>>>>> - def export(self, data): >>>>>>>>> + def export(self, data, dateTime): >>>>>>>>> """Rsync the data.""" >>>>>>>>> >>>>>>>>> - packet_time = >>>>>>>>> datetime.datetime.fromtimestamp(data['dateTime']) >>>>>>>>> + log.info("export: data: %r" % data) >>>>>>>>> + packet_time = datetime.datetime.fromtimestamp(dateTime) >>>>>>>>> self.rsync_data(packet_time) >>>>>>>>> >>>>>>>>> def rsync_data(self, packet_time): >>>>>>>>> @@ -1900,7 +1901,7 @@ class RealtimeGaugeDataThread(threading.Thread): >>>>>>>>> self.last_write = time.time() >>>>>>>>> # export gauge-data.txt if we have an exporter >>>>>>>>> object >>>>>>>>> if self.exporter: >>>>>>>>> - self.exporter.export(data) >>>>>>>>> + self.exporter.export(data, >>>>>>>>> packet['dateTime']) >>>>>>>>> # log the generation >>>>>>>>> if weewx.debug == 2: >>>>>>>>> log.info("gauge-data.txt (%s) generated in >>>>>>>>> %.5f seconds" % (cached_packet['dateTime'], >>>>>>>>> >>>>>>>>>>> On May 22, 2022, at 6:08 AM, John Kline <[email protected]> wrote: >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Oh, yes, two brackets. What failures did you get with two brackets? >>>>>>>>>> >>>>>>>>>>>> On May 22, 2022, at 2:59 AM, [email protected] >>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>> Thanks John >>>>>>>>>>> Can I just confirm how many bracket Rsync would have please >>>>>>>>>>> Have tried 1 bracket and got lots of log failures and then tried it >>>>>>>>>>> with 2 brackets as [[StringFormats]] below has 2 brackets , and got >>>>>>>>>>> a different failures. >>>>>>>>>>> Both were critical errors and stopped RTGD from working. >>>>>>>>>>> In both cases the line raspberrypi weewx-weewx[352] INFO user.rtgd: >>>>>>>>>>> gauge-data.txt will not be exported. had gone. >>>>>>>>>>> Phil >>>>>>>>>>> >>>>>>>>>>>> On Saturday, May 21, 2022 at 7:01:24 PM UTC+1 [email protected] >>>>>>>>>>>> wrote: >>>>>>>>>>>> BTW, this is the commit that moved the rsync parms (and, it looks >>>>>>>>>>>> like HTTP post parms) into a separate section: >>>>>>>>>>>> https://github.com/gjr80/weewx-realtime_gauge-data/commit/534032cef8f1bfba45b90cf7ef939c16084a5b2e0 >>>>>>>>>>>> >>>>>>>>>>>> The README should probably change to match. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>>> On May 21, 2022, at 6:48 AM, John Kline <[email protected]> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> I believe the rsync_xxx entries need to be under an Rsync section: >>>>>>>>>>>>> [RealtimeGaugeData] >>>>>>>>>>>>> . >>>>>>>>>>>>> . >>>>>>>>>>>>> . >>>>>>>>>>>>> [Rsync] >>>>>>>>>>>>> rsync_server=xxx >>>>>>>>>>>>> Rsync_user=LushEr21 >>>>>>>>>>>>> etc. >>>>>>>>>>>>> >>>>>>>>>>>>> If that doesn’t work, I will install the extension and get it >>>>>>>>>>>>> working. >>>>>>>>>>>>> >>>>>>>>>>>>>>> On May 21, 2022, at 3:18 AM, [email protected] >>>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>> [RealtimeGaugeData] >>>>>>>>>>>>>> date_format = %Y.%m.%d %H:%M >>>>>>>>>>>>>> rtgd_path = /home/weewx/public_html >>>>>>>>>>>>>> min_interval = 15 >>>>>>>>>>>>>> response_text = success >>>>>>>>>>>>>> rsync_server = xxx.xxx.x.xxx >>>>>>>>>>>>>> rsync_user = LushEr21 >>>>>>>>>>>>>> rsync_remote_rtgd_dir = /volume1/web/SheringhamDDNS >>>>>>>>>>>>>> rsync_compress = False >>>>>>>>>>>>>> rsync_log_success = True >>>>>>>>>>>>>> rsync_ssh_options = "-o ConnectTimeout=1" >>>>>>>>>>>>>> rsync_timeout = 1 >>>>>>>>>>>>>> rsync_skip_if_older_than = 4 >>>>>>>>>>>>>> >>>>>>>>>>>>>> The key/passwordless to the server works with >>>>>>>>>>>>>> rsync -v /home/weewx/public_html/gauge-data.txt >>>>>>>>>>>>>> [email protected]:/volume1/web/SheringhamDDNS so the >>>>>>>>>>>>>> key/passwordless so am assuming the above works >>>>>>>>>>>>>> Thanks for your help >>>>>>>>>>>>>> Phil >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Thursday, May 19, 2022 at 4:27:56 PM UTC+1 >>>>>>>>>>>>>>> [email protected] wrote: >>>>>>>>>>>>>>> You need to specify the rsync parameters in the >>>>>>>>>>>>>>> RealtimeGaugeData section of weewx.conf. Are you doing that? >>>>>>>>>>>>>>> Perhaps send the RealtimeGaugeData section of weewx.conf so we >>>>>>>>>>>>>>> can have a look. From your previous emails, it isn’t clear >>>>>>>>>>>>>>> that you have the parameters in the correct section. I can >>>>>>>>>>>>>>> tell you that this worked for me for a long time (in fact, I >>>>>>>>>>>>>>> submitted the pull request for this functionality). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> # Fill out the following fields: >>>>>>>>>>>>>>> # rsync_server : The server to which >>>>>>>>>>>>>>> gauge-data.txt will be copied. >>>>>>>>>>>>>>> # rsync_user : The userid on rsync_server with >>>>>>>>>>>>>>> write >>>>>>>>>>>>>>> # permission to >>>>>>>>>>>>>>> rsync_remote_rtgd_dir. >>>>>>>>>>>>>>> # rsync_remote_rtgd_dir : The directory on rsync_server >>>>>>>>>>>>>>> where >>>>>>>>>>>>>>> # gauge-data.txt will be copied. >>>>>>>>>>>>>>> # rsync_compress : True to compress the file before >>>>>>>>>>>>>>> sending. >>>>>>>>>>>>>>> # Default is False. >>>>>>>>>>>>>>> # rsync_log_success : True to write success with >>>>>>>>>>>>>>> timing messages to >>>>>>>>>>>>>>> # the log (for debugging). >>>>>>>>>>>>>>> Default is False. >>>>>>>>>>>>>>> # rsync_ssh_options : ssh options Default is '-o >>>>>>>>>>>>>>> ConnectTimeout=1' >>>>>>>>>>>>>>> # (When connecting, time out in 1 >>>>>>>>>>>>>>> second.) >>>>>>>>>>>>>>> # rsync_timeout : I/O timeout. Default is 1. >>>>>>>>>>>>>>> (When sending, >>>>>>>>>>>>>>> # timeout in 1 second.) >>>>>>>>>>>>>>> # rsync_skip_if_older_than : Don't bother to rsync if greater >>>>>>>>>>>>>>> than this >>>>>>>>>>>>>>> # number of seconds. Default is >>>>>>>>>>>>>>> 4. (Skip this >>>>>>>>>>>>>>> # and move on to the next if this >>>>>>>>>>>>>>> data is older >>>>>>>>>>>>>>> # than 4 seconds. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On May 19, 2022, at 6:37 AM, [email protected] >>>>>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Just to update >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Have a cron job working Ok with >>>>>>>>>>>>>>>> rsync -v /home/weewx/public_html/gauge-data.txt >>>>>>>>>>>>>>>> [email protected]:/volume1/web/SheringhamDDNS so the >>>>>>>>>>>>>>>> key/passwordless is working >>>>>>>>>>>>>>>> Changed >>>>>>>>>>>>>>>> rsync_server = xxx.xxx.x.xxx >>>>>>>>>>>>>>>> rsync_user = LushEr21 >>>>>>>>>>>>>>>> And it still doesnt work >>>>>>>>>>>>>>>> If somebody has tried this and got it to work your ideas on >>>>>>>>>>>>>>>> why I cant get rsync to work with RTGD would be most welcome >>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Tuesday, May 17, 2022 at 12:00:44 PM UTC+1 >>>>>>>>>>>>>>>>>> [email protected] wrote: >>>>>>>>>>>>>>>>>> Hi Guys >>>>>>>>>>>>>>>>>> Ive tried and failed to get the rsync to work using >>>>>>>>>>>>>>>>>> realtimeguagedata >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I never get passed gauge-data.txt file will not be exported. >>>>>>>>>>>>>>>>>> Here is an extract of the log. >>>>>>>>>>>>>>>>>> May 17 11:38:34 raspberrypi weewx-weewx[352] INFO user.rtgd: >>>>>>>>>>>>>>>>>> version is 0.5.5 >>>>>>>>>>>>>>>>>> May 17 11:38:34 raspberrypi weewx-weewx[352] INFO user.rtgd: >>>>>>>>>>>>>>>>>> RealTimeGaugeData scroller text will use a fixed string >>>>>>>>>>>>>>>>>> May 17 11:38:34 raspberrypi weewx-tides[363] INFO >>>>>>>>>>>>>>>>>> weewx.engine: Using archive interval of 600 seconds >>>>>>>>>>>>>>>>>> (specified in weewx configuration) >>>>>>>>>>>>>>>>>> May 17 11:38:34 raspberrypi weewx-weewx[352] INFO user.rtgd: >>>>>>>>>>>>>>>>>> gauge-data.txt will not be exported. >>>>>>>>>>>>>>>>>> May 17 11:38:34 raspberrypi weewx-weewx[352] INFO user.rtgd: >>>>>>>>>>>>>>>>>> '/home/weewx/public_html/gauge-data.txt' will be generated. >>>>>>>>>>>>>>>>>> min_interval is 15 seconds >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I have the following in the weewx.conf file >>>>>>>>>>>>>>>>>> rsync_server = SynLusher >>>>>>>>>>>>>>>>>> rsync_user = guest >>>>>>>>>>>>>>>>>> rsync_remote_rtgd_dir = /volume1/web/SheringhamDDNS >>>>>>>>>>>>>>>>>> rsync_log_success = True >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> weewx is version 4.5.1 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The HTTP POST is not being used. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I have set debug = 0, 1, 2 and 3 to try and find where it >>>>>>>>>>>>>>>>>> might be failing but its always the same. >>>>>>>>>>>>>>>>>> The server is a Synology NAS and I can copy from the >>>>>>>>>>>>>>>>>> rasberry pi to the rtgd_dir OK >>>>>>>>>>>>>>>>>> Just wondered if its possible to get more logs so give me a >>>>>>>>>>>>>>>>>> clue to where is failing >>>>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>>>> Phil >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>>>>> Google Groups "weewx-user" group. >>>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/39fff649-3aa4-4a7b-aab5-cd6628080faen%40googlegroups.com. >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>>> Google Groups "weewx-user" group. >>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/a68f9075-80c3-4980-a3bc-e80f3dfb2ce1n%40googlegroups.com. >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>>>> Groups "weewx-user" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>>> send an email to [email protected]. >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/dbca6111-df73-45be-8d3f-487a90e8bb7en%40googlegroups.com. >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "weewx-user" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>>> an email to [email protected]. >>>>>> >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/weewx-user/fd71e6f8-fa8b-4fab-bfe9-e9d9d0abd6cen%40googlegroups.com. >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google Groups >>>>> "weewx-user" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send an >>>>> email to [email protected]. >>>> >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/weewx-user/2db45023-d08b-4691-8609-e43b5ce2fba6n%40googlegroups.com. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >> >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/3af547de-13a5-4e41-9bb0-aadf108b4a52n%40googlegroups.com. > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/0cf5d4d4-b044-420d-8808-306365b69b76n%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/EC783367-C5BA-4AD8-913A-CD1E8594108F%40johnkline.com.
