On Saturday, October 8, 2016 at 2:44:15 PM UTC-7, mwall wrote: > > > > On Saturday, October 8, 2016 at 5:23:30 PM UTC-4, Sam Roza wrote: >> >> >> >> On Saturday, October 8, 2016 at 1:53:31 PM UTC-7, mwall wrote: >>> >>> On Saturday, October 8, 2016 at 4:29:20 PM UTC-4, Sam Roza wrote: >>>> >>>> >>>> With sed command, it strips the unwanted data from the output, but for >>>> some reason, it's not working. I've tried with both curl and nc, but I >>>> don't see any issues. Is there a way to debug this? >>>> >>> >>> take it one step at a time. >>> >>> make sure that the interceptor is actually responding. with weewx >>> running and debug=1, tail the log file in one shell while you do this in >>> another: >>> >>> curl -s -d >>> 'mac=0004a369e0d6&id=c2&pv=0&lb=0&ac=0®=1009&lost=0000&baro=1005&ptr=0&wfor=0&p=1' >>> >>> http://localhost:9999 >>> <http://www.google.com/url?q=http%3A%2F%2Flocalhost%3A9999&sa=D&sntz=1&usg=AFQjCNH3GTDQFsI-ROBznvcn8KMGT66stQ> >>> >>> you should see a response from the interceptor in the weewx log output. >>> what output do you see from curl? >>> >> >> >>>Here is the Interceptor response to the curl command: >> >> ~~~ >> Oct 8 14:02:40 raspberrypi weewx[6718]: interceptor: ServerThread: POST: >> mac=0004a369e0d6&id=c2&pv=0&lb=0&ac=0®=1009&lost=0000&baro=1005&ptr=0&wfor=0&p=1 >> Oct 8 14:02:40 raspberrypi weewx[6718]: interceptor: MainThread: raw >> data: >> mac=0004a369e0d6&id=c2&pv=0&lb=0&ac=0®=1009&lost=0000&baro=1005&ptr=0&wfor=0&p=1 >> Oct 8 14:02:40 raspberrypi weewx[6718]: interceptor: MainThread: raw >> packet: {'id..0004a369e0d6': 'c2', 'baro..0004a369e0d6': 1005.0, >> 'mac..0004a369e0d6': '0004a369e0d6', 'ptr..0004a369e0d6': '0', >> 'pv..0004a369e0d6': '0', 'lost..0004a369e0d6': '0000', 'dateTime': >> 1475960561, 'reg..0004a369e0d6': '1009', 'wfor..0004a369e0d6': '0', >> 'ac..0004a369e0d6': '0', 'p..0004a369e0d6': '1', 'usUnits': 17, >> 'lb..0004a369e0d6': '0'} >> Oct 8 14:02:40 raspberrypi weewx[6718]: interceptor: MainThread: mapped >> packet: {'barometer': 1005.0, 'usUnits': 17, 'dateTime': 1475960561} >> ~~~ >> >> Curl just presents the screen with a new line, so the command completes. >> >> >>> >>> from the other end, make sure that the ngrep-sed-curl chain is working. >>> the main thing we are looking for here is that we get one curl connection >>> for each line of data from ngrep. with weewx *not* running, run 'nc -l >>> 9999' as a fake server in one shell, then do this as a 'client' in another >>> shell: >>> >>> ngrep -l -q -d eth0 '0004a369e0d6'| sed -e 's/T.*//;/^\s*$/d' | curl -s >>> -d @- http://localhost:9999 >>> >>> what do you see in the 'server' shell, and what do you see in the >>> 'client' shell? i suspect we'll have to use xargs to split the newlines >>> into separate curl invocations. >>> >>> m >>> >>> So, the ngrep-sed-curl chain is where it breaks down. The manual push of >> data using curl works. The ngrep-sed command works. Putting them together >> doesn't. >> >> I ran this command: >> >> # ngrep -l -q -d eth0 '0004a369e0d6'| sed -e 's/T.*//;/^\s*$/d' | curl -s >> -d @- http://localhost:9999 >> >> While tailing /var/log/syslog: >> >> ~~~ >> Oct 8 14:16:03 raspberrypi weewx[6718]: interceptor: MainThread: empty >> queue >> Oct 8 14:16:03 raspberrypi rsyslogd-2007: action 'action 17' suspended, >> next retry is Sat Oct 8 14:17:03 2016 [try http://www.rsyslog.com/e/2007 >> ] >> Oct 8 14:16:13 raspberrypi weewx[6718]: interceptor: MainThread: empty >> queue >> Oct 8 14:16:23 raspberrypi weewx[6718]: interceptor: MainThread: empty >> queue >> Oct 8 14:16:33 raspberrypi weewx[6718]: interceptor: MainThread: empty >> queue >> Oct 8 14:16:43 raspberrypi weewx[6718]: interceptor: MainThread: empty >> queue >> ~~~ >> >> So during the same time, I ran another ngrep command to make sure it was >> picking up packets during the same period of time: >> >> ~~~ >> ^Croot@raspberrypi:/etc/weewx# ngrep -l -q -d eth0 '0004a369e0d6'| sed -e >> 's/T.*//;/^\s*$/d' >> interface: eth0 (192.168.1.0/255.255.255.0) >> match: 0004a369e0d6 >> >> mac=0004a369e0d6&id=c2&pv=0&lb=0&ac=0®=1009&lost=0000&baro=1004&ptr=0&wfor=3&p=1 >> >> >> >> mac=0004a369e0d6&id=90&rid=d0&pwr=0&gw=0&av=0&wd=225&wg=0.9&ws=0.0&ch=1&p=1 >> >> ~~~ >> >> I saw a note about adding the unbuffered option to sed, but that doesn't >> seem to be the issue. No change with -u option enabled. >> >> I did want to be sure interceptor was still working, so I pushed that >> curl command again and here was the log response: >> >> ~~~ >> Oct 8 14:21:07 raspberrypi weewx[6718]: interceptor: ServerThread: POST: >> mac=0004a369e0d6&id=c2&pv=0&lb=0&ac=0®=1009&lost=0000&baro=1005&ptr=0&wfor=0&p=1 >> Oct 8 14:21:07 raspberrypi weewx[6718]: interceptor: MainThread: raw >> data: >> mac=0004a369e0d6&id=c2&pv=0&lb=0&ac=0®=1009&lost=0000&baro=1005&ptr=0&wfor=0&p=1 >> Oct 8 14:21:07 raspberrypi weewx[6718]: interceptor: MainThread: raw >> packet: {'id..0004a369e0d6': 'c2', 'baro..0004a369e0d6': 1005.0, >> 'mac..0004a369e0d6': '0004a369e0d6', 'ptr..0004a369e0d6': '0', >> 'pv..0004a369e0d6': '0', 'lost..0004a369e0d6': '0000', 'dateTime': >> 1475961667, 'reg..0004a369e0d6': '1009', 'wfor..0004a369e0d6': '0', >> 'ac..0004a369e0d6': '0', 'p..0004a369e0d6': '1', 'usUnits': 17, >> 'lb..0004a369e0d6': '0'} >> Oct 8 14:21:07 raspberrypi weewx[6718]: interceptor: MainThread: mapped >> packet: {'barometer': 1005.0, 'usUnits': 17, 'dateTime': 1475961667} >> Oct 8 14:21:07 raspberrypi weewx[6718]: manager: added record 2016-10-08 >> 14:10:00 PDT (1475961000) to database 'weewx.sdb' >> Oct 8 14:21:07 raspberrypi weewx[6718]: manager: added record 2016-10-08 >> 14:10:00 PDT (1475961000) to daily summary in 'weewx.sdb' >> Oct 8 14:21:07 raspberrypi weewx[6718]: restx: CWOP: record 2016-10-08 >> 14:10:00 PDT (1475961000) is stale (667 > 60). >> Oct 8 14:21:07 raspberrypi weewx[6718]: reportengine: Running reports >> for latest time in the database. >> Oct 8 14:21:07 raspberrypi weewx[6718]: reportengine: Running report >> StandardReport >> Oct 8 14:21:07 raspberrypi weewx[6718]: reportengine: Found >> configuration file /etc/weewx/skins/Standard/skin.conf for report >> StandardReport >> Oct 8 14:21:07 raspberrypi weewx[6718]: restx: Wunderground-PWS: >> Published record 2016-10-08 14:10:00 PDT (1475961000) >> Oct 8 14:21:09 raspberrypi weewx[6718]: cheetahgenerator: Generated 14 >> files for report StandardReport in 2.07 seconds >> Oct 8 14:21:11 raspberrypi weewx[6718]: genimages: Generated 12 images >> for StandardReport in 1.25 seconds >> ~~~ >> >> Matt, might it be the whitespace that I seem to see in front of the data? >> > > that is possible, but not likely (LW30x.parse_identifiers does a split, so > the spaces would be pre-pended to the dict key for 'mac'). > > you can easily verify by doing the first test with this: > > curl -s -d ' > mac=0004a369e0d6&id=c2&pv=0&lb=0&ac=0®=1009&lost=0000&baro=1005&ptr=0&wfor=0&p=1' > > http://localhost:9999 > > notice the leading spaces in the data string. > > more likely is that curl is waiting for the end of the output from ngrep > instead of executing on each newline. > > try the ngrep-sed-curl test like this instead: > > ngrep -l -q -d eth0 '0004a369e0d6'| sed -e 's/T.*//;/^\s*$/d' | xargs curl > http://localhost:9999 > -s -d <http://localhost:9999> > > m >
You were right about the whitespace. No change. So I tested the command string with xargs and no change. I broke the command up again, to ensure that sed was passing only the data along, and again, it printed the data on the screen. So that was working. I pushed some actual data using curl (without xargs, because xargs did nothing) and it again was logged. Here are some of the attempts I made this time around: root@raspberrypi:/home/pi/Downloads# ngrep -l -q -d eth0 '0004a369e0d6'| sed -e 's/T.*//;/^\s*$/d' | curl -s -d @- http://localhost:9999 > /dev/null^C root@raspberrypi:/home/pi/Downloads# ngrep -l -q -d eth0 '0004a369e0d6'| sed -e 's/T.*//;/^\s*$/d' | xargs curl http://localhost:9999 -s -d ^C root@raspberrypi:/home/pi/Downloads# ngrep -l -q -d eth0 '0004a369e0d6'| sed -e 's/T.*//;/^\s*$/d' | xargs curl http://localhost:9999 -s -d @- ^C root@raspberrypi:/home/pi/Downloads# ngrep -l -q -d eth0 '0004a369e0d6'| sed -e 's/T.*//;/^\s*$/d' | xargs curl http://localhost:9999 -N -s -d @- ^C None of them seemed to ever do anything when fully put together like this. Note that I did add '@-' to the end of the command with the xargs on it, since I believe it is necessary when using the -d option. -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
