On Saturday, October 8, 2016 at 5:23:30 PM UTC-4, Sam Roza wrote: > > > > On Saturday, October 8, 2016 at 1:53:31 PM UTC-7, mwall wrote: >> >> On Saturday, October 8, 2016 at 4:29:20 PM UTC-4, Sam Roza wrote: >>> >>> >>> With sed command, it strips the unwanted data from the output, but for >>> some reason, it's not working. I've tried with both curl and nc, but I >>> don't see any issues. Is there a way to debug this? >>> >> >> take it one step at a time. >> >> make sure that the interceptor is actually responding. with weewx >> running and debug=1, tail the log file in one shell while you do this in >> another: >> >> curl -s -d >> 'mac=0004a369e0d6&id=c2&pv=0&lb=0&ac=0®=1009&lost=0000&baro=1005&ptr=0&wfor=0&p=1' >> >> http://localhost:9999 >> <http://www.google.com/url?q=http%3A%2F%2Flocalhost%3A9999&sa=D&sntz=1&usg=AFQjCNH3GTDQFsI-ROBznvcn8KMGT66stQ> >> >> you should see a response from the interceptor in the weewx log output. >> what output do you see from curl? >> > > >>>Here is the Interceptor response to the curl command: > > ~~~ > Oct 8 14:02:40 raspberrypi weewx[6718]: interceptor: ServerThread: POST: > mac=0004a369e0d6&id=c2&pv=0&lb=0&ac=0®=1009&lost=0000&baro=1005&ptr=0&wfor=0&p=1 > Oct 8 14:02:40 raspberrypi weewx[6718]: interceptor: MainThread: raw > data: > mac=0004a369e0d6&id=c2&pv=0&lb=0&ac=0®=1009&lost=0000&baro=1005&ptr=0&wfor=0&p=1 > Oct 8 14:02:40 raspberrypi weewx[6718]: interceptor: MainThread: raw > packet: {'id..0004a369e0d6': 'c2', 'baro..0004a369e0d6': 1005.0, > 'mac..0004a369e0d6': '0004a369e0d6', 'ptr..0004a369e0d6': '0', > 'pv..0004a369e0d6': '0', 'lost..0004a369e0d6': '0000', 'dateTime': > 1475960561, 'reg..0004a369e0d6': '1009', 'wfor..0004a369e0d6': '0', > 'ac..0004a369e0d6': '0', 'p..0004a369e0d6': '1', 'usUnits': 17, > 'lb..0004a369e0d6': '0'} > Oct 8 14:02:40 raspberrypi weewx[6718]: interceptor: MainThread: mapped > packet: {'barometer': 1005.0, 'usUnits': 17, 'dateTime': 1475960561} > ~~~ > > Curl just presents the screen with a new line, so the command completes. > > >> >> from the other end, make sure that the ngrep-sed-curl chain is working. >> the main thing we are looking for here is that we get one curl connection >> for each line of data from ngrep. with weewx *not* running, run 'nc -l >> 9999' as a fake server in one shell, then do this as a 'client' in another >> shell: >> >> ngrep -l -q -d eth0 '0004a369e0d6'| sed -e 's/T.*//;/^\s*$/d' | curl -s >> -d @- http://localhost:9999 >> >> what do you see in the 'server' shell, and what do you see in the >> 'client' shell? i suspect we'll have to use xargs to split the newlines >> into separate curl invocations. >> >> m >> >> So, the ngrep-sed-curl chain is where it breaks down. The manual push of > data using curl works. The ngrep-sed command works. Putting them together > doesn't. > > I ran this command: > > # ngrep -l -q -d eth0 '0004a369e0d6'| sed -e 's/T.*//;/^\s*$/d' | curl -s > -d @- http://localhost:9999 > > While tailing /var/log/syslog: > > ~~~ > Oct 8 14:16:03 raspberrypi weewx[6718]: interceptor: MainThread: empty > queue > Oct 8 14:16:03 raspberrypi rsyslogd-2007: action 'action 17' suspended, > next retry is Sat Oct 8 14:17:03 2016 [try http://www.rsyslog.com/e/2007 > ] > Oct 8 14:16:13 raspberrypi weewx[6718]: interceptor: MainThread: empty > queue > Oct 8 14:16:23 raspberrypi weewx[6718]: interceptor: MainThread: empty > queue > Oct 8 14:16:33 raspberrypi weewx[6718]: interceptor: MainThread: empty > queue > Oct 8 14:16:43 raspberrypi weewx[6718]: interceptor: MainThread: empty > queue > ~~~ > > So during the same time, I ran another ngrep command to make sure it was > picking up packets during the same period of time: > > ~~~ > ^Croot@raspberrypi:/etc/weewx# ngrep -l -q -d eth0 '0004a369e0d6'| sed -e > 's/T.*//;/^\s*$/d' > interface: eth0 (192.168.1.0/255.255.255.0) > match: 0004a369e0d6 > > mac=0004a369e0d6&id=c2&pv=0&lb=0&ac=0®=1009&lost=0000&baro=1004&ptr=0&wfor=3&p=1 > > > > mac=0004a369e0d6&id=90&rid=d0&pwr=0&gw=0&av=0&wd=225&wg=0.9&ws=0.0&ch=1&p=1 > > ~~~ > > I saw a note about adding the unbuffered option to sed, but that doesn't > seem to be the issue. No change with -u option enabled. > > I did want to be sure interceptor was still working, so I pushed that curl > command again and here was the log response: > > ~~~ > Oct 8 14:21:07 raspberrypi weewx[6718]: interceptor: ServerThread: POST: > mac=0004a369e0d6&id=c2&pv=0&lb=0&ac=0®=1009&lost=0000&baro=1005&ptr=0&wfor=0&p=1 > Oct 8 14:21:07 raspberrypi weewx[6718]: interceptor: MainThread: raw > data: > mac=0004a369e0d6&id=c2&pv=0&lb=0&ac=0®=1009&lost=0000&baro=1005&ptr=0&wfor=0&p=1 > Oct 8 14:21:07 raspberrypi weewx[6718]: interceptor: MainThread: raw > packet: {'id..0004a369e0d6': 'c2', 'baro..0004a369e0d6': 1005.0, > 'mac..0004a369e0d6': '0004a369e0d6', 'ptr..0004a369e0d6': '0', > 'pv..0004a369e0d6': '0', 'lost..0004a369e0d6': '0000', 'dateTime': > 1475961667, 'reg..0004a369e0d6': '1009', 'wfor..0004a369e0d6': '0', > 'ac..0004a369e0d6': '0', 'p..0004a369e0d6': '1', 'usUnits': 17, > 'lb..0004a369e0d6': '0'} > Oct 8 14:21:07 raspberrypi weewx[6718]: interceptor: MainThread: mapped > packet: {'barometer': 1005.0, 'usUnits': 17, 'dateTime': 1475961667} > Oct 8 14:21:07 raspberrypi weewx[6718]: manager: added record 2016-10-08 > 14:10:00 PDT (1475961000) to database 'weewx.sdb' > Oct 8 14:21:07 raspberrypi weewx[6718]: manager: added record 2016-10-08 > 14:10:00 PDT (1475961000) to daily summary in 'weewx.sdb' > Oct 8 14:21:07 raspberrypi weewx[6718]: restx: CWOP: record 2016-10-08 > 14:10:00 PDT (1475961000) is stale (667 > 60). > Oct 8 14:21:07 raspberrypi weewx[6718]: reportengine: Running reports for > latest time in the database. > Oct 8 14:21:07 raspberrypi weewx[6718]: reportengine: Running report > StandardReport > Oct 8 14:21:07 raspberrypi weewx[6718]: reportengine: Found configuration > file /etc/weewx/skins/Standard/skin.conf for report StandardReport > Oct 8 14:21:07 raspberrypi weewx[6718]: restx: Wunderground-PWS: > Published record 2016-10-08 14:10:00 PDT (1475961000) > Oct 8 14:21:09 raspberrypi weewx[6718]: cheetahgenerator: Generated 14 > files for report StandardReport in 2.07 seconds > Oct 8 14:21:11 raspberrypi weewx[6718]: genimages: Generated 12 images > for StandardReport in 1.25 seconds > ~~~ > > Matt, might it be the whitespace that I seem to see in front of the data? >
that is possible, but not likely (LW30x.parse_identifiers does a split, so the spaces would be pre-pended to the dict key for 'mac'). you can easily verify by doing the first test with this: curl -s -d ' mac=0004a369e0d6&id=c2&pv=0&lb=0&ac=0®=1009&lost=0000&baro=1005&ptr=0&wfor=0&p=1' http://localhost:9999 notice the leading spaces in the data string. more likely is that curl is waiting for the end of the output from ngrep instead of executing on each newline. try the ngrep-sed-curl test like this instead: ngrep -l -q -d eth0 '0004a369e0d6'| sed -e 's/T.*//;/^\s*$/d' | xargs curl http://localhost:9999 -s -d <http://localhost:9999> m -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
