If you are taking historic data from the WU databases why not use wee_import which does all the rain calculations from wunderground to weewx for you – together with any other conversions as required.
I can understand your position for live data if you are sniffing the data BEFORE it is uploaded to WU, but your last comment implies you are using historic WU data – in which case wee_import does the jobe perfectly AFIK. I certainly used it OK for 3 yrs worth of data. Sent from Mail for Windows 10 From: Pete Nevill Sent: 19 May 2017 12:04 To: [email protected] Subject: Re: [weewx-user] Re: Wunderground import SOLVED (Hopefully).... Three steps to get this working. 1) As Gary pointed out. Use the cumulative rain (since midnight) to compute the rain since last update.So that the total rain _packet['rain'] is correct for the update period (in my case this is 5min). 2) Only send this information to weewx, ignore all the other rain parameters, i.e. let weewx handle that. 3) as per step 2) set in weewx.conf calculations section "rainRate" and "dayRain" to "software". Initial signs are looking good. Although after 24hrs of heavy rain here in West London we now have blue sky! Thanks for all you help and guidance Gary. Just need to extract my correct rain data from Wunderground and update weewx db... Pete. F.Y.I. here is a link to my PWS on wundergound. I have a webcam attached, note I have overlain with imagemagik output from weewx (info created using cheetah) on the webcam image. Hope you like it. https://www.wunderground.com/personal-weather-station/dashboard?ID=ILONDON542 On Fri, May 19, 2017 at 9:00 AM, Pete Nevill <[email protected]> wrote: Gary, wow... now I feel stupid... The answer was staring me in the face...(even had a warning) def check_rain(self, daily_rain_counter): # *** DO NOT use the &rainin= data! *** # Handle the rain accum by taking the &dailyrainin= reading ONLY. # Then submit those minor increments of daily rain to weewx. rain = 0.0 current_rain = float(daily_rain_counter) if self.lastrain is not None: if (current_rain >= self.lastrain): rain = float(current_rain) - float(self.lastrain) self.lastrain = current_rain return rain Although looking at this, is there an issue with flip after midnight? Assuming it is raining and the bucket tipped just after midnight but before the packet is sent we will have a situation where current_rain < self.lastrain (but not zero). self.lastrain will be updated but rain will be zero. Next packet will be the difference, and so miss the first bucket tip of the day. Now on a 5 minute interval that I have set up I am sure this is that significant but just thought I would raise it as a potential problem. As such I added an else (where current_rain < self.lastrain, simply passes total since midnight): rain = float(current_rain) I appreciate your comments regarding careful use of drivers etc as well as space/indents in my python script. As you can tell I am enthusiastic amateur...! I am still a little lost as to why my daily rain value reads zero after forcing (faking) the rainrate (constant 2mm) for over 24hrs. Thanks again, Pete. On Wed, May 17, 2017 at 2:13 PM, gjr80 <[email protected]> wrote: Hi Pete, A few thoughts for you as you continue... I know you have just posted a small extract from your weewx.conf so we cannot see your complete setup, but be careful with your 'drivers'. The weeWX architecture basically supports one 'driver' and multiple 'services'. So for example some of the folks that have a PWS and one or more one-wire sensors might be running the driver for their PWS and a one-wire service to pick up data from their (separate to the PWS) one-wire sensors. It might be just semantics but something to keep in mind. If you are just using a socketlogger.py-based driver then I would expect that woudl be your driver and thought should be all you need weeWX wise. As mentioned previously, the weeWX rain field holds the total rainfall that ocurred over the period concerned. For loop data (ie data that comes in regualrly from your PWS, typically several times each archive period) this is the total rainfall since the last loop packet. For archive data (ie the accumulated data over an archive period, say 5 minutes) this is the total rainfall over the archive period. WeeWX can calculate rain rate from rain data but cannot calculate rain from rain rate data. If you look in the socketlogger driver you will see that each data packet it receives has a running total, the socketlogger keeps track of the previous running total value and calculates the difference (refer check_rain() method). This difference is what is yielded by the driver. 'Getting data into weeWX'. In its simplest form the driver decodes data from the data source (nominally a PWS), does any processing required (eg calculates rain from a running total) and then yields a loop packet. This packet may contain data from some or all sensors. The weeWX machinery then accumulates these loop packets, and at a fixed interval (the archive interval), runs the accumulated data through various services before it is stored in the database and reports generated based on the accumulated data. The process then repeats until the next archive interval is complete. The drivers job is to continuously decodes raw sensor data and publish loop packets. Of course there is a format for the loop packets coming out of the driver; they use a python dictionary with a number of mandatory fields (dateTime - a timestamp for the packet and usUnits - the unit system used in the packet) and one or more observation fields eg outTemp, inHumidity etc. These observation fields are typically a weeWX archive table field name though they do not have to be. Basically, if your driver emits outTemp and outTemp exists in the weeWX database schema, weeWx will automatically store the outTemp data in the outTemp related fields in the database. The key method in a driver is genLoopPackets(), this method is responsible for yielding the loop packets used by the rest of weeWX. I am not sure if it was the process of posting to Google groups or if it was in the source file, but it appears that the socketlogger.py you posted has a tabs in it. Python has some strict indentation rules and whilst tabs can be made to work, it is likely that you will eventually trip yourself up as spaces get mixed in with the tabs. Better to get yourself a decent editor and set it to uses spaces only (when you hit the tab key). If you have not already read them you might find the following give some insight into how weeWX works and how to customise it: - Customization guide - Introduction - Customization Guide - Porting to new hardware - Notes for Developers - various drivers, fileparse.py (examples/fileparse/bin/user/fileparse.py) is a basic one others are in /home/weewx/bin/weewx/drivers, file size is probably a good indicator of the simpler v more complex drivers. Hope this helps. Gary On Wednesday, 17 May 2017 19:38:10 UTC+10, Pete Nevill wrote: Gary, Many thanks for your reply. This is very helpful. First issue fixed. units from Wunderground url as you say are in in/hr so I fixed that, i.e. /25.4. I have written my own driver based on one called socketlogge from here https://github.com/poblabs/weewx-socketlogger in my weewx.conf file i specify the driver and parameters. [W8681] # This section is for the W8681 wifi series of weather stations. # The driver to use: driver = weewx.drivers.w8681 [SocketLogger] # This section is for the SocketLogger driver # Station info: hardware = Generic intercept for wunderground urls # IP address, port number and timeout in seconds of the socket to connect to mode = sniff # driver mode either sniff of listen host_ip = XX.XX.XX.XX # in sniff mode ip of weather station else should be localhost for listen host_iface = eth1 # only required in sniff mode host_port = 80 # port to listen on or port to sniff timeout = 320 # timeout in seconds # this should be greater than the interval set on your pws # The driver to use driver = user.socketlogger I have attached the py script (still work in progress so use at own risk). Passes data to weewex with (for example) _packet['temp']. Not sure if this simply inserts data into the database or whether weewx actually "processes" the data. I am just trying to figure out how the rainrate is handled. Is weewx smart enough to know if I provide a rain rate of mm/hr every 5 minutes what the cumulative rainfall is? Just started running my script with a constant rainrate for 24hrs, will see what happens. So far the rain rate on the web interface is correct. Although the "Today's Rain" still says zero. Will let it run for a few days (we should have a total of 48mm tomorrow). Pete. -- You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/4MWfKEqNO2A/unsubscribe. To unsubscribe from this group and all its topics, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/4MWfKEqNO2A/unsubscribe. To unsubscribe from this group and all its topics, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
