Hi Manuel, I don't yet know the reason for the error in time format,
however that was a debugging statement, so I can comment out the error for
now.
Try this v1.2 !
Thanks, Rich
On Sunday, April 8, 2018 at 9:44:53 AM UTC-7, Manuel wrote:
>
> Hello Rich I have been stealing the second file storm.py and I have this
> error when generating the report:
>
> Apr 8 02:56:24 raspberrypi weewx[3785]: reportengine: Caught
> unrecoverable exception ingenerator weewx.cheetahgenerator.CheetahGenerator
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** time data
> '2017-07-05 16:00:00-PST' does not match format '%Y-%m-%d %H:%M:%S-%Z'
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** Traceback (most
> recent call last):
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/share/weewx/weewx/reportengine.py", line 239, in run
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** obj.start()
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/share/weewx/weewx/reportengine.py", line 273, in start
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** self.run()
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/share/weewx/weewx/cheetahgenerator.py", line 158, in run
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** ngen =
> self.generate(gen_dict[section_name], self.gen_ts)
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/share/weewx/weewx/cheetahgenerator.py", line 232, in generate
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** ngen +=
> self.generate(section[subsection], gen_ts)
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/share/weewx/weewx/cheetahgenerator.py", line 232, in generate
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** ngen +=
> self.generate(section[subsection], gen_ts)
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/share/weewx/weewx/cheetahgenerator.py", line 320, in generate
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** default_binding)
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/share/weewx/weewx/cheetahgenerator.py", line 372, in _getSearchList
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** searchList +=
> obj.get_extension_list(timespan, db_lookup)
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/share/weewx/user/storm.py", line 100, in get_extension_list
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** StructTime =
> time.strptime('2017-07-05 16:00:00-PST', '%Y-%m-%d %H:%M:%S-%Z')
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/lib/python2.7/_strptime.py", line 467, in _strptime_time
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** return
> _strptime(data_string,format)[0]
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/lib/python2.7/_strptime.py", line 325, in _strptime
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** (data_string,
> format))
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** ValueError: time
> data '2017-07-0516:00:00-PST' does not match format '%Y-%m-%d %H:%M:%S-%Z'
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** Generator terminated
> Apr 8 02:56:24 raspberrypi weewx[3785]: reportengine: Caught
> unrecoverable exception ingenerator weewx.cheetahgenerator.CheetahGenerator
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** time data
> '2017-07-05 16:00:00-PS' does not match format '%Y-%m-%d %H:%M:%S-%Z'
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** Traceback (most
> recent call last):
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/share/weewx/weewx/reportengine.py", line 239, in run
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** obj.start()
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/share/weewx/weewx/reportengine.py", line 273, in start
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** self.run()
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/share/weewx/weewx/cheetahgenerator.py", line 158, in run
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** ngen =
> self.generate(gen_dict[section_name], self.gen_ts)
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/share/weewx/weewx/cheetahgenerator.py", line 232, in generate
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** ngen +=
> self.generate(section[subsection], gen_ts)
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/share/weewx/weewx/cheetahgenerator.py", line 232, in generate
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** ngen +=
> self.generate(section[subsection], gen_ts)
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/share/weewx/weewx/cheetahgenerator.py", line 320, in generate
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** default_binding)
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/share/weewx/weewx/cheetahgenerator.py", line 372, in _getSearchList
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** searchList +=
> obj.get_extension_list(timespan, db_lookup)
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/share/weewx/user/storm.py", line 100, in get_extension_list
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** StructTime =
> time.strptime('2017-07-05 16:00:00-PST', '%Y-%m-%d %H:%M:%S-%Z')
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/lib/python2.7/_strptime.py", line 467, in _strptime_time
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** return
> _strptime(data_string,format)[0]
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** File
> "/usr/lib/python2.7/_strptime.py", line 325, in _strptime
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** (data_string,
> format))
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** ValueError: time
> data '2017-07-0516:00:00-PST' does not match format '%Y-%m-%d %H:%M:%S-%Z'
> Apr 8 02:56:24 raspberrypi weewx[3785]: **** Generator terminated
>
> El sábado, 7 de abril de 2018, 23:36:42 (UTC+2), Rich Altmaier escribió:
>>
>> I have an update, storm v1.1. I misunderstood the timestamp on archive
>> records. I now see the timestamp value is the time at end of the interval
>> of rain accumulation. Fixed that!
>> Rich
>>
>>
>> On Thursday, April 5, 2018 at 4:39:41 PM UTC-7, Rich Altmaier wrote:
>>>
>>> This is my first try at an extension. This is one file to place in user
>>> (/usr/share/weewx/user), with info in the header for the skin.conf and
>>> Standard/index.html.tmpl changes to place the data into a report.
>>> Please do give feedback!
>>> Thanks, Rich
>>>
>>>
>>>
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
# Copyright 2018 Rich Altmaier [email protected]
#
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
"""
storm.py v1.2
This search list extension calculates tags, one is a time interval,
and one is a duration, of the most recent rainstorm:
'last_rain_storm': finds the last rain interval, allowing stats to
be calculated over that interval.
We search back to find the storm end, which might be still on-going,
and search still further back to find the rain start (after a dry period)
'storm_duration': duration in days
*******************************************************************************
Usage:
1) copy this file to the user directory
/usr/share/weewx/user
2) add the option search_list_extensions in the
/etc/weewx/skins/Standard/skin.conf configuration file,
adding the name of this extension. When you're done,
it will look like this:
[CheetahGenerator]
search_list_extensions = user.storm.RainStorm
3) in the html template /etc/weewx/skins/Standard/index.html.tmpl
in the Current Conditions section, perhaps after Inside Temperature,
add this table row:
<tr>
<td class="stats_label">last storm total rain,
<br />ending $last_rain_storm.end
<br />of duration $storm_duration days</td>
<td class="stats_data">$last_rain_storm.rain.sum</td>
</tr>
You can also use other tags such as $last_rain_storm.outTemp.max for the max
temperature, or $last_rain_storm.rain.sum for the total rainfall in the last
storm
*******************************************************************************
"""
import datetime
import time
from weewx.cheetahgenerator import SearchList
from weewx.tags import TimespanBinder
from weeutil.weeutil import TimeSpan
class RainStorm(SearchList):
def __init__(self, generator):
SearchList.__init__(self, generator)
def get_extension_list(self, timespan, db_lookup):
"""Returns a search list extension for storm detection
Parameters:
timespan: An instance of weeutil.weeutil.TimeSpan. This will
hold the start and stop times of the domain of
valid times.
db_lookup: This is a function that, given a data binding
as its only parameter, will return a database manager
object.
"""
# first find end of most recent storm.
# may be still ongoing (e.g. rain in most recent record).
# parameterized with:
storm_dry_period = 24 # number of hours of no rain to delimit
# storm end and storm start.
storm_in_past = 20 # number of days to look back for a storm
storm_now = int(time.time() ) # right now this second
# can choose other epoch value for testing!
# set now to a testing time, to look at various past rain records
#StructTime = time.strptime('2017-02-05 16:00:00-PST', '%Y-%m-%d %H:%M:%S-%Z')
# 0.80in, from 2017-02-01 to 2017-02-04
#StructTime = time.strptime('2017-01-05 16:00:00-PST', '%Y-%m-%d %H:%M:%S-%Z')
# 1.58 in, from 2017-01-01 to 2017-01-05
#StructTime = time.strptime('2017-07-05 16:00:00-PST', '%Y-%m-%d %H:%M:%S-%Z')
# 0.0 in.
#storm_now = int(time.mktime(StructTime))
wx_db_mgr = db_lookup()
c = wx_db_mgr.connection.cursor()
#caching method:
# temporary table RainStormcache,
# fields: entryEpoch, startE, endE all are Integer
# just one record
# when entryEpoch+N > now, then the entry is recent enough
CacheValid = False
NowEpoch = int(time.time() )
c.execute("PRAGMA table_info(RainStormcache)")
CacheInfo = c.fetchone()
if CacheInfo is not None:
c.execute("select count(*) from RainStormcache")
#print "number of rows in storm cache ", int(c.fetchone()[0])
c.execute("SELECT * from RainStormcache")
CacheData = c.fetchone()
#print "CacheData ", CacheData
CacheEpoch = CacheData[0]
StartTepoch = CacheData[1]
storm_end_epoch = CacheData[2]
if CacheEpoch+30 > NowEpoch: # if aged cache is still ahead of now...
CacheValid = True
#print "Cache valid ", CacheValid, " NowEpoch ", NowEpoch, " start ", StartTepoch, " end ", storm_end_epoch
#end cache fetch. CacheValid tells us if data found
if CacheValid == False:
c.execute("drop table IF EXISTS RainWeek")
c.execute("""
create TEMPORARY table RainWeek as
select dateTime, rain,
datetime(dateTime, 'unixepoch', 'localtime') as dateString,
dateTime as startEpoch, dateTime as endEpoch
from archive where dateTime >= :now - (24 * 3600 * :past )
AND dateTime <= :now
ORDER BY dateTime
""", {"now": storm_now, "past": storm_in_past})
wx_db_mgr.connection.commit()
c.execute("select count(*) from temp.RainWeek")
print "number of rows in storm viewing table is ", int(c.fetchone()[0])
c.execute("""
-- find end of most recent wet period ending at or before storm_now
-- or storm_now if has been wet within storm_dry_period hours of now.
-- Note: dateTime stamp is end time of that row of accum rainfall
select CASE
WHEN
((select MAX(dateTime) as RainEndT from temp.RainWeek where rain <> 0 )
+ (:dry * 3600))
>
:now
THEN
:now -- storm is still on-going
ELSE -- find most recent last wet time
(select MAX(dateTime) as LastWetT from temp.RainWeek where rain <> 0 )
--above is most recent record with rainfall
END as RainEndTime
""", {"now": storm_now, "past": storm_in_past, "dry": storm_dry_period})
EndT = c.fetchone()
if EndT[0] is None:
StormFound = False
print "No storm in past", storm_in_past, " days"
storm_end_epoch = storm_now - storm_in_past*24*3600 -1
else:
StormFound = True
#print "most recent wet record ", EndT
storm_end_epoch = int(EndT[0])
print "Storm end epoch ", storm_end_epoch, ", local ", time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(storm_end_epoch))
if StormFound:
c.execute("""
-- find dry period preceeding storm_end_epoch.
-- look at storm_dry_period hours of records preceeding each record to find
-- a preceeding sequence of 0 rain lasting storm_dry_period.
select MAX(dateTime) as RainStartT from
(select f1.rowid AS ROWID, sum(f2.rain) as RunningRain, f1.dateTime
FROM temp.RainWeek f1 INNER JOIN temp.RainWeek f2
ON f1.dateTime >= f2.dateTime AND f2.dateTime > (f1.dateTime - ( :dry *3600))
GROUP BY ROWID
HAVING RunningRain = 0 and f1.dateTime < :ending -- Rain end time
)
""", {"ending": storm_end_epoch, "dry": storm_dry_period})
row = c.fetchone()
if row[0] is not None:
StartTepoch = int(row[0])
print "Storm start time epoch ", StartTepoch, ", local ", time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(StartTepoch))
else:
StartTepoch = storm_now + storm_in_past*24*3600
print "No dry start found in search interval"
#if we hit the earliest record in the TEMPORARY table, then
# there is really not a beginning of storm found,
# and our rain accum calculation will start to reduce as
# early portions of the storm go out of our search window.
#Therefore don't match a storm which is getting too old!
if StartTepoch < storm_now - storm_in_past*24*2600:
#also interpret this rain receeding into the past as
#no storm found
print "Storm is too far past, dont use it"
StartTepoch = storm_now -1
storm_end_epoch = storm_now
else: # no rain :-(
StartTepoch = storm_now -1
storm_end_epoch = storm_now
#now stuff data into the cache
c.execute("drop table IF EXISTS RainStormcache")
c.execute("""create TEMPORARY table RainStormcache (
CacheEpoch integer not null,
CacheStartE integer not null,
CacheEndE integer not null)
""")
c.execute("""INSERT INTO RainStormcache VALUES (
:stamp , :start , :end )
""", {"stamp": int(time.time()), "start": StartTepoch, "end": storm_end_epoch})
#end of if not CacheValid
#have storm delimiting times, form TimespanBinder
# Form a TimespanBinder object, using the time span we just
# calculated:
LastStorm = TimespanBinder(TimeSpan(StartTepoch, storm_end_epoch),
db_lookup,
formatter=self.generator.formatter,
converter=self.generator.converter,
skin_dict=self.generator.skin_dict)
StormDurationDays = ( storm_end_epoch - StartTepoch +(3600*24)-2) / (3600*24)
# Now create a small dictionary
search_list_extension = {
'last_rain_storm' : LastStorm,
'storm_duration' : StormDurationDays}
# Finally, return our extension as a list:
return [search_list_extension]
