The red line is just like you expect: the "outTemp" for each archive
record, no aggregation. The chart library I use to draw the plot, has a
feature to show things like the max and min value of the plot. The user (in
that case: myself) wants to have the min/max of each day shown in the plot.
If one doesn't know, that the archive doesn't contain the daily min/max
values, but the daily summaries do, he might be confused that the plot
shows a different maximum, than the NOAA data show, even if it is just a
little off. There is a perfectly "legal" technical explanation, why this is
the case. Anyway, if the max of the plot matches the max of a particular
day, no questions ever will be asked in that regard.
What I will do, is augment the archive data with the max/min values and
their timestamps, and combine archive with archive_day, if the min/max
values from the daily summary is within the timestamp.
import weecfg
import weewx.manager
import weewx.xtypes
from weeutil.weeutil import TimeSpan
from datetime import datetime, time, timedelta
config_path, config_dict = weecfg.read_config(
'/home/michi/weewx-data/weewx.conf')
with weewx.manager.open_manager_with_config(config_dict, 'wx_binding') as
dbmanager:
start_time = 1726347600
end_time = datetime.now()
start_of_day = datetime.combine(datetime.fromtimestamp(start_time), time
.min)
min_list = []
while start_of_day < end_time:
min_tupel = []
start_of_next_day = start_of_day + timedelta(days=1)
min_value = weewx.xtypes.get_aggregate("outTemp", TimeSpan(
start_of_day.timestamp(), start_of_next_day.timestamp()), "min", dbmanager)[
0]
min_time = weewx.xtypes.get_aggregate("outTemp", TimeSpan(
start_of_day.timestamp(), start_of_next_day.timestamp()), "mintime",
dbmanager)[0]
min_tupel.append(min_time * 1000)
min_tupel.append(min_value)
if min_time >= start_time and min_time <= end_time.timestamp():
min_list.append(min_tupel)
start_of_day = start_of_next_day
print(min_list)
The result (min_list) will be a list of timestamp/values, that will be
inserted in the data series for the plot.
Tom Keffer schrieb am Sonntag, 15. September 2024 um 17:23:01 UTC+2:
> I don't know what aggregation type, if any, was used for the red line.
> Most likely no aggregation type was called for, in which case the curve
> will represent "outTemp" for each archive record, which by default (it
> depends on the accumulator extractor type) will be the average value of all
> LOOP packets seen in the archive interval.
>
> By contrast, the max value in the table is the max value seen in any LOOP
> packet. This can, and most likely will be, higher than these averages.
>
> Not sure what the user expected. WeeWX does not save every LOOP packet, so
> the curve can only represent the values in the archive records.
>
> In any case, for a plot of the day's temperatures, the daily archives
> don't come into play at all. They are only used if the aggregation interval
> is a multiple of a whole day.
>
> Hope that helps.
>
> -tk
>
>
>
>
> On Sun, Sep 15, 2024 at 12:19 AM '[email protected]' via
> weewx-development <[email protected]> wrote:
>
>> It's connected to this issue I am to resolve currently:
>> https://github.com/brewster76/fuzzy-archer/issues/156
>>
>> The plot is showing a number lower/higher than the real max/min from the
>> stats/NOAA data, which obtain their min/max values from the daily
>> summaries, while the plot is only showing the average of the values for
>> every archive_interval. My approach will be the following:
>>
>> - determine every day boundary for the given timespan
>> - get the min/max for every calendar day in that timespan
>> - if the datetime of a single min/max is within the timespan, insert it
>> to the values of the plot (most likely leading to an extra data point
>> between two existing data point, but that's desired behavior)
>> - if the datetime of a single daily min/max is not within the timespan,
>> ignore
>>
>> This way we have the real min/max values, with their exact datetime of
>> arrival, for every day, in the plot, if they are within the given timespan.
>> If they are not in the timespan, we simply don't have a more precise
>> information about the min/max value for an archive_interval in the plot. If
>> they are within the timespan, but not the min/max value of the plot, we
>> know it is a min/max value of that certain calendar day that value lies
>> within. The user probably won't recognize these values, if not zooming in
>> on the chart, but that's perfectly OK.
>>
>>
>> Tom Keffer schrieb am Samstag, 14. September 2024 um 22:06:39 UTC+2:
>>
>>> Yes, it is. In the interests of clarity, I did not give the complete set
>>> of conditions for the daily summaries to be used. The start time must
>>> either be the first record in the database, or a midnight boundary. The
>>> stop time must either be the last record in the database, or a midnight
>>> boundary. Otherwise, the archive database is used.
>>>
>>> So, if the ending timestamp of your first example is the last record in
>>> the database, then the daily summary would have been used.
>>>
>>> I'm glad you're showing an interest in this! It's deep in the guts of
>>> WeeWX, but important.
>>>
>>> -tk
>>>
>>> On Sat, Sep 14, 2024 at 1:01 PM '[email protected]' via
>>> weewx-development <[email protected]> wrote:
>>>
>>>> You are right, none of those examples meets the condition (starting and
>>>> ending on day boundaries), but the first example starts at the start of
>>>> today, ending with the latest timestamp in the database. Which is still
>>>> today, but not a day's boundary. So my best guess is, that the shortcuts
>>>> using the daily summaries, is also taken in that particular case.
>>>>
>>>> And yes, your explanation helps.
>>>> Tom Keffer schrieb am Samstag, 14. September 2024 um 21:27:01 UTC+2:
>>>>
>>>>> I'm not following your question --- too many numbers flying around ---
>>>>> but let me try to explain the calculation in the hopes that it helps.
>>>>>
>>>>> When calculating an aggregate, there's always a well-defined answer.
>>>>> In this case, it's the minimum temperature over the aggregate interval,
>>>>> so
>>>>> you scan all the relevant records and look for the minimum and the
>>>>> timestamp of the record that contains it.
>>>>>
>>>>> However, if the interval starts and ends on day boundaries (that is,
>>>>> at midnight), then a shortcut can be taken. The algorithm can use the
>>>>> daily
>>>>> summaries to speed the calculation. Otherwise, the main archive table is
>>>>> used.
>>>>>
>>>>> I didn't look too closely, but I don't think either of your examples
>>>>> satisfies this condition. The aggregation interval is something like 20
>>>>> hours or 27 hours. So, the main archive table should have been used.
>>>>>
>>>>> Either way, you should get the same answer, although the daily
>>>>> summaries can be slightly more precise for the timestamp. The reason is
>>>>> that while the archive table has the minimum value seen in the interval,
>>>>> it
>>>>> doesn't have the precise time, only the timestamp of the record with the
>>>>> lowest value. By contrast, the daily summaries have the timestamp of the
>>>>> LOOP packet that provided the minimum value, so the precision can be down
>>>>> to a few seconds.
>>>>>
>>>>> Otherwise, there should be no difference.
>>>>>
>>>>> Does that help?
>>>>>
>>>>> -tk
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Sep 14, 2024 at 10:56 AM '[email protected]' via
>>>>> weewx-development <[email protected]> wrote:
>>>>>
>>>>>> Is it possible, that this function returns different values, if the
>>>>>> timespan is across more than a single calendar day?
>>>>>>
>>>>>> results = weewx.xtypes.get_aggregate("outTemp", TimeSpan(1726264800,
>>>>>> 1726335600), "min", dbmanager)
>>>>>> print(results)
>>>>>> results = weewx.xtypes.get_aggregate("outTemp", TimeSpan(
>>>>>> 1726264800, 1726335600), "mintime", dbmanager)
>>>>>> print(results)
>>>>>> yields
>>>>>> (4.5, 'degree_C', 'group_temperature')
>>>>>> (1726303300, 'unix_epoch', 'group_time')
>>>>>> Where 1726264800 is the start of today in my timezone. 1726303300 is
>>>>>> the min outtemp of toady as in archive_day_outtemp.
>>>>>>
>>>>>> results = weewx.xtypes.get_aggregate("outTemp", TimeSpan(1726238400,
>>>>>> 1726335600), "min", dbmanager)
>>>>>> print(results)
>>>>>> results = weewx.xtypes.get_aggregate("outTemp", TimeSpan(
>>>>>> 1726238400, 1726335600), "mintime", dbmanager)
>>>>>> print(results)
>>>>>> yields
>>>>>> (4.555172413793103, 'degree_C', 'group_temperature')
>>>>>> (1726303500, 'unix_epoch', 'group_time')
>>>>>> Where 1726238400 is some random time yesterday in my timezone.
>>>>>> 1726303500 is the first timestamp of the record in archive, where
>>>>>> outtemp
>>>>>> has the minimum value.
>>>>>>
>>>>>> Obviously, when doing the second "query", I don't get the exact
>>>>>> minimum and it's timestamp. Can I only achieve my desired result, if I
>>>>>> limit the timespan to not overlap a calendar day change?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> [email protected] schrieb am Samstag, 14. September 2024 um
>>>>>> 16:15:43 UTC+2:
>>>>>>
>>>>>>> Thanks! Although obviously there, I didn't get it :)
>>>>>>>
>>>>>>> Tom Keffer schrieb am Samstag, 14. September 2024 um 15:35:04 UTC+2:
>>>>>>>
>>>>>>>> Just substitute "mintime" / "maxtime" for "min" / "max".
>>>>>>>>
>>>>>>>> See the table of Aggregation Types
>>>>>>>> <https://www.weewx.com/docs/5.1/reference/aggtypes/>.
>>>>>>>>
>>>>>>>> On Sat, Sep 14, 2024 at 2:02 AM '[email protected]' via
>>>>>>>> weewx-development <[email protected]> wrote:
>>>>>>>>
>>>>>>>>> Thank you Tom, this works for the max/min value as I need it. But
>>>>>>>>> the dateTime for that value isn't in the result and I don't have a
>>>>>>>>> clue,
>>>>>>>>> how to get it for that particular value.
>>>>>>>>>
>>>>>>>>> Tom Keffer schrieb am Freitag, 13. September 2024 um 22:27:31
>>>>>>>>> UTC+2:
>>>>>>>>>
>>>>>>>>>> The xtypes system can do this. See the section, *XTypes API
>>>>>>>>>> <https://github.com/weewx/weewx/wiki/xtypes#xtypes-api>*.
>>>>>>>>>>
>>>>>>>>>> Your example would be:
>>>>>>>>>>
>>>>>>>>>> results = weewx.xtypes.get_aggregate("outTemp",
>>>>>>>>>> TimeSpan(1726092000, 1726178400), "max", dbmanager)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> where "dbmanager" is an open database manager. The results will
>>>>>>>>>> be a ValueTuple
>>>>>>>>>> <https://www.weewx.com/docs/5.1/reference/valuetuple/>.
>>>>>>>>>>
>>>>>>>>>> Here's a more complete example:
>>>>>>>>>>
>>>>>>>>>> import weecfg
>>>>>>>>>> import weewx.manager
>>>>>>>>>> import weewx.xtypes
>>>>>>>>>> from weeutil.weeutil import TimeSpan
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> config_path, config_dict =
>>>>>>>>>> weecfg.read_config('/home/weewx/weewx.conf')
>>>>>>>>>>
>>>>>>>>>> with weewx.manager.open_manager_with_config(config_dict,
>>>>>>>>>> 'wx_binding') as dbmanager:
>>>>>>>>>>
>>>>>>>>>> results = weewx.xtypes.get_aggregate("outTemp",
>>>>>>>>>> TimeSpan(1726092000, 1726178400), "max", dbmanager)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> print(results)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> (68.4, 'degree_F', 'group_temperature')
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Sep 13, 2024 at 10:27 AM '[email protected]' via
>>>>>>>>>> weewx-development <[email protected]> wrote:
>>>>>>>>>>
>>>>>>>>>>> When I write an extension in python, can I get the
>>>>>>>>>>> minimum/maximum value for an obs_type and a given timespan and
>>>>>>>>>>> their
>>>>>>>>>>> related timestamp?
>>>>>>>>>>>
>>>>>>>>>>> Something like:
>>>>>>>>>>>
>>>>>>>>>>> maxValue, datetime = getMax("outTemp", 1726092000, 1726178400)
>>>>>>>>>>>
>>>>>>>>>>> The min/max has to be obtained from the archive_day table, the
>>>>>>>>>>> archive table might not contain the exact max/min value, and also
>>>>>>>>>>> not the
>>>>>>>>>>> exact timestamp the value arrived.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>> Google Groups "weewx-development" group.
>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>>>>>>>>> it, send an email to [email protected].
>>>>>>>>>>> To view this discussion on the web visit
>>>>>>>>>>> https://groups.google.com/d/msgid/weewx-development/6d13cde1-43c4-4ca3-a4a4-d220fc5e2040n%40googlegroups.com
>>>>>>>>>>>
>>>>>>>>>>> <https://groups.google.com/d/msgid/weewx-development/6d13cde1-43c4-4ca3-a4a4-d220fc5e2040n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>>>>>>> .
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>>> Groups "weewx-development" group.
>>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>>> send an email to [email protected].
>>>>>>>>>
>>>>>>>> To view this discussion on the web visit
>>>>>>>>> https://groups.google.com/d/msgid/weewx-development/2aaf12b7-25ad-4013-8185-b379168b4bc5n%40googlegroups.com
>>>>>>>>>
>>>>>>>>> <https://groups.google.com/d/msgid/weewx-development/2aaf12b7-25ad-4013-8185-b379168b4bc5n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>>>>> .
>>>>>>>>>
>>>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "weewx-development" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to [email protected].
>>>>>>
>>>>> To view this discussion on the web visit
>>>>>> https://groups.google.com/d/msgid/weewx-development/c183248b-1a04-47bd-8359-51ec5f215741n%40googlegroups.com
>>>>>>
>>>>>> <https://groups.google.com/d/msgid/weewx-development/c183248b-1a04-47bd-8359-51ec5f215741n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>> .
>>>>>>
>>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "weewx-development" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>>
>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/weewx-development/8138a283-82d2-418b-8a06-d919daee47b0n%40googlegroups.com
>>>>
>>>> <https://groups.google.com/d/msgid/weewx-development/8138a283-82d2-418b-8a06-d919daee47b0n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "weewx-development" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>>
> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/weewx-development/1c0b6e38-6d6f-4ce9-ae27-107f430cc310n%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/weewx-development/1c0b6e38-6d6f-4ce9-ae27-107f430cc310n%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>
--
You received this message because you are subscribed to the Google Groups
"weewx-development" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-development/66060704-cfb5-4a5a-8aaa-1c3db3589ba0n%40googlegroups.com.
