OK. What I don't understand is that my code had always a test
if hauteur_soleil > 3: (according to Météo France)
or more recently
if hauteur_soleil > 0:
in the function, and this test is not on your function.
It should be :
hauteur_soleil = asin(sin((pi / 180) * latitude) * sin((pi / 180) *
declinaison) + cos(
(pi / 180) * latitude) * cos((pi / 180) * declinaison) * cos((pi / 180)
* angle_horaire)) * (180 / pi)
If hauteur_soleil > 0:
seuil = (0.73 + 0.06 * cos((pi / 180) * 360 * dayofyear / 365)) * 1080
* pow(
(sin(pi / 180) * hauteur_soleil), 1.25) * coeff
else :
seuil = 0
return seuil
> Le 30 juin 2022 à 22:52, 'Peter Fletcher' via weewx-user
> <[email protected]> a écrit :
>
> That was going to be my next step! In fact, iterating through a list of the
> dateTime values that produce the errors in the real code and passing each
> value to the function confirms that it is the specific dateTime values that
> are causing the function to misbehave. The returned results are all complex
> numbers with negative and numerically identical (for a given dateTime) real
> and imaginary components. It does seem to be a bug in the function. I assume
> that hauteur_soleil should always be >=0. It appears that, for my latitude
> and longitude and for the given specific values of dateTime, it becomes
> negative. The last step in the calculation then involves raising a negative
> number to a non-integral power, which is guaranteed to produce interesting
> results! The really odd thing is that math.pow is not returning a ValueError,
> which the docs say is what should happen under these circumstances, but
> apparently trying to return a (possibly) valid complex result.
> On Thursday, June 30, 2022 at 3:07:38 PM UTC-4 [email protected] wrote:
> The only clue I have is that the problem is not due to an « overloading » of
> your raspberry pi, but seems to occur with specific dateTime values.
> You can try to run your script only with a « bad » dateTime :
>
> "SELECT dateTime, Radiation from archive where dateTime = 1592614500 »
>
> Does the error occurs ? If yes, you can try to add debugging print commands
> inside the sunshineThreshold function to try to understand.
>
>
>
>
>> Le 30 juin 2022 à 19:51, 'Peter Fletcher' via weewx-user
>> <[email protected]
>> <applewebdata://53FD5BF2-209C-4577-B6BF-532766329623>> a écrit :
>>
>
>> It did as it seems you predicted - passed 1592614800 and stopped at
>> 1632611100. You obviously have a clue as to what is going on. Please explain!
>>
>> On Thursday, June 30, 2022 at 8:59:48 AM UTC-4 [email protected]
>> <http://gmail.com/> wrote:
>> If you exclude the first one,1592614500 , with a query like "SELECT
>> dateTime, Radiation from archive where dateTime <> 1592614500", will the
>> script stop at 1592614800 ( the next dateTime) or will it continue and stop
>> at 1632611100 ?
>>
>>
>>> Le 30 juin 2022 à 14:34, 'Peter Fletcher' via weewx-user
>>> <[email protected] <>> a écrit :
>>>
>>
>>> 1592614500
>>> 1632611100
>>> 1632611400
>>> 1647688800
>>>
>>> I can't see a pattern or any common features.
>>>
>>> On Thursday, June 30, 2022 at 3:55:49 AM UTC-4 [email protected]
>>> <http://gmail.com/> wrote:
>>> No, I never had weewx crashes related to the sunshine calculations.
>>>
>>> What are the dateTime values that trigger the error ?
>>>
>>>
>>>
>>> Le mercredi 29 juin 2022 à 23:23:16 UTC+2, Peter Fletcher a écrit :
>>> Have you had any odd weewx errors or crashes related to the sunshine
>>> calculations? I ask because I hadn't, but I decided to try to 'backfill' my
>>> database with sunshine times, based on the 5-minute radiation values, and I
>>> ran into a bizarre bug. I used the code shown below (on a copy of my live
>>> weewx database). As you will see, the threshold calculation code is
>>> essentially identical to yours, except that it has been converted to a
>>> regular function (no 'self' parameter) and my station's latitude and
>>> longitude are hard coded in it. When the code is run under Python 3.9.2 on
>>> my Pi, it initially runs without problems, but crashes after 8,000+ records
>>> have been processed with a ValueError on the MaxThreshold vs threshold
>>> comparison, reporting that it can't compare a complex with a float! If I
>>> intercept and log the errors, it turns out that, for a few specific values
>>> of dateTime, the function returns a complex number! Even more bizarrely, it
>>> only seems to do that in the context of the running code. If I manually run
>>> through all the operations from the function code at the Python command
>>> line, using the value of dateTime that produces the first crash, all the
>>> intermediate results and the final result are sane floats.
>>> There appears to be a second issue, possibly related to my reading and
>>> writing the database at relatively high frequency, which stalls the process
>>> after about 18,000 records have been processed, but removing the database
>>> writes allows it to run to completion without abolishing the consistent,
>>> albeit infrequent, ValueErrors.
>>>
>>> [backfill.py]
>>> import sqlite3
>>> from datetime import datetime
>>> import time
>>> from math import sin, cos, pi, asin
>>>
>>> def sunshineThreshold(mydatetime):
>>> coeff = 0.9 # change to calibrate with your sensor
>>> utcdate = datetime.utcfromtimestamp(mydatetime)
>>> dayofyear = int(time.strftime("%j", time.gmtime(mydatetime)))
>>> theta = 360 * dayofyear / 365
>>> equatemps = 0.0172 + 0.4281 * cos((pi / 180) * theta) - 7.3515 * sin(
>>> (pi / 180) * theta) - 3.3495 * cos(2 * (pi / 180) * theta) - 9.3619
>>> * sin(
>>> 2 * (pi / 180) * theta)
>>>
>>> latitude = 43.0346213
>>> longitude = -78.689362
>>>
>>> corrtemps = longitude * 4
>>> declinaison = asin(0.006918 - 0.399912 * cos((pi / 180) * theta) +
>>> 0.070257 * sin(
>>> (pi / 180) * theta) - 0.006758 * cos(2 * (pi / 180) * theta) +
>>> 0.000908 * sin(
>>> 2 * (pi / 180) * theta)) * (180 / pi)
>>> minutesjour = utcdate.hour * 60 + utcdate.minute
>>> tempsolaire = (minutesjour + corrtemps + equatemps) / 60
>>> angle_horaire = (tempsolaire - 12) * 15
>>> hauteur_soleil = asin(sin((pi / 180) * latitude) * sin((pi / 180) *
>>> declinaison) + cos(
>>> (pi / 180) * latitude) * cos((pi / 180) * declinaison) * cos((pi /
>>> 180) * angle_horaire)) * (180 / pi)
>>> seuil = (0.73 + 0.06 * cos((pi / 180) * 360 * dayofyear / 365)) * 1080
>>> * pow(
>>> (sin(pi / 180) * hauteur_soleil), 1.25) * coeff
>>> return seuil
>>>
>>>
>>> database = 'weewx.sdb'
>>>
>>> maxThreshold=0
>>> count=0
>>> conn=sqlite3.connect(database)
>>> cur=conn.execute("SELECT dateTime, Radiation from archive")
>>> for row in cur:
>>> count += 1
>>> if (row[1] is not None) and (row[1] > 20):
>>> threshold = sunshineThreshold(row[0])
>>> if threshold > maxThreshold:
>>> maxThreshold = threshold
>>> if row[1] > threshold:
>>> conn.execute("UPDATE archive set SunshineTime = 5 WHERE dateTime =
>>> " + str(row[0]))
>>> if count % 1000 == 0:
>>> print(count, 'Max Threshold', maxThreshold)
>>> conn.close
>>> [/backfill.py]
>>>
>>> On Friday, June 10, 2022 at 3:29:40 AM UTC-4 [email protected] <> wrote:
>>> On my side, I have looked at the CPU utilization on my raspberry Pi 3B+. I
>>> have the mqtt service service installed, so at each loop all data of the
>>> packet are sent to the mqtt broker.
>>>
>>> With mqtt and when calculations of the sunshine threshold is done for each
>>> loop packet, the total CPU utilization of python3 is about 0.75%
>>> With mqtt and without calculation of sunshine threshold : 0.5% of total CPU.
>>>
>>> So one can estimate that 0.25 % of total CPU is needed for the calculation
>>> of the threshold value for each LOOP packet.
>>>
>>>
>>>
>>>> Le 9 juin 2022 à 22:26, 'Peter Fletcher' via weewx-user
>>>> <[email protected] <>> a écrit :
>>>>
>>>
>>>> After some experimentation, I found that the radiation value in the VP2
>>>> LOOP packets does, indeed, normally change every 50-52 seconds, but,
>>>> perhaps about a fifth of the 'gaps' are a multiple of that time - most
>>>> often 100+ or 150+ seconds, but occasionally more than that (I saw one
>>>> 250+ second 'gap'). I saw this under conditions of variable sunshine and
>>>> clouds when it seemed unlikely that the actual radiation value would have
>>>> been precisely constant for that length of time, so I am not sure exactly
>>>> what is going on. In any event, I am revising the code I am using on the
>>>> basis of doing the threshold calculation when the radiation level changes,
>>>> but at least every minute, if it remains constant for more than the normal
>>>> 50-52 seconds..
>>>>
>>>> On Sunday, June 5, 2022 at 12:33:47 PM UTC-4 [email protected]
>>>> <http://gmail.com/> wrote:
>>>> I think it is also OK to do an average for every 30 seconds. It depends
>>>> also on the weather station used.
>>>> For instance, a Davis Vantage Pro 2 ISS transmits an updated solar
>>>> radiation value every 50 to 60 seconds. So with this weather station, even
>>>> a 1 minute average would not be very different since anyway the solar
>>>> radiation values of the LOOP packet are the same for at least 50 seconds.!
>>>>
>>>>
>>>>> Le 5 juin 2022 à 18:02, 'Peter Fletcher' via weewx-user
>>>>> <[email protected] <>> a écrit :
>>>>>
>>>>
>>>>> I chose to average the LOOP radiation readings and only to do the
>>>>> threshold calculation and make the sun/no sun determination every 30
>>>>> seconds because I thought doing it on every LOOP might overload LOOP
>>>>> processing (I am running weewx on a Pi 3B, which is also doing a few
>>>>> other things which use the CPU). If this is an unnecessary concern, as it
>>>>> may very well be, your modified code is much cleaner than mine.
>>>>>
>>>>> On Saturday, June 4, 2022 at 12:41:08 PM UTC-4 [email protected]
>>>>> <http://gmail.com/> wrote:
>>>>> It is a very good idea to calculate the sunshine duration for each LOOP
>>>>> packet and sum these values to make the final archive sunshine duration.
>>>>> I have modified my script accordingly :
>>>>> https://github.com/Jterrettaz/sunduration
>>>>> <https://github.com/Jterrettaz/sunduration>.
>>>>> The logic is the following : for each received LOOP packet, the
>>>>> radiation is compared to a calculated threshold. If the radiation is
>>>>> above the threshold value, the sunshine time for the LOOP packet is equal
>>>>> to the time elapsed between the previous loop packet and this packet
>>>>> (most of the time 2 seconds with a Vantage Davis Pro).
>>>>> The final archive sunshine duration is the sum of all the LOOP value
>>>>> within the archive period.
>>>>> Le vendredi 3 juin 2022 à 21:59:36 UTC+2, Peter Fletcher a écrit :
>>>>> That makes some sense when you are getting data from an 'external'
>>>>> sensor, though there are (IMHO) simpler ways of doing it. weewx already
>>>>> has access to the LOOP radiation data from the VP2, so handling the
>>>>> processing and data storage within weewx makes more sense to me in this
>>>>> case.
>>>>>
>>>>> On Friday, June 3, 2022 at 3:24:23 PM UTC-4 vince wrote:
>>>>> On Friday, June 3, 2022 at 11:17:00 AM UTC-7 Meteo Oberwallis wrote:
>>>>> if the interval of Weewx and the data logger is set to 10 minutes, I
>>>>> would have liked to read the value of the solar sensor every minute and
>>>>> then write it into a separate .sdb database as possible sunshine.
>>>>>
>>>>> Personally I'd use an external program called via cron and posting a
>>>>> message to a MQTT topic. Have weewx subscribe to that topic to get the
>>>>> data into your db.
>>>>>
>>>>> This is how I used to get my DS18b20 temperature sensor data into weewx.
>>>>>
>>>>>
>>>>
>>>>> --
>>>>> You received this message because you are subscribed to a topic in the
>>>>> Google Groups "weewx-user" group.
>>>>> To unsubscribe from this topic, visit
>>>>> https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe
>>>>> <https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe>.
>>>>> To unsubscribe from this group and all its topics, send an email to
>>>>> [email protected] <>.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/weewx-user/0e631671-0a74-4963-9f1c-e5f81bc7c366n%40googlegroups.com
>>>>>
>>>>> <https://groups.google.com/d/msgid/weewx-user/0e631671-0a74-4963-9f1c-e5f81bc7c366n%40googlegroups.com?utm_medium=email&utm_source=footer>.
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to a topic in the
>>>> Google Groups "weewx-user" group.
>>>> To unsubscribe from this topic, visit
>>>> https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe
>>>> <https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe>.
>>>> To unsubscribe from this group and all its topics, send an email to
>>>> [email protected] <>.
>>>
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/weewx-user/f0ecc86f-a615-4a24-a43f-ee0d3963b8adn%40googlegroups.com
>>>>
>>>> <https://groups.google.com/d/msgid/weewx-user/f0ecc86f-a615-4a24-a43f-ee0d3963b8adn%40googlegroups.com?utm_medium=email&utm_source=footer>.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "weewx-user" group.
>>> To unsubscribe from this topic, visit
>>> https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe
>>> <https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe>.
>>> To unsubscribe from this group and all its topics, send an email to
>>> [email protected] <>.
>>
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/weewx-user/39cf6daa-80ca-4ffb-89d3-0f00b971481an%40googlegroups.com
>>>
>>> <https://groups.google.com/d/msgid/weewx-user/39cf6daa-80ca-4ffb-89d3-0f00b971481an%40googlegroups.com?utm_medium=email&utm_source=footer>.
>>
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "weewx-user" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe
>> <https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe>.
>> To unsubscribe from this group and all its topics, send an email to
>> [email protected]
>> <applewebdata://53FD5BF2-209C-4577-B6BF-532766329623>.
>
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/weewx-user/c07ed2bb-1e3d-43e2-b08c-08a7a3aa92dbn%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/weewx-user/c07ed2bb-1e3d-43e2-b08c-08a7a3aa92dbn%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
>
> --
> You received this message because you are subscribed to a topic in the Google
> Groups "weewx-user" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe
> <https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe>.
> To unsubscribe from this group and all its topics, send an email to
> [email protected]
> <mailto:[email protected]>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/f287c1b3-1005-409c-82a9-a072e375d5e9n%40googlegroups.com
>
> <https://groups.google.com/d/msgid/weewx-user/f287c1b3-1005-409c-82a9-a072e375d5e9n%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/B75027C1-53BA-40C8-B6C5-C943A0B5368F%40gmail.com.
