OK. The variable "hauteur_soleil is the elevation of the sun, in degree. >From sunset to sunrise, this value will be negative (i.e. sun below the >horizon). The formula developed by Metro France was validated to be effective only if sun elevation is > 3°.
In my own implantation of this algorithm, I decided to start the calculation of the sun threshold as soon as the sun elevation was > 0° and that the global radiation as measured by the Davis pyranometer was greeter than 20 W/m2. I suspect that the few dateTime that trigger your errors were in a situation in which sun elevation was just below zero and that the measured radiation was just higher than 20 W/m2 > Le 1 juil. 2022 à 16:19, 'Peter Fletcher' via weewx-user > <[email protected]> a écrit : > > Something like that is obviously needed. I don't know how my copy of the code > omitted the test for hauteur_soleil being > 0. There are a number of sources > from which I may have copied it, and I didn't keep links to all of them, but > I can't imagine why I would have deleted the test if my source had included > it. I will obviously add it to my working code. > > On Friday, July 1, 2022 at 2:45:42 AM UTC-4 [email protected] > <http://gmail.com/> wrote: > OK. What I don't understand is that my code had always a test > > if hauteur_soleil > 3: (according to Météo France) > > or more recently > > if hauteur_soleil > 0: > > in the function, and this test is not on your function. > > It should be : > > hauteur_soleil = asin(sin((pi / 180) * latitude) * sin((pi / 180) * > declinaison) + cos( > (pi / 180) * latitude) * cos((pi / 180) * declinaison) * cos((pi / > 180) * angle_horaire)) * (180 / pi) > If hauteur_soleil > 0: > > seuil = (0.73 + 0.06 * cos((pi / 180) * 360 * dayofyear / 365)) * 1080 > * pow( > (sin(pi / 180) * hauteur_soleil), 1.25) * coeff > else : > seuil = 0 > return seuil > > >> Le 30 juin 2022 à 22:52, 'Peter Fletcher' via weewx-user >> <[email protected] >> <applewebdata://F6977839-C71A-44A1-80D2-D947E47C5B5D>> a écrit : >> > >> That was going to be my next step! In fact, iterating through a list of the >> dateTime values that produce the errors in the real code and passing each >> value to the function confirms that it is the specific dateTime values that >> are causing the function to misbehave. The returned results are all complex >> numbers with negative and numerically identical (for a given dateTime) real >> and imaginary components. It does seem to be a bug in the function. I assume >> that hauteur_soleil should always be >=0. It appears that, for my latitude >> and longitude and for the given specific values of dateTime, it becomes >> negative. The last step in the calculation then involves raising a negative >> number to a non-integral power, which is guaranteed to produce interesting >> results! The really odd thing is that math.pow is not returning a >> ValueError, which the docs say is what should happen under these >> circumstances, but apparently trying to return a (possibly) valid complex >> result. >> On Thursday, June 30, 2022 at 3:07:38 PM UTC-4 [email protected] >> <http://gmail.com/> wrote: >> The only clue I have is that the problem is not due to an « overloading » of >> your raspberry pi, but seems to occur with specific dateTime values. >> You can try to run your script only with a « bad » dateTime : >> >> "SELECT dateTime, Radiation from archive where dateTime = 1592614500 » >> >> Does the error occurs ? If yes, you can try to add debugging print commands >> inside the sunshineThreshold function to try to understand. >> >> >> >> >>> Le 30 juin 2022 à 19:51, 'Peter Fletcher' via weewx-user <weewx...@ >>> <>googlegroups.com <http://googlegroups.com/>> a écrit : >>> >> >>> It did as it seems you predicted - passed 1592614800 and stopped at >>> 1632611100. You obviously have a clue as to what is going on. Please >>> explain! >>> >>> On Thursday, June 30, 2022 at 8:59:48 AM UTC-4 [email protected] >>> <http://gmail.com/> wrote: >>> If you exclude the first one,1592614500 , with a query like "SELECT >>> dateTime, Radiation from archive where dateTime <> 1592614500", will the >>> script stop at 1592614800 ( the next dateTime) or will it continue and stop >>> at 1632611100 ? >>> >>> >>>> Le 30 juin 2022 à 14:34, 'Peter Fletcher' via weewx-user <weewx...@ >>>> <>googlegroups.com <http://googlegroups.com/>> a écrit : >>>> >>> >>>> 1592614500 >>>> 1632611100 >>>> 1632611400 >>>> 1647688800 >>>> >>>> I can't see a pattern or any common features. >>>> >>>> On Thursday, June 30, 2022 at 3:55:49 AM UTC-4 [email protected] >>>> <http://gmail.com/> wrote: >>>> No, I never had weewx crashes related to the sunshine calculations. >>>> >>>> What are the dateTime values that trigger the error ? >>>> >>>> >>>> >>>> Le mercredi 29 juin 2022 à 23:23:16 UTC+2, Peter Fletcher a écrit : >>>> Have you had any odd weewx errors or crashes related to the sunshine >>>> calculations? I ask because I hadn't, but I decided to try to 'backfill' >>>> my database with sunshine times, based on the 5-minute radiation values, >>>> and I ran into a bizarre bug. I used the code shown below (on a copy of my >>>> live weewx database). As you will see, the threshold calculation code is >>>> essentially identical to yours, except that it has been converted to a >>>> regular function (no 'self' parameter) and my station's latitude and >>>> longitude are hard coded in it. When the code is run under Python 3.9.2 on >>>> my Pi, it initially runs without problems, but crashes after 8,000+ >>>> records have been processed with a ValueError on the MaxThreshold vs >>>> threshold comparison, reporting that it can't compare a complex with a >>>> float! If I intercept and log the errors, it turns out that, for a few >>>> specific values of dateTime, the function returns a complex number! Even >>>> more bizarrely, it only seems to do that in the context of the running >>>> code. If I manually run through all the operations from the function code >>>> at the Python command line, using the value of dateTime that produces the >>>> first crash, all the intermediate results and the final result are sane >>>> floats. >>>> There appears to be a second issue, possibly related to my reading and >>>> writing the database at relatively high frequency, which stalls the >>>> process after about 18,000 records have been processed, but removing the >>>> database writes allows it to run to completion without abolishing the >>>> consistent, albeit infrequent, ValueErrors. >>>> >>>> [backfill.py] >>>> import sqlite3 >>>> from datetime import datetime >>>> import time >>>> from math import sin, cos, pi, asin >>>> >>>> def sunshineThreshold(mydatetime): >>>> coeff = 0.9 # change to calibrate with your sensor >>>> utcdate = datetime.utcfromtimestamp(mydatetime) >>>> dayofyear = int(time.strftime("%j", time.gmtime(mydatetime))) >>>> theta = 360 * dayofyear / 365 >>>> equatemps = 0.0172 + 0.4281 * cos((pi / 180) * theta) - 7.3515 * sin( >>>> (pi / 180) * theta) - 3.3495 * cos(2 * (pi / 180) * theta) - >>>> 9.3619 * sin( >>>> 2 * (pi / 180) * theta) >>>> >>>> latitude = 43.0346213 >>>> longitude = -78.689362 >>>> >>>> corrtemps = longitude * 4 >>>> declinaison = asin(0.006918 - 0.399912 * cos((pi / 180) * theta) + >>>> 0.070257 * sin( >>>> (pi / 180) * theta) - 0.006758 * cos(2 * (pi / 180) * theta) + >>>> 0.000908 * sin( >>>> 2 * (pi / 180) * theta)) * (180 / pi) >>>> minutesjour = utcdate.hour * 60 + utcdate.minute >>>> tempsolaire = (minutesjour + corrtemps + equatemps) / 60 >>>> angle_horaire = (tempsolaire - 12) * 15 >>>> hauteur_soleil = asin(sin((pi / 180) * latitude) * sin((pi / 180) * >>>> declinaison) + cos( >>>> (pi / 180) * latitude) * cos((pi / 180) * declinaison) * cos((pi / >>>> 180) * angle_horaire)) * (180 / pi) >>>> seuil = (0.73 + 0.06 * cos((pi / 180) * 360 * dayofyear / 365)) * 1080 >>>> * pow( >>>> (sin(pi / 180) * hauteur_soleil), 1.25) * coeff >>>> return seuil >>>> >>>> >>>> database = 'weewx.sdb' >>>> >>>> maxThreshold=0 >>>> count=0 >>>> conn=sqlite3.connect(database) >>>> cur=conn.execute("SELECT dateTime, Radiation from archive") >>>> for row in cur: >>>> count += 1 >>>> if (row[1] is not None) and (row[1] > 20): >>>> threshold = sunshineThreshold(row[0]) >>>> if threshold > maxThreshold: >>>> maxThreshold = threshold >>>> if row[1] > threshold: >>>> conn.execute("UPDATE archive set SunshineTime = 5 WHERE dateTime = >>>> " + str(row[0])) >>>> if count % 1000 == 0: >>>> print(count, 'Max Threshold', maxThreshold) >>>> conn.close >>>> [/backfill.py] >>>> >>>> On Friday, June 10, 2022 at 3:29:40 AM UTC-4 jterr...@ <>gmail.com >>>> <http://gmail.com/> wrote: >>>> On my side, I have looked at the CPU utilization on my raspberry Pi 3B+. I >>>> have the mqtt service service installed, so at each loop all data of the >>>> packet are sent to the mqtt broker. >>>> >>>> With mqtt and when calculations of the sunshine threshold is done for each >>>> loop packet, the total CPU utilization of python3 is about 0.75% >>>> With mqtt and without calculation of sunshine threshold : 0.5% of total >>>> CPU. >>>> >>>> So one can estimate that 0.25 % of total CPU is needed for the calculation >>>> of the threshold value for each LOOP packet. >>>> >>>> >>>> >>>>> Le 9 juin 2022 à 22:26, 'Peter Fletcher' via weewx-user <weewx...@ >>>>> <>googlegroups.com <http://googlegroups.com/>> a écrit : >>>>> >>>> >>>>> After some experimentation, I found that the radiation value in the VP2 >>>>> LOOP packets does, indeed, normally change every 50-52 seconds, but, >>>>> perhaps about a fifth of the 'gaps' are a multiple of that time - most >>>>> often 100+ or 150+ seconds, but occasionally more than that (I saw one >>>>> 250+ second 'gap'). I saw this under conditions of variable sunshine and >>>>> clouds when it seemed unlikely that the actual radiation value would have >>>>> been precisely constant for that length of time, so I am not sure exactly >>>>> what is going on. In any event, I am revising the code I am using on the >>>>> basis of doing the threshold calculation when the radiation level >>>>> changes, but at least every minute, if it remains constant for more than >>>>> the normal 50-52 seconds.. >>>>> >>>>> On Sunday, June 5, 2022 at 12:33:47 PM UTC-4 [email protected] >>>>> <http://gmail.com/> wrote: >>>>> I think it is also OK to do an average for every 30 seconds. It depends >>>>> also on the weather station used. >>>>> For instance, a Davis Vantage Pro 2 ISS transmits an updated solar >>>>> radiation value every 50 to 60 seconds. So with this weather station, >>>>> even a 1 minute average would not be very different since anyway the >>>>> solar radiation values of the LOOP packet are the same for at least 50 >>>>> seconds.! >>>>> >>>>> >>>>>> Le 5 juin 2022 à 18:02, 'Peter Fletcher' via weewx-user <weewx...@ >>>>>> <>googlegroups.com <http://googlegroups.com/>> a écrit : >>>>>> >>>>> >>>>>> I chose to average the LOOP radiation readings and only to do the >>>>>> threshold calculation and make the sun/no sun determination every 30 >>>>>> seconds because I thought doing it on every LOOP might overload LOOP >>>>>> processing (I am running weewx on a Pi 3B, which is also doing a few >>>>>> other things which use the CPU). If this is an unnecessary concern, as >>>>>> it may very well be, your modified code is much cleaner than mine. >>>>>> >>>>>> On Saturday, June 4, 2022 at 12:41:08 PM UTC-4 [email protected] >>>>>> <http://gmail.com/> wrote: >>>>>> It is a very good idea to calculate the sunshine duration for each LOOP >>>>>> packet and sum these values to make the final archive sunshine duration. >>>>>> I have modified my script accordingly : >>>>>> https://github.com/Jterrettaz/sunduration >>>>>> <https://github.com/Jterrettaz/sunduration>. >>>>>> The logic is the following : for each received LOOP packet, the >>>>>> radiation is compared to a calculated threshold. If the radiation is >>>>>> above the threshold value, the sunshine time for the LOOP packet is >>>>>> equal to the time elapsed between the previous loop packet and this >>>>>> packet (most of the time 2 seconds with a Vantage Davis Pro). >>>>>> The final archive sunshine duration is the sum of all the LOOP value >>>>>> within the archive period. >>>>>> Le vendredi 3 juin 2022 à 21:59:36 UTC+2, Peter Fletcher a écrit : >>>>>> That makes some sense when you are getting data from an 'external' >>>>>> sensor, though there are (IMHO) simpler ways of doing it. weewx already >>>>>> has access to the LOOP radiation data from the VP2, so handling the >>>>>> processing and data storage within weewx makes more sense to me in this >>>>>> case. >>>>>> >>>>>> On Friday, June 3, 2022 at 3:24:23 PM UTC-4 vince wrote: >>>>>> On Friday, June 3, 2022 at 11:17:00 AM UTC-7 Meteo Oberwallis wrote: >>>>>> if the interval of Weewx and the data logger is set to 10 minutes, I >>>>>> would have liked to read the value of the solar sensor every minute and >>>>>> then write it into a separate .sdb database as possible sunshine. >>>>>> >>>>>> Personally I'd use an external program called via cron and posting a >>>>>> message to a MQTT topic. Have weewx subscribe to that topic to get the >>>>>> data into your db. >>>>>> >>>>>> This is how I used to get my DS18b20 temperature sensor data into weewx. >>>>>> >>>>>> >>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to a topic in the >>>>>> Google Groups "weewx-user" group. >>>>>> To unsubscribe from this topic, visit >>>>>> https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe >>>>>> <https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe>. >>>>>> To unsubscribe from this group and all its topics, send an email to >>>>>> weewx-user+...@ <>googlegroups.com <http://googlegroups.com/>. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/weewx-user/0e631671-0a74-4963-9f1c-e5f81bc7c366n%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/weewx-user/0e631671-0a74-4963-9f1c-e5f81bc7c366n%40googlegroups.com?utm_medium=email&utm_source=footer>. >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to a topic in the >>>>> Google Groups "weewx-user" group. >>>>> To unsubscribe from this topic, visit >>>>> https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe >>>>> <https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe>. >>>>> To unsubscribe from this group and all its topics, send an email to >>>>> weewx-user+...@ <>googlegroups.com <http://googlegroups.com/>. >>>> >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/weewx-user/f0ecc86f-a615-4a24-a43f-ee0d3963b8adn%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/weewx-user/f0ecc86f-a615-4a24-a43f-ee0d3963b8adn%40googlegroups.com?utm_medium=email&utm_source=footer>. >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "weewx-user" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe >>>> <https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe>. >>>> To unsubscribe from this group and all its topics, send an email to >>>> weewx-user+...@ <>googlegroups.com <http://googlegroups.com/>. >>> >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/weewx-user/39cf6daa-80ca-4ffb-89d3-0f00b971481an%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/weewx-user/39cf6daa-80ca-4ffb-89d3-0f00b971481an%40googlegroups.com?utm_medium=email&utm_source=footer>. >>> >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "weewx-user" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe >>> <https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe>. >>> To unsubscribe from this group and all its topics, send an email to >>> weewx-user+...@ <>googlegroups.com <http://googlegroups.com/>. >> >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/c07ed2bb-1e3d-43e2-b08c-08a7a3aa92dbn%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/weewx-user/c07ed2bb-1e3d-43e2-b08c-08a7a3aa92dbn%40googlegroups.com?utm_medium=email&utm_source=footer>. >> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "weewx-user" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe >> <https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe>. >> To unsubscribe from this group and all its topics, send an email to >> [email protected] >> <applewebdata://F6977839-C71A-44A1-80D2-D947E47C5B5D>. > >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/weewx-user/f287c1b3-1005-409c-82a9-a072e375d5e9n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/weewx-user/f287c1b3-1005-409c-82a9-a072e375d5e9n%40googlegroups.com?utm_medium=email&utm_source=footer>. > > > -- > You received this message because you are subscribed to a topic in the Google > Groups "weewx-user" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe > <https://groups.google.com/d/topic/weewx-user/19ylVTRqbh4/unsubscribe>. > To unsubscribe from this group and all its topics, send an email to > [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/77cd4de6-3b1c-4bc2-9dbc-8d8316bd8a9bn%40googlegroups.com > > <https://groups.google.com/d/msgid/weewx-user/77cd4de6-3b1c-4bc2-9dbc-8d8316bd8a9bn%40googlegroups.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/75000179-CCB4-42D0-9793-D5749D8D9C76%40gmail.com.
