the RankWarning is just a warning, you should still have image files produced. I expect the polyfit it just ignores higher order terms and just returns a quadratic.
That is because the polynomial is a poor representation of the background, and the spline fit works better, if you get the parameters correct. Initially you are better just to view the results without trend removal. If the peaks do not stand out then background removal is unlikely to help much. Also, with my code, make sure you start with oversampling set to 1. Only adjust it if you see a stair-step effect. On Friday, 21 January 2022 at 1:09:28 pm UTC+10 [email protected] wrote: > Modified the query and added the print statement as suggested by Cameron D > and here are the results: > > *Morrowwn Script:* > > raspberrypi:~/Desktop/Tonga $ python3 tonga_barometer.py > distance to eruption 13293.9 km arrival at 1642261629 (2022-01-15 10:47:08) > select datetime, barometer from archive where datetime > 1642172400 and > datetime < 1642693629 order by dateTime; > Traceback (most recent call last): > File "tonga_barometer.py", line 47, in <module> > coeff = np.polyfit(xdata, ydata, 2) > File "/usr/lib/python3/dist-packages/numpy/lib/polynomial.py", line 590, > in polyfit > y = NX.asarray(y) + 0.0 > TypeError: unsupported operand type(s) for +: 'NoneType' and 'float' > > > *Cameron D Script*: > > raspberrypi:~/Desktop/Tonga $ python3 tonga_baro.py > distance to eruption 12255.7 km arrival at 1642258384 (2022-01-15 09:53:04) > opposite pulse arrival at 1642306911 (2022-01-15 23:21:50) > second time around pulse arrival at 1642383509 (2022-01-16 20:38:29) > query returned 96 data points > query returned 96 data points > query returned 95 data points > tonga_baro.py:108: RankWarning: Polyfit may be poorly conditioned > coeff = np.polyfit(xdata, ydata, 5 ) > > Note: LAT/Long and speed of sound are identical in both scripts. > On Thursday, January 20, 2022 at 9:27:13 PM UTC-5 Cameron D wrote: > >> and add a debug printout for how many result lines there are after ... >> result = cursor.fetchall() >> add the line: >> >> *print( "query returned {} data points".format(len(result)))* >> >> On Friday, 21 January 2022 at 12:21:59 pm UTC+10 Cameron D wrote: >> >>> To eliminate NULL data points use a query like: >>> select datetime, barometer from archive where datetime > 1642172400 and >>> datetime < 1642693629* and barometer is not null *order by dateTime; >>> You could add the part in bold into the query in the python script. >>> >>> Of course, if they are all null... >>> >>> On Friday, 21 January 2022 at 12:14:44 pm UTC+10 [email protected] >>> wrote: >>> >>>> Download the latest script from >>>> https://github.com/morrowwm/weewx_tonga_browse. Beside installing the >>>> Python Modules Vince stated above, I needed to also install >>>> python3-scipy. I do have data between 1642172400 and 1642693629; >>>> except for a couple "null" in that time period. So when I run the >>>> script, >>>> I get the following error. >>>> >>>> raspberrypi:~/Desktop/Tonga $ python3 tonga_barometer.py >>>> distance to eruption 13293.9 km arrival at 1642261629 (2022-01-15 >>>> 10:47:08) >>>> select datetime, barometer from archive where datetime > 1642172400 and >>>> datetime < 1642693629 order by dateTime; >>>> Traceback (most recent call last): >>>> File "tonga_barometer.py", line 47, in <module> >>>> coeff = np.polyfit(xdata, ydata, 2) >>>> File "/usr/lib/python3/dist-packages/numpy/lib/polynomial.py", line >>>> 590, in polyfit >>>> y = NX.asarray(y) + 0.0 >>>> TypeError: unsupported operand type(s) for +: 'NoneType' and 'float' >>>> >>>> There should be a way to check for "null" data within the time period. >>>> On Thursday, January 20, 2022 at 8:12:55 PM UTC-5 Cameron D wrote: >>>> >>>>> yes, definitely looks like there is no data. >>>>> I have attached another version of mine, in which the trend line is >>>>> disabled by default, but I suspect that would just delay the inevitable >>>>> and >>>>> it would crash trying to do the plot. >>>>> >>>>> I also fixed up a few plotting errors in my code to do with the >>>>> mysteries (to me) of layer ordering. >>>>> I also had a background bar showing either side of expected arrival - >>>>> in this version I have now changed that to start at the expected arrival >>>>> and stop 1 hour later. >>>>> >>>>> On Friday, 21 January 2022 at 3:29:36 am UTC+10 [email protected] wrote: >>>>> >>>>>> Hello, >>>>>> Not being a programmer, I probably shouldn't have messed with this, >>>>>> but being curious... >>>>>> >>>>>> I tried the code posted on github as well as the one by Cameron D. In >>>>>> both cases I got the following error: >>>>>> ``` >>>>>> root@n4mrv:/home/bg/weewx_tonga_browse-main# python3 ./tonga.py [file >>>>>> from Cameron D] >>>>>> >>>>>> distance to eruption 12056.6 km arrival at 1642258360 (2022-01-15 >>>>>> 09:52:39) >>>>>> opposite pulse arrival at 1642308921 (2022-01-15 23:55:21) >>>>>> second time around pulse arrival at 1642385471 (2022-01-16 >>>>>> 21:11:10) >>>>>> Traceback (most recent call last): >>>>>> File "./tonga.py", line 178, in <module> >>>>>> plot_burst( cursor, arrival_time, hour_span, "primary" ) >>>>>> File "./tonga.py", line 54, in plot_burst >>>>>> coeff = np.polyfit(xdata, ydata, background_order ) >>>>>> File "<__array_function__ internals>", line 180, in polyfit >>>>>> File >>>>>> "/usr/local/lib/python3.8/dist-packages/numpy/lib/polynomial.py", line >>>>>> 638, >>>>>> in polyfit >>>>>> raise TypeError("expected non-empty vector for x") >>>>>> TypeError: expected non-empty vector for x >>>>>> ``` >>>>>> I added my lat/lon information but may have missed something else I >>>>>> need to change. Python modules were installed as directed. Copy of >>>>>> weewx.sdb is in the same directory as the program. >>>>>> Thanks. >>>>>> Bob >>>>>> On Wednesday, January 19, 2022 at 10:32:49 AM UTC-5 [email protected] >>>>>> wrote: >>>>>> >>>>>>> On Wednesday, January 19, 2022 at 12:42:04 a.m. UTC-4 Cameron D >>>>>>> wrote: >>>>>>> >>>>>>>> >>>>>>>> - as you get closer to the equator, tidal changes dominate the >>>>>>>> baseline in that timescale - I tried higher order polynomials, but >>>>>>>> they are >>>>>>>> next to useless. >>>>>>>> >>>>>>>> I also had little luck with higher order polynomials to remove the >>>>>>> general trend. >>>>>>> >>>>>>> I've put the script here: >>>>>>> https://github.com/morrowwm/weewx_tonga_browse >>>>>>> >>>>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/f5346eac-ed55-4719-9e35-efe9a7595f12n%40googlegroups.com.
