Tim, thanks so much for sharing your "arrows in the back" experience.
As I am still in development of my analytics product, just having ability to easily grab data across my LAN as I am testing will suffice, so having the local MariaDB as the new WeeWx store is the first step. After that we are heading the same direction, i.e., towards using the local generated data in a web based analytics product. I am intrigued to consider MQTT for the ongoing push of data to the cloud, after doing bulk copy of local SQL data to the cloud / analytics database. WeeWx has a MQTT extension so trying it should be pretty easy. I will chime back in as I find my way on this. Stephen On Wednesday, February 10, 2021 at 2:50:53 PM UTC-8 tim lambert wrote: > Hi Stephen, > > I started out with weeWX in late December using SQLite, then transitioned > to MySQL on a remotely hosted server -- simply backed-up the weewx.conf and > made the necessary changes to point to MySQL. Data migration from SQLite > to MySQL was seamless. > > However, I began to notice significant delays with archiving data and > reporting data -- thanks to the feedback from this forum, I concluded the > problem was the remote MySQL server. I validated the conclusion using a > clone of my Production install and establishing a MySQL DB on an SSD > connected to the Dev Pi 4. Now that I was confident I had a resolution to > the performance issue, I installed a 128GB thumb-drive on the Production Pi > and setup MySQL to use the thumb-drive, then modified the WeeWX to point to > the local MySQL DB. Once I confirmed weeWX was functioning without issue, > and inserting data into the local MySQL DB, I simply did an export from the > remote MySQL DB and imported the data into the local MySQL DB. > > I'm actively working an approach to get the local MySQL data to the remote > (public accessible) server to allow for some other analytics/data > aggregation -- simply trying to avoid routing traffic through the router to > hit the local DB directly. > > Good Luck! > > -- Tim > > On Wednesday, February 10, 2021 at 12:23:20 PM UTC-8 Stephen Rogers wrote: > >> >> Thanks guys. Seems straightforward, and I will post results of my >> experience doing this! >> Stephen >> On Wednesday, February 10, 2021 at 12:06:33 PM UTC-8 gjr80 wrote: >> >>> Hi, >>> >>> To move from using SQLite to MariaDB you might want to have a read of >>> the Transfer from sqlite to MySQL wiki entry >>> <https://github.com/weewx/weewx/wiki/Transfer%20from%20sqlite%20to%20MySQL>. >>> >>> As far as I know the approaches covered there are still valid. >>> >>> Gary >>> On Thursday, 11 February 2021 at 04:56:12 UTC+10 Stephen Rogers wrote: >>> >>>> Hi there, my first post as groups member. I am a software developer >>>> working primarily in Javascript/Vue/Express/MariaDB stack. >>>> >>>> I have been running WeeWx for almost 2 years, on Raspberry Pi >>>> 3/Raspbian Linux, using SQLite for data storage. >>>> >>>> I now want to be able to routinely pull specific dimensions of data >>>> (starting with temperature) from the WeeWx database and integrate that >>>> data >>>> into my own analytics program. SQLite, as far as I know, cannot be >>>> remotely accessed so I will need a different database that can. >>>> >>>> I'd like any advice you can provide on the most expedient way to >>>> accomplish my goal. Example idea: >>>> >>>> 1. install MariaDB on the WeeWx Pi >>>> 2. reconfigure WeeWx to use MariaDB using essentially the same >>>> steps as a fresh install (should I do a fresh WeeWxinstall?) >>>> 3. Pull data from SQLite and migrate it to MariaDB (best utility >>>> for doing this?) so I have all the almost 2 years of collected data >>>> available. >>>> >>>> I could also dump the SQLite data and reconfigure a different approach >>>> for making the new data available to my application, e.g. MQTT. >>>> >>>> Appreciate any advice you WeeWx veterans may have for me. >>>> >>>> >>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/6c04a4ae-cd85-41c1-8e77-d8af7f92a684n%40googlegroups.com.
