Hey Doug, thanks for chiming in. As a matter of fact I'm not really worried about the data to be transmitted incrementally which, as you say, is not large by any mean.
What I'm trying to avoid is having to transfer the whole database dump every time that a backup is performed, because that can become demanding when the archive grows for many years: I have another station running a similar DB and after more than 10 years the full DB has become about 400 MB in size... On Thursday, July 28, 2022 at 4:31:55 PM UTC+2 do...@dougjenkins.com wrote: > Guido: > > I was going to recommend if this is that important to you to configure a > database cluster. MariaDB has Galera Clustering (What is MariaDB Galera > Cluster? - MariaDB Knowledge Base > <https://mariadb.com/kb/en/what-is-mariadb-galera-cluster/>) which will > perform database replication and distribution for you. You need at least 3 > nodes/servers to have a proper cluster. > > Keep in mind that WeeWX only writes to the database during the archive > operation, so if you archive every minute, you are only writing 1440 > records to the archive table (and another 1440+ records to the summary > tables per measurement). that is literally kilobytes of data per day. With > today's modern networks, performance is not an issue. > > You can spend the time to build an incremental backup solution, but I > think that is a lot of work for a solution that writes so little amount of > data each day. > > Just my $0.02 > > DDJ > > On Thu, Jul 28, 2022 at 10:27 AM Guido Cioni <guido...@gmail.com> wrote: > >> From what I understand weewx uses sqlite because of the default, and with >> sqlite the only possibility is to dump the whole database every time ( >> https://github.com/glennmckechnie/weewx-sqlbackup/wiki/SQLBackup-README), >> which is obviously time consuming. >> >> With Pandas you can get around that by doing queries and requesting only >> the data that you need to transfer every time, and the data gets saved into >> the memory allocated by Python so the db is left untouched. Of course one >> does not necessarily have to use Pandas, it's just one of the framework >> that I'm familiar with. >> >> Plus, it seems that none of the extensions in the wiki allows for saving >> data directly into a DB but only for making a physical copy of the DB >> (again, from my understanding). >> >> On Thursday, July 28, 2022 at 4:23:04 PM UTC+2 vince...@gmail.com wrote: >> >>> Can you get there with doing mysql replication at the database level ? >> >> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-development" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to weewx-developm...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/weewx-development/391c83da-fb02-447e-bcee-fed1b42bbbccn%40googlegroups.com >> >> <https://groups.google.com/d/msgid/weewx-development/391c83da-fb02-447e-bcee-fed1b42bbbccn%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "weewx-development" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-development+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/b45cf968-ee04-4256-bd06-b70269d1eb4bn%40googlegroups.com.
