I wrote this script to do an incremental backup of my running system. Use
at your own risk.
This script reads the weewx.sdb tables and insert/updates
records in a backup database. It gets the last datetime from each of the
weewx archive*
tables. For the archive table, it inserts any new records in the backup
database. For the archive_day_* tables, it deletes the last record in the
backup database and then inserts any new records.
The exception is the archive_day__metadata table. It is not touched.
I copied the weewx.sdb file to weewxCopy.sdb and then run the following to
backup the database.
zkwx_incrementCopy.py -f PATH_TO/weewx.sdb -t PATH_TO/weewxCopy.sdb
Good Luck,
Oscar
On Sunday, December 25, 2016 at 2:55:39 AM UTC-7, Per Edström wrote:
>
> I run my Weewx on Raspberry Pi and on a "PC" (Ebox 3350 and Ubuntu 10).
> Both have the OS on an SD/SDHC-card.
>
> Now, I find it messy to create a backup image:
> 1. Halt system
> 2. Power down
> 3. Remove SD-card
> 4. Put in laptop and create an image to file (5-10 min)
> 5. Insert SD-card in taget again and power up.
> 6. Write image file to new SD-card (15-20 min).
>
> (NB! I have noted that not all 16GB SD-card have the exact same size, it
> varies quite a lot. This became an issue the trying to find a new SD-card
> for the one that failed..)
>
> This procedure takes time and requires quite a lot of physical
> intervention.
>
> Is there any way of to create or maintain an updated image from the
> running system. Maybe it's Ok to just backup some files (user installed)?
>
> Also, it would be nice to have this "backup SD" as an alternative boot
> SD-card in case the primary one fails. I don't know if that is possible..
>
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
#!/usr/bin/python
################################################################################
# Filename: zk_wx_incrementCopy.py
# Description: This script eads the weewx.sdb tables and insert/updates
# records in a backup database. It gets the last datetime from the archive*
# tables. For the archive table, it inserts any new records in the backup
# database. For the archive_day_* tables, it deletes the last record in the
# backup database and then inserts any new records. It does not copy the
# archive_day__metadata table.
#
###
# Date Author Comment
# 2016-12-16 obarlow Create initial script
################################################################################
# IMPORT libraries
import datetime
import getopt
import os.path
import sqlite3
import sys
################################################################################
# define Variables
################################################################################
debugLevel = 1
################################################################################
# Connect to a data base
################################################################################
def connect_db( sqliteDB, readOnly ):
connection = None
try:
if readOnly == True:
# connection = sqlite3.connect(sqliteDB + '?mode=ro')
connection = sqlite3.connect(sqliteDB)
else:
connection = sqlite3.connect(sqliteDB)
return connection
except sqlite3.Error, e:
print "Error opening DB %s:" % sqliteDB
print "Error %s:" % e.args[0]
sys.exit(1)
################################################################################
# Execute sql
################################################################################
def execute_sql( connection, sqliteSQL ):
try:
cursor = connection.cursor()
cursor.execute(sqliteSQL)
return cursor
except sqlite3.Error, e:
print "Error SQL %s:" % sqliteSQL
print "Error %s:" % e.args[0]
sys.exit(1)
################################################################################
# getStartTime
################################################################################
def getStartTime(i_archiveCon ):
SQL = 'SELECT min(dateTime) minTime from archive'
wvArchiveCur = execute_sql( i_archiveCon, SQL )
wvArchiveRow = wvArchiveCur.fetchone()
return wvArchiveRow[0]
################################################################################
# get parameters
################################################################################
try:
opts, args = getopt.getopt(sys.argv[1:],"f:t:")
except getopt.GetoptError:
print 'no parms'
sys.exit(2)
if opts:
for opt, arg in opts:
if opt == '-f':
sourceDB = arg
elif opt == '-t':
targetDB = arg
else:
print "Unknown Parm:" + opt
sys.exit(2)
fromError = False
toError = False
if os.path.isfile(sourceDB):
a = 1
else:
fromError = True
if os.path.isfile(targetDB):
a = 1
else:
toError = True
if fromError or toError:
if fromError:
print "From database not found: " + sourceDB
if toError:
print "To database not found: " + targetDB
sys.exit(2)
################################################################################
# Main Process starts here
################################################################################
try:
commitCounter = 0;
commitMax = 100;
sourceCon = connect_db( sourceDB, True )
targetCon = connect_db( targetDB, False )
tableSQL = "SELECT name FROM sqlite_master WHERE type = 'table'"
tableSQL = tableSQL + " and name NOT IN ('archive_day__metadata', 'reportDate')";
tableCur = execute_sql ( sourceCon, tableSQL )
tableList = tableCur.fetchall()
for wvTable in tableList:
dateColumn = 'dateTime'
myTable = wvTable[0]
if debugLevel > 0:
print "Table=" + myTable,
# this is not executed because the table is excluded from the list
if myTable == 'archive_day__metadata':
mySQL = "delete from archive_day__metadata"
targetCon.execute ( mySQL )
dataSQL = "SELECT * FROM " + myTable
elif myTable == 'archive':
mySQL = "SELECT max(" + dateColumn + ") from " + myTable
myCur = execute_sql ( targetCon, mySQL )
startDateTime = myCur.fetchone()
dataSQL = "SELECT * FROM " + myTable + " WHERE " + dateColumn + " > " + str(startDateTime[0])
else:
mySQL = "SELECT max(" + dateColumn + ") from " + myTable
myCur = execute_sql ( targetCon, mySQL )
startDateTime = myCur.fetchone()
mySQL = "delete from " + myTable + " WHERE " + dateColumn + " >= " + str(startDateTime[0])
myCur = execute_sql ( targetCon, mySQL )
dataSQL = "SELECT * FROM " + myTable + " WHERE " + dateColumn + " >= " + str(startDateTime[0])
rowCounter = 0
dataCur = execute_sql ( sourceCon, dataSQL )
rows = dataCur.fetchall()
for row in rows:
insertSQL = "INSERT INTO " + myTable + " VALUES ("
for i in range(len(row)):
if i == 0:
myComma = ''
if myTable != 'metainfo':
myDateTime = str(row[i])
else:
myComma = ','
insertSQL = insertSQL + myComma + "'" + str(row[i]) + "'"
insertSQL = insertSQL + ")"
if debugLevel > 4:
print "SQL=" + insertSQL
targetCon.execute ( insertSQL )
commitCounter = commitCounter + 1
rowCounter = rowCounter + 1
if commitCounter >= commitMax:
targetCon.commit()
commitCounter = 0
# end for row in rows
targetCon.commit()
if debugLevel > 0:
print "Rows=" + str(rowCounter)
# end for wvTable in tableList
sourceCon.close()
targetCon.close()
# end of wviewDB in dbList
except sqlite3.Error, e:
# print "Error processing %s:" % sourceDB
# print "SQL=" + mySQL
print "Error %s:" % e.args[0]
print "dateTime=" + myDateTime
sys.exit(1)
