Stupid me. The answer is in how you are invoking wee_database. Any positional argument is interpreted as the path to weewx.conf, so wee_database is thinking that /home/weewx/archive/weewx.sdb is your configuration file, weewx.conf. Naturally, it being a sqlite database, it has all sorts of unusual byte sequences.
wee_database finds your database from the information in the configuration file, not from the command line. This way, it can work on MySQL databases, which do not use a single file. So, you want *wee_database --reconfigure /home/weewx/weewx.conf* On Thu, Aug 20, 2020 at 4:43 AM David Marshall <[email protected]> wrote: > > 1. Yes > 2. Seemed to work ok > 3. Syslog attached – but seems to me to be ok. > > > > The problem only occurs when using wee_database -interestingly the > offending byte has moved from position 26 to 31! > > > > pi@raspberrypi:~ $ /home/weewx/bin/wee_database --reconfigure > /home/weewx/archive/weewx.sdb > > Traceback (most recent call last): > > File "/home/weewx/bin/wee_database", line 974, in <module> > > main() > > File "/home/weewx/bin/wee_database", line 128, in main > > config_path, config_dict = weecfg.read_config(options.config_path, > args) > > File "/home/weewx/bin/weecfg/__init__.py", line 179, in read_config > > default_encoding='utf-8') > > File "/usr/lib/python3/dist-packages/configobj.py", line 1229, in > __init__ > > self._load(infile, configspec) > > File "/usr/lib/python3/dist-packages/configobj.py", line 1287, in _load > > content = self._handle_bom(content) > > File "/usr/lib/python3/dist-packages/configobj.py", line 1437, in > _handle_bom > > return self._decode(infile, self.encoding) > > File "/usr/lib/python3/dist-packages/configobj.py", line 1517, in _decode > > infile[i] = line.decode(encoding) > > UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf2 in position 31: > invalid continuation byte > > > > Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for > Windows 10 > > > > *From: *Tom Keffer <[email protected]> > *Sent: *20 August 2020 00:51 > *To: *weewx-user <[email protected]> > *Subject: *Re: [weewx-user] UnicodeDecodeError in database > > > > I don't have any problem with either copy of weewx.conf that you sent to > me. Things to try: > > > > 1. This is /home/weewx/weewx.conf, right? > > > > 2. Try reading it directly: > > > > *python -c "import configobj; > c=configobj.ConfigObj('/home/weewx/weewx.conf')"* > > > > 3. Set debug=1 in weewx.conf, then try running weewxd, then send us the > log. > > > > -tk > > > > > > > > On Wed, Aug 19, 2020 at 7:19 AM David Marshall <[email protected]> > wrote: > > Hi Tom > > > > 1. Yes wee_database > 2. Yes same path different copies > 3. No edits – in fact I tried not to change anything in the final > install apart from setting it to simulator. > > > > I attach the last version of weewx.conf, which is still producing the > error. > > > > David > > > > Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for > Windows 10 > > > > *From: *Tom Keffer <[email protected]> > *Sent: *19 August 2020 15:59 > *To: *weewx-user <[email protected]> > *Subject: *Re: [weewx-user] UnicodeDecodeError in database > > > > 1. All these attempts were with running wee_database? Or, something else? > > > > 2. I assume all these different attempts used the same path, namely > /home/weewx/weewx.conf. And, we're sure they used different copies (albeit, > at the same path location)? > > > > 3. It's hard to explain how fresh, untouched versions of weewx.conf are > all getting the same error. Are you editing these copies before running? > > > > Can you send me the offending weewx.conf to my personal email? tkeffer at > gmail.com > > > > -tk > > > > On Wed, Aug 19, 2020 at 6:35 AM David Marshall <[email protected]> > wrote: > > Thanks Tom > > > > Tried your code to show the offending line but nothing appears. Then tried > a clean weewx.conf from the install folder but still the same error > message. Then tried a full reinstall, but still the same error. > > Then tried it with the the default database using simulator and still the > same error - so you are right it is not my database. What next? > > On Wednesday, August 19, 2020 at 2:51:31 PM UTC+2, Tom Keffer wrote: > > The decode error is actually in your configuration file, weewx.conf, and > not in the database. > > > > Somehow, your copy of weewx.conf got corrupted. One of the lines has a > character sequence that starts out as a utf-8 encoding, but the following > byte is not what was expected. > > > > Unfortunately, the error does not tell you which line the bad character > sequence is in, only that it is in byte position 26 of that line. Perhaps > this will help (NOT TESTED): > > > > *od -x -A d weewx.conf | grep e0* > > > > This should print out the line and offending sequence. > > > > -tk > > > > > > > > On Wed, Aug 19, 2020 at 5:25 AM David Marshall <[email protected]> > wrote: > > Hi > > > > Installed a fresh version of weewx 4.1.1 without problem using setup.py on > a raspberry. > > > > Transferred my existing sqlite database from weewx 3.9.1. It is big, about > 3 years of data. This worked fine. > > > > But when I tried to use reconfigure I got the following error > > > > pi@raspberrypi:~ $ /home/weewx/bin/wee_database --reconfigure > /home/weewx/archive/weewx.sdb > > Traceback (most recent call last): > > File "/home/weewx/bin/wee_database", line 974, in <module> > > main() > > File "/home/weewx/bin/wee_database", line 128, in main > > config_path, config_dict = weecfg.read_config(options.config_path, > args) > > File "/home/weewx/bin/weecfg/__init__.py", line 179, in read_config > > default_encoding='utf-8') > > File "/usr/lib/python3/dist-packages/configobj.py", line 1229, in > __init__ > > self._load(infile, configspec) > > File "/usr/lib/python3/dist-packages/configobj.py", line 1287, in _load > > content = self._handle_bom(content) > > File "/usr/lib/python3/dist-packages/configobj.py", line 1437, in > _handle_bom > > return self._decode(infile, self.encoding) > > File "/usr/lib/python3/dist-packages/configobj.py", line 1517, in _decode > > infile[i] = line.decode(encoding) > > UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe0 in position 26: > invalid continuation byte > > > > Anyone know how to fix this? > > > > > > > > > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/97c28bf4-30c7-455c-851f-469187c31c79o%40googlegroups.com > <https://groups.google.com/d/msgid/weewx-user/97c28bf4-30c7-455c-851f-469187c31c79o%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/93094213-6958-49a1-ba13-7f3ca3a90274o%40googlegroups.com > <https://groups.google.com/d/msgid/weewx-user/93094213-6958-49a1-ba13-7f3ca3a90274o%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/CAPq0zEDyDz5xh8gtJRyeCKxmCbJGuAYqChLJeq4d%2BGQrD3vUFA%40mail.gmail.com > <https://groups.google.com/d/msgid/weewx-user/CAPq0zEDyDz5xh8gtJRyeCKxmCbJGuAYqChLJeq4d%2BGQrD3vUFA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/5f3d34eb.1c69fb81.4c509.d9cd%40mx.google.com > <https://groups.google.com/d/msgid/weewx-user/5f3d34eb.1c69fb81.4c509.d9cd%40mx.google.com?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/CAPq0zEB99R%3D5MrZB6EYfi1MOPcW96E06bPs3ayV5FAPF_aK7NA%40mail.gmail.com > <https://groups.google.com/d/msgid/weewx-user/CAPq0zEB99R%3D5MrZB6EYfi1MOPcW96E06bPs3ayV5FAPF_aK7NA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEAKFYrufMVdf%3DL3YKuy2uXp%2B5sXdQeD77DV7LEWXB6qrg%40mail.gmail.com.
