good morning, i solved this with a workaround. since the data is coming in, but i can't get it directly into the weewx database, i now rewrite it via shell script into a csv file, which i then import cron-based with the weewx import module. it's not 100% live. but at least i'm in. i changed the original logger to write to csv: https://pastebin.com/mhAHT2fP cron does this: * * * * * /home/pi/git/logger/logger.sh >/dev/null 2>&1 # every minute */10 * * * * /home/pi/git/logger/logger-import.sh >/dev/null 2>&1 # every 10 minutes
On Tuesday, August 17, 2021 at 10:41:50 AM UTC+2 gjr80 wrote: > No, I never said that at all. In fact looking at the driver again it seems > I misread and the driver does not get the time initially from the last > record in the WeeWX archive but rather it gets a timestamp from the > dateTime field in table last_sensor. The last_sensor table appears to be > another table created by the logger. Otherwise it’s as I said previously; > the time is then obtained from the last record in the sensors table if it > can’t be obtained from the last_sensor table. > > So the upshot is that you need to get the last_sensor and sensors tables > being populated by the logger. I would be looking closely at the logger > part of the system and then worry about the driver, > > Gary > On Tuesday, 17 August 2021 at 17:55:28 UTC+10 Loden Hos wrote: > >> thanks gjr80 for your explanations. until now i did not at all understand >> what the vueiss driver does. if i get you right, what i should try is: seed >> some plausible database entries. allow me to add a question: how would i do >> that "manually"? anyway: in a few days i will be away for a couple weeks. >> but i will definitely come back to that approach. >> best, lh >> >> On Tuesday, August 17, 2021 at 5:14:27 AM UTC+2 gjr80 wrote: >> >>> Can’t say I’ve ever seen a driver like this before. From a quick look at >>> the code it appears the system is in two parts. First there is a ‘logger’ >>> that runs independently of the driver, the logger intercepts the RF from >>> the ISS and records data (including time stamps) in a separate table (table >>> sensors) in the WeeWX database. Then the driver reads this data and emits >>> loop packets. I would more accurately describe the driver as ‘reading data >>> from a database and emitting loop packets’ rather than reading data from >>> the ISS. >>> >>> The driver gets its ‘time’ from (initially) the last good record in the >>> WeeWX archive. If there are no records in the archive the ‘time’ is picked >>> up from this intermediate table created and used by the logger. If you have >>> a new install then you likely have no archive records, so there is no last >>> good timestamp from the archive. In this case the driver will go to the >>> intermediate sensors table and get a timestamp. Judging by the log which >>> says ‘starting at 0’ I’m guessing this table is not being populated >>> correctly and hence no ‘time’ is being picked up resulting in the large >>> discrepancy (the driver is effectively being told by the ‘station’ it is >>> the start of Unix epoch time (t = 0)) >>> >>> As Vince said you need to start by fixing the time, without that being >>> fixed WeeWX will never see a loop packet, you will never have archive >>> records created and hence nothing in the archive and no reports; WeeWX will >>> just appear to do nothing. Given the peculiar arrangement within the driver >>> for time I suspect that the logger, whilst maybe getting data from the ISS, >>> is not populating the sensors table. If you can’t figure that out my first >>> point of call would be the author. And if that yields no joy I would >>> seriously consider buying a console :) >>> >>> Gary >>> >>> On Tuesday, 17 August 2021 at 07:53:51 UTC+10 Loden Hos wrote: >>> >>>> thanks vince, you made me try all options of getting good time: disable >>>> all time services and reboot. i even installed a piface shim real time >>>> clock and rebooted, allways the same: simulator driver works with correct >>>> time. vueiss comes up with a clock error of ~51 years (1.1.1970??). if >>>> there is no time service and no rtc at all, weewx is still working >>>> properly: if there is no time, it logs "Feb 14 11:12:06 wetterbot2 >>>> python2[389]: weewx[389] INFO __main__: Waiting for sane time. Current >>>> time >>>> is 2019-02-14 11:12:06 CET (1550139126)". and it is really waiting, and if >>>> i then start a time demon or the rtc service, it continues starting - >>>> unfortunately not with the correct time provided meanwhile but with its >>>> 51year-time error. >>>> that all drives me to believe that the time error doesn't result from >>>> false rpi or weewx setup but has to do with the vueiss driver. >>>> >>>> >>>> On Monday, August 16, 2021 at 7:52:03 PM UTC+2 vince wrote: >>>> >>>>> I would start by fixing the clock error. Make sure you install and >>>>> enable ntp in your os and get the clock to match true time. You might >>>>> also >>>>> check the clock on your Vantage to verify it is correct. >>>>> >>>>> >>>>> On Monday, August 16, 2021 at 3:41:47 AM UTC-7 Loden Hos wrote: >>>>> >>>>>> Good morning, >>>>>> >>>>>> I am trying to run weewx on a raspberry pi 2b (Rev 1.1) with a custom >>>>>> driver named vueiss.py. >>>>>> this driver can fetch data from davis vantage vue without the davis >>>>>> console. >>>>>> Its installation and use is not very good documented, see the authors >>>>>> ressources: >>>>>> https://buwx.de/index.php/technik >>>>>> https://buwx.de/index.php/technik/79-logger2 >>>>>> https://github.com/buwx/ >>>>>> Sorry, the links are only in german. >>>>>> >>>>>> But with a little effort, everything works fine. >>>>>> The Simulator is writing to the mariadb and reports are showing up. >>>>>> The pi is receiving data from the Vue ISS on its konsole. >>>>>> The vueiss driver is successfully installed and activated in weewx. >>>>>> Though i noted a massive clock error with vueiss-driver: >>>>>> --Aug 16 00:32:46 wetterbot2 weewx[21924] INFO weewx.engine: Clock >>>>>> error is -1629066766.19 seconds (positive is fast) >>>>>> compared to simulater driver: >>>>>> --Aug 16 02:10:46 wetterbot2 weewx[1477] INFO weewx.engine: Clock >>>>>> error is -0.22 seconds (positive is fast) >>>>>> >>>>>> Only problem: With the vueiss driver no data is written into the >>>>>> database. no reports. >>>>>> >>>>>> Please see my logs: >>>>>> For wee_debug --info --verbosity=2, see https://pastebin.com/0N3m5cJP >>>>>> For tail -f /var/log/syslog after a restart of weewx service, see >>>>>> https://pastebin.com/67F02BTG >>>>>> >>>>>> Finally I am attaching my install notes, as suggested by the forum >>>>>> guidelines. >>>>>> >>>>>> Looking forward to your hints, >>>>>> best, Loden >>>>>> >>>>>> -----install os----- >>>>>> >>>>>> # sd-card: image 2021-05-07-raspios-buster-armhf-lite.zip >>>>>> # flashed with balena etcher >>>>>> # >>>>>> # file ssh created in /boot >>>>>> # >>>>>> # raspi booted with new sd >>>>>> >>>>>> sudo raspi-config >>>>>> # changes in: >>>>>> # -local >>>>>> # -timezone >>>>>> # -keyboard >>>>>> # -spi >>>>>> # -hostname >>>>>> >>>>>> sudo apt update >>>>>> sudo apt dist-upgrade >>>>>> >>>>>> sudo shutdown -r now >>>>>> >>>>>> -----setup python----- >>>>>> # >>>>>> https://www.techcoil.com/blog/how-to-setup-raspbian-stretch-lite-on-raspberry-pi-3-to-run-python-3-applications/ >>>>>> >>>>>> sudo apt-get install python3-pip python-pip virtualenv >>>>>> pip3 list >>>>>> >>>>>> python -V >>>>>> python3 -V >>>>>> sudo update-alternatives --list python >>>>>> sudo update-alternatives --install /usr/bin/python python >>>>>> /usr/bin/python2.7 1 >>>>>> sudo update-alternatives --install /usr/bin/python python >>>>>> /usr/bin/python3.7 2 >>>>>> >>>>>> >>>>>> -----install nginx, mariadb, php7, phpmyadmin----- >>>>>> # https://www.raspifun.de/viewtopic.php?t=24 >>>>>> sudo apt-get install nginx >>>>>> >>>>>> sudo apt-get install mariadb-server mariadb-client >>>>>> sudo mariadb >>>>>> CREATE USER 'davis'@'localhost' IDENTIFIED BY 'davis'; >>>>>> GRANT ALL ON *.* TO 'davis'@'localhost'; >>>>>> FLUSH PRIVILEGES; >>>>>> exit >>>>>> >>>>>> sudo apt-get install php7.3 php7.3-fpm php7.3-mysql libzip4 php-pear >>>>>> php7.3-curl php7.3-gd php7.3-intl php7.3-xml php7.3-zip php7.3-mbstring >>>>>> sudo nano /etc/nginx/sites-available/default >>>>>> # pass PHP scripts to FastCGI server >>>>>> # >>>>>> location ~ \.php$ { >>>>>> include snippets/fastcgi-php.conf; >>>>>> # >>>>>> # # With php-fpm (or other unix sockets): >>>>>> fastcgi_pass unix:/var/run/php/php7.3-fpm.sock; >>>>>> # # With php-cgi (or other tcp sockets): >>>>>> # fastcgi_pass 127.0.0.1:9000; >>>>>> } >>>>>> >>>>>> # Add index.php to the list if you are using PHP >>>>>> index index.php index.html index.htm; >>>>>> >>>>>> sudo /etc/init.d/php7.3-fpm reload && sudo /etc/init.d/nginx reload >>>>>> >>>>>> sudo apt-get install phpmyadmin >>>>>> sudo nano /etc/nginx/sites-available/default >>>>>> # add: >>>>>> server { >>>>>> listen 81; >>>>>> server_name wetterbot2; >>>>>> root /usr/share/phpmyadmin; >>>>>> index index.php index.html index.htm; >>>>>> if (!-e $request_filename) { >>>>>> rewrite ^/(.+)$ /index.php?url=$1 last; >>>>>> break; >>>>>> } >>>>>> location ~ \.php$ { >>>>>> try_files $uri =404; >>>>>> fastcgi_pass unix:/var/run/php/php7.3-fpm.sock; >>>>>> fastcgi_index index.php; >>>>>> fastcgi_param SCRIPT_FILENAME >>>>>> $document_root/$fastcgi_script_name; >>>>>> include /etc/nginx/fastcgi_params; >>>>>> } >>>>>> } >>>>>> sudo /etc/init.d/php7.3-fpm reload && sudo /etc/init.d/nginx reload >>>>>> # find phpmyadmin at https://192.168.1.164:81 >>>>>> >>>>>> -----install weewx----- >>>>>> # >>>>>> https://www.bitblokes.de/wetterstation-mit-raspberry-pi-sense-hat-und-weewx-schritt-fuer-schritt/ >>>>>> >>>>>> wget -qO - http://weewx.com/keys.html | sudo apt-key add - >>>>>> wget -qO - http://weewx.com/apt/weewx.list | sudo tee >>>>>> /etc/apt/sources.list.d/weewx.list >>>>>> sudo apt-get update >>>>>> sudo apt-get install weewx >>>>>> Die folgenden NEUEN Pakete werden installiert: >>>>>> libimagequant0 liblcms2-2 libwebpdemux2 libwebpmux3 python-cheetah >>>>>> python-configobj python-olefile python-pil python-serial python-usb >>>>>> weewx >>>>>> lat, long: 52.492580,13.428213 >>>>>> sudo service weewx status >>>>>> >>>>>> echo "weewx_reports /var/weewx/reports tmpfs >>>>>> size=20M,noexec,nosuid,nodev 0 0" | sudo tee -a /etc/fstab >>>>>> sudo mkdir -p /var/weewx/reports >>>>>> sudo mount -a >>>>>> ggf. >>>>>> sudo cp -r /var/www/html/weewx/ /var/weewx/reports/ >>>>>> sudo rm -r /var/www/html/weewx >>>>>> sudo sed -i -e 's%HTML_ROOT =.*%HTML_ROOT = /var/weewx/reports%' >>>>>> /etc/weewx/weewx.conf >>>>>> sudo service weewx restart >>>>>> sudo ln -s /var/weewx/reports /var/www/html/weewx >>>>>> sudo chmod -R 755 /var/www/html/weewx >>>>>> # find weewx at http://192.168.1.164/weewx/ >>>>>> >>>>>> # https://github.com/weewx/weewx/wiki/Raspberry%20Pi >>>>>> sudo apt-get purge fake-hwclock >>>>>> sudo apt-get install ftp >>>>>> >>>>>> # https://github.com/weewx/weewx/wiki/Minimize-writes-on-SD-cards >>>>>> sudo dphys-swapfile swapoff >>>>>> >>>>>> # http://www.weewx.com/docs/usersguide.htm#configuring_mysql >>>>>> sudo apt install python3-mysqldb python-mysqldb >>>>>> [[wx_binding]] >>>>>> # The database should match one of the sections in [Databases] >>>>>> database = archive_mysql >>>>>> [[MySQL]] >>>>>> driver = weedb.mysql >>>>>> host = localhost >>>>>> user = davis >>>>>> password = davis >>>>>> sudo service weewx restart >>>>>> >>>>>> # up to here: success. simulator data is saved in mariadb and shown >>>>>> in reports. >>>>>> >>>>>> -----install RaspiRFM II radio module software----- >>>>>> # http://www.seegel-systeme.de/produkt/raspyrfm-ii/ >>>>>> # >>>>>> http://www.seegel-systeme.de/2015/09/02/ein-funkmodul-fuer-den-raspberry-raspyrfm/#Installation >>>>>> sudo apt-get install git-core python-dev python-pip >>>>>> cd git >>>>>> git clone https://github.com/Phunkafizer/RaspyRFM.git >>>>>> cd RaspyRFM >>>>>> sudo pip3 install -e . >>>>>> sudo pip install -e . >>>>>> cd >>>>>> >>>>>> -----setup VueISS device software and driver----- >>>>>> # https://buwx.de/index.php/technik >>>>>> # https://buwx.de/index.php/technik/79-logger2 >>>>>> # https://github.com/buwx/ >>>>>> sudo apt install git >>>>>> mkdir git >>>>>> cd git >>>>>> # driver: >>>>>> git clone https://github.com/buwx/weewx >>>>>> # software: >>>>>> git clone https://github.com/buwx/logger >>>>>> cd >>>>>> pi@wetterbot2:~ $ python3 git/logger/raspi_rfm.py >>>>>> 2021-08-15 22:06:43,649 INFO Starting Davis-ISS logging >>>>>> 2021-08-15 22:06:47,949 INFO I 100 50 00 7A FF 71 00 AE 41 >>>>>> -51.5 -9 1 rr=0mm/h w(0km/h, 172) >>>>>> 2021-08-15 22:06:50,512 INFO I 101 80 00 7A 2E 09 00 D9 A1 >>>>>> -50.5 -12 1 t=23.1C w(0km/h, 172) >>>>>> 2021-08-15 22:06:53,074 INFO I 102 70 00 7A 02 C1 80 B5 CF >>>>>> -51.0 -10 1 sol=11 w(0km/h, 172) >>>>>> 2021-08-15 22:06:58,199 INFO I 104 50 00 7A FF 71 00 AE 41 >>>>>> -50.5 -13 2 rr=0mm/h w(0km/h, 172) >>>>>> ^C >>>>>> # at this point the ISS is sending data, pi is receiving it. >>>>>> >>>>>> Setup weewx to use the driver named vueiss.py >>>>>> --------------------------------------------- >>>>>> # add tables to database weewx (phpmyadmin): >>>>>> CREATE TABLE sensor ( >>>>>> dateTime BIGINT NOT NULL, >>>>>> data VARCHAR(80), >>>>>> description VARCHAR(80), >>>>>> INDEX (dateTime) >>>>>> ); >>>>>> >>>>>> CREATE TABLE last_sensor ( >>>>>> dateTime BIGINT NOT NULL >>>>>> ); >>>>>> >>>>>> INSERT INTO last_sensor VALUES(0); >>>>>> >>>>>> cd >>>>>> sudo cp git/weewx/bin/user/drivers/vueiss.py >>>>>> /usr/share/weewx/user/vueiss.py >>>>>> sudo cp git/weewx/skins/Standard/* /etc/weewx/skins/Standard/ >>>>>> >>>>>> sudo nano /etc/weewx/weewx.conf >>>>>> # apply changes from ~/git/weewx/weewx.conf.part >>>>>> # but adjust the driver location: >>>>>> [VueISS] >>>>>> # The driver to use: >>>>>> #driver = user.drivers.vueiss >>>>>> driver = user.vueiss >>>>>> # and because of >>>>>> https://weewx.com/docs/usersguide.htm#archive_interval >>>>>> # uncomment: >>>>>> [StdArchive] >>>>>> # If the station hardware supports data logging then the archive >>>>>> interval >>>>>> # will be downloaded from the station. Otherwise, specify it (in >>>>>> seconds). >>>>>> archive_interval = 60 >>>>>> >>>>>> sudo apt install python3-numpy python-numpy libatlas-base-dev >>>>>> >>>>>> sudo wee_config --reconfigure >>>>>> description [Wetterbot2, Balkon, Friedel]: >>>>>> altitude [42, meter]: >>>>>> latitude [52.492580]: >>>>>> longitude [13.428213]: >>>>>> Include station in the station registry (y/n)? [n]: >>>>>> units [metric]: >>>>>> Installed drivers include: >>>>>> 0) VueISS (user.vueiss) >>>>>> 1) AcuRite (weewx.drivers.acurite) >>>>>> ... >>>>>> choose a driver [0]: >>>>>> >>>>>> sudo service weewx restart >>>>>> >>>>>> # at this point vueiss.py causes a couple of errors >>>>>> # probably due to python2-python3-incompatibilities. >>>>>> >>>>>> python /usr/share/weewx/user/vueiss.py >>>>>> File "/usr/share/weewx/user/vueiss.py", line 532 >>>>>> print weeutil.weeutil.timestamp_to_string(packet['dateTime']), >>>>>> packet >>>>>> ^ >>>>>> SyntaxError: invalid syntax >>>>>> sudo nano /usr/share/weewx/user/vueiss.py >>>>>> Alt+c >>>>>> ^_ >>>>>> 532 >>>>>> old: print >>>>>> weeutil.weeutil.timestamp_to_string(packet['dateTime']), packet >>>>>> new: >>>>>> print(weeutil.weeutil.timestamp_to_string(packet['dateTime']), packet) >>>>>> >>>>>> ModuleNotFoundError: No module named 'Queue' >>>>>> sudo apt-get install libmariadb-dev-compat libmariadb-dev >>>>>> sudo nano /usr/share/weewx/user/vueiss.py >>>>>> Alt+c >>>>>> ^_ >>>>>> 17 >>>>>> old: import Queue >>>>>> new: try: >>>>>> import queue >>>>>> except ImportError: >>>>>> import Queue as queue >>>>>> 459 >>>>>> old: self.packets = Queue.Queue() >>>>>> new: self.packets = queue.Queue() >>>>>> >>>>>> >>>>>> # now driver is loaded successfully and can easily be switched on and >>>>>> off by >>>>>> # sudo wee_config --reconfigure --driver=user.vueiss --no-prompt >>>>>> # sudo wee_config --reconfigure --driver=weewx.drivers.simulator >>>>>> --no-prompt >>>>>> # sudo service weewx restart >>>>>> # my version of vueiss.py: https://pastebin.com/YHEqnr3u >>>>>> # >>>>>> # though noted a massive clock error with vueiss-driver: >>>>>> Aug 16 00:32:46 wetterbot2 weewx[21924] INFO weewx.engine: Clock >>>>>> error is -1629066766.19 seconds (positive is fast) >>>>>> # compared to simulater driver: >>>>>> Aug 16 02:10:46 wetterbot2 weewx[1477] INFO weewx.engine: Clock error >>>>>> is -0.22 seconds (positive is fast) >>>>>> >>>>>> # Problem: No data is saved in database and/or shown in reports. >>>>>> # See the logs: >>>>>> sudo wee_debug --info --verbosity=2 >>>>>> # see https://pastebin.com/0N3m5cJP >>>>>> tail -f /var/log/syslog >>>>>> # see https://pastebin.com/67F02BTG >>>>>> >>>>>> >>>>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/7aae7925-09eb-471e-8a35-56a6df4fc715n%40googlegroups.com.
