Hello, I have a problem with weewx in connection with MQTT. I only use sensors that transmit the data via MQTT to the broker Mosquitto (Rasberrypi 2) with the ESP8266 (no commercial weather station). Unfortunately, the data is not transferred to weewx. I attach to logfile, my weewx.conf, wxMesh.py and a copy of Mosquitto data from the broker. Can someone help me? weewx is perfect for me to record my data from my self-made weather station. thanks in advance! Greetings Franz
----- weewx.service - LSB: weewx weather system Loaded: loaded (/etc/init.d/weewx) Active: active (exited) since So 2018-05-20 19:13:31 CEST; 12s ago Process: 1203 ExecStop=/etc/init.d/weewx stop (code=exited, status=0/SUCCESS) Process: 1275 ExecStart=/etc/init.d/weewx start (code=exited, status=0/SUCCESS) Mai 20 19:13:31 raspberrypi weewx[1289]: **** self.setupStation(config_dict) Mai 20 19:13:31 raspberrypi weewx[1289]: **** File "/home/weewx/bin/weewx/engine.py", line 95, in setupStation Mai 20 19:13:31 raspberrypi weewx[1289]: **** __import__(driver) Mai 20 19:13:31 raspberrypi weewx[1289]: **** File "/home/weewx/bin/user/wxMesh.py", line 48 Mai 20 19:13:31 raspberrypi weewx[1289]: **** syslog.syslog(dst, 'wxMesh: %s' % msg) Mai 20 19:13:31 raspberrypi weewx[1289]: **** ^ Mai 20 19:13:31 raspberrypi weewx[1289]: **** SyntaxError: invalid syntax Mai 20 19:13:31 raspberrypi weewx[1289]: **** Exiting. Mai 20 19:13:31 raspberrypi systemd[1]: Started LSB: weewx weather system. Mai 20 19:13:31 raspberrypi weewx[1275]: Starting weewx weather system: weewx. ---------------------------------- # WEEWX CONFIGURATION FILE # # Copyright (c) 2009-2015 Tom Keffer <[email protected]> # See the file LICENSE.txt for your rights. ############################################################################## # This section is for general configuration information. # Set to 1 for extra debug info, otherwise comment it out or set to zero debug = 1 # Root directory of the weewx data file hierarchy for this station WEEWX_ROOT = /home/weewx # How long to wait before timing out a socket (FTP, HTTP) connection socket_timeout = 20 # Do not modify this. It is used when installing and updating weewx. version = 3.8.0 ############################################################################## # This section is for information about the station. [Station] station_type = wxMesh [wxMesh] host = 192.168.1.30 username = xxxxxx password = yyyyyy topic = weewx/+ driver = user.wxMesh poll_interval = 1 loop_on_init = true [[label_map]] TIME = dateTime HUMT = outTemp RHUM = outHumidity # Description of the station location location = Wetter Rheinbach Voreifel # Latitude and longitude in decimal degrees latitude = 50.3552 longitude = 6.5346 # Altitude of the station, with unit it is in. This is downloaded from # from the station if the hardware supports it. altitude = 270, meter # Choose 'foot' or 'meter' for unit # Set to type of station hardware. There must be a corresponding stanza # in this file with a 'driver' parameter indicating the driver to be used. # station_type = Simulator # If you have a website, you may specify an URL #station_url = http://www.example.com # The start of the rain year (1=January; 10=October, etc.). This is # downloaded from the station if the hardware supports it. rain_year_start = 1 # Start of week (0=Monday, 6=Sunday) week_start = 6 ############################################################################## [Simulator] # This section is for the weewx weather station simulator # The time (in seconds) between LOOP packets. loop_interval = 2.5 # The simulator mode can be either 'simulator' or 'generator'. # Real-time simulator. Sleep between each LOOP packet. mode = simulator # Generator. Emit LOOP packets as fast as possible (useful for testing). #mode = generator # The start time. Format is YYYY-mm-ddTHH:MM. If not specified, the default # is to use the present time. #start = 2011-01-01T00:00 # The driver to use: # driver = weewx.drivers.simulator ############################################################################## # This section is for uploading data to Internet sites [StdRESTful] [[MQTT]] topic = weewx append_units_label = True retain = False aggregation = individual unit_system = METRIC binding = loop [[StationRegistry]] # To register this weather station with weewx, set this to true register_this_station = false [[AWEKAS]] # This section is for configuring posts to AWEKAS. # If you wish to do this, set the option 'enable' to true, # and specify a username and password. enable = false username = replace_me # To guard against parsing errors, put your password in quotes: password = replace_me [[CWOP]] # This section is for configuring posts to CWOP. # If you wish to do this, set the option 'enable' to true, # and specify the station ID (e.g., CW1234). enable = false station = replace_me # If this is an APRS (radio amateur) station, uncomment # the following and replace with a passcode (e.g., 12345). #passcode = replace_me (APRS stations only) [[PWSweather]] # This section is for configuring posts to PWSweather.com. # If you wish to do this, set the option 'enable' to true, # and specify a station and password. enable = false station = replace_me # To guard against parsing errors, put your password in quotes: password = replace_me [[WOW]] # This section is for configuring posts to WOW. # If you wish to do this, set the option 'enable' to true, # and specify a station and password. enable = false station = replace_me # To guard against parsing errors, put your password in quotes: password = replace_me [[Wunderground]] # This section is for configuring posts to the Weather Underground. # If you wish to do this, set the option 'enable' to true, # and specify a station (e.g., 'KORHOODR3') and password. enable = false station = replace_me # To guard against parsing errors, put your password in quotes: password = replace_me # Set the following to True to have weewx use the WU "Rapidfire" # protocol. Not all hardware can support it. See the User's Guide. rapidfire = False ############################################################################## # This section specifies what reports, using which skins, to generate. [StdReport] # Where the skins reside, relative to WEEWX_ROOT SKIN_ROOT = skins # Where the generated reports should go, relative to WEEWX_ROOT HTML_ROOT = public_html # The database binding indicates which data should be used in reports. data_binding = wx_binding # Each of the following subsections defines a report that will be run. [[StandardReport]] # See the customizing guide to change the units, plot types and line # colors, modify the fonts, display additional sensor data, and other # customizations. Many of those changes can be made here by overriding # parameters, or by modifying templates within the skin itself. # The StandardReport uses the 'Standard' skin, which contains the # images, templates and plots for the report. skin = Standard [[[Units]]] [[[[Groups]]]] group_altitude = meter group_speed2 = meter_per_second2 group_pressure = mbar group_rain = mm group_rainrate = mm_per_hour group_temperature = degree_C group_degree_day = degree_C_day group_speed = meter_per_second [[FTP]] # FTP'ing the results to a webserver is treated as just another report, # albeit one with an unusual report generator! skin = Ftp # If you wish to use FTP, uncomment and fill out the next four lines. #user = replace with the ftp username #password = replace with the ftp password; put in quotes to guard against parsing errors. #server = replace with the ftp server name, e.g, www.threefools.org #path = replace with the ftp destination directory (e.g., /weather) # Set to True for an FTP over TLS (FTPS) connection. Not all servers # support this. secure_ftp = False # To upload files from something other than what HTML_ROOT is set # to above, specify a different HTML_ROOT here. #HTML_ROOT = public_html # Most FTP servers use port 21 port = 21 # Set to 1 to use passive mode, zero for active mode passive = 1 [[RSYNC]] # rsync'ing to a webserver is treated as just another report skin = Rsync # If you wish to use rsync, you must configure passwordless ssh using # public/private key authentication from the user account that weewx # runs as to the user account on the remote machine where the files # will be copied. # # The following three lines determine where files will be sent. #server = replace with the rsync server name, e.g, www.threefools.org #path = replace with the rsync destination directory (e.g., /weather) #user = replace with the rsync username # Rsync can be configured to remove files from the remote server if # they don't exist under HTML_ROOT locally. USE WITH CAUTION: if you # make a mistake in the remote path, you could could unintentionally # cause unrelated files to be deleted. Set to 1 to enable remote file # deletion, zero to allow files to accumulate remotely. delete = 0 ############################################################################## # This service acts as a filter, converting the unit system coming from # the hardware to a unit system in the database. [StdConvert] # The target_unit affects only the unit system in the database. Once # chosen it cannot be changed without converting the entire database. # Modification of target_unit after starting weewx will result in # corrupt data - the database will contain a mix of US and METRIC data. # # The value of target_unit does not affect the unit system for # reporting - reports can display US, Metric, or any combination of units. # # In most cases, target_unit should be left as the default: US # # In particular, those migrating from a standard wview installation # should use US since that is what the wview database contains. # DO NOT MODIFY THIS VALUE UNLESS YOU KNOW WHAT YOU ARE DOING! target_unit = US # Options are 'US', 'METRICWX', or 'METRIC' ############################################################################## # This section can adjust data using calibration expressions. [StdCalibrate] [[Corrections]] # For each type, an arbitrary calibration expression can be given. # It should be in the units defined in the StdConvert section. # Example: foo = foo + 0.2 ############################################################################## # This section is for quality control checks. If units are not specified, # values must be in the units defined in the StdConvert section. [StdQC] [[MinMax]] barometer = 26, 32.5, inHg outTemp = -40, 120, degree_F inTemp = 10, 120, degree_F outHumidity = 0, 100 inHumidity = 0, 100 windSpeed = 0, 120, mile_per_hour pressure = 24, 34.5, inHg ############################################################################## # This section controls the origin of derived values. [StdWXCalculate] [[Calculations]] # Derived quantities are calculated by this service. Possible values are: # hardware - use the value provided by hardware # software - use the value calculated by weewx # prefer_hardware - use value provide by hardware if available, # otherwise use value calculated by weewx pressure = prefer_hardware barometer = prefer_hardware altimeter = prefer_hardware windchill = prefer_hardware heatindex = prefer_hardware dewpoint = prefer_hardware inDewpoint = prefer_hardware rainRate = prefer_hardware ############################################################################## # For hardware that supports it, this section controls how often the # onboard clock gets updated. [StdTimeSynch] # How often to check the weather station clock for drift (in seconds) clock_check = 14400 # How much it can drift before we will correct it (in seconds) max_drift = 5 ############################################################################## # This section is for configuring the archive service. [StdArchive] # If the station hardware supports data logging then the archive interval # will be downloaded from the station. Otherwise, specify it (in seconds). archive_interval = 300 # If possible, new archive records are downloaded from the station # hardware. If the hardware does not support this, then new archive # records will be generated in software. # Set the following to "software" to force software record generation. record_generation = hardware # Whether to include LOOP data in hi/low statistics loop_hilo = True # The data binding used to save archive records data_binding = wx_binding ############################################################################## # This section binds a data store to a database. [DataBindings] [[wx_binding]] # The database must match one of the sections in [Databases]. # This is likely to be the only option you would want to change. database = archive_sqlite # The name of the table within the database table_name = archive # The manager handles aggregation of data for historical summaries manager = weewx.wxmanager.WXDaySummaryManager # The schema defines the structure of the database. # It is *only* used when the database is created. schema = schemas.wview.schema ############################################################################## # This section defines various databases. [Databases] # A SQLite database is simply a single file [[archive_sqlite]] database_type = SQLite database_name = weewx.sdb # MySQL [[archive_mysql]] database_type = MySQL database_name = weewx ############################################################################## # This section defines defaults for the different types of databases. [DatabaseTypes] # Defaults for SQLite databases [[SQLite]] driver = weedb.sqlite # Directory in which the database files are located SQLITE_ROOT = %(WEEWX_ROOT)s/archive # Defaults for MySQL databases [[MySQL]] driver = weedb.mysql # The host where the database is located host = localhost # The user name for logging in to the host user = weewx # The password for the user name. Put in quotes to guard against parsing errors. password = weewx ############################################################################## # This section configures the internal weewx engine. [Engine] [[Services]] # This section specifies the services that should be run. They are # grouped by type, and the order of services within each group # determines the order in which the services will be run. prep_services = weewx.engine.StdTimeSynch data_services = , process_services = weewx.engine.StdConvert, weewx.engine.StdCalibrate, weewx.engine.StdQC, weewx.wxservices.StdWXCalculate archive_services = weewx.engine.StdArchive restful_services = weewx.restx.StdStationRegistry, weewx.restx.StdWunderground, weewx.restx.StdPWSweather, weewx.restx.StdCWOP, weewx.restx.StdWOW, weewx.restx.StdAWEKAS report_services = weewx.engine.StdPrint, weewx.engine.StdReport ---------------- #!/usr/bin/python # # weewx driver that reads data from MQTT subscription # # This program is free software: you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free Software # Foundation, either version 3 of the License, or any later version. # # This program is distributed in the hope that it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS # FOR A PARTICULAR PURPOSE. # # See http://www.gnu.org/licenses/ # # The units must be weewx.US: (this comment need to be updated) # degree_F, inHg, inch, inch_per_hour, mile_per_hour # # To use this driver, put this file in the weewx user directory, then make # the following changes to weewx.conf: # # [Station] # station_type = wxMesh # [wxMesh] # host = localhost # MQTT broker hostname # topic = weewx # topic, mqtt topics should have format "topic/label_map", ex: weather/humi # driver = user.wxMesh # # If the variables in the file have names different from those in weewx, then # create a mapping such as this: # # [wxMesh] # ... # [[label_map]] # temp = outTemp # humi = outHumidity # in_temp = inTemp # in_humid = inHumidity from __future__ import with_statement import syslog import time import paho.mqtt.client as mqtt import weewx.drivers DRIVER_VERSION = "0.1" def logmsg(dst, msg): syslog.syslog(dst, 'wxMesh: %s' % msg) def logdbg(msg): logmsg(syslog.LOG_DEBUG, msg) def loginf(msg): logmsg(syslog.LOG_INFO, msg) def logerr(msg): logmsg(syslog.LOG_ERR, msg) def _get_as_float(d, s): v = None if s in d: try: v = float(d[s]) except ValueError, e: logerr("cannot read value for '%s': %s" % (s, e)) return v def loader(config_dict, engine): return wxMesh(**config_dict['wxMesh']) class wxMesh(weewx.drivers.AbstractDevice): """weewx driver that reads data from a file""" def __init__(self, **stn_dict): # where to find the data file self.host = stn_dict.get('host', 'localhost') # subscribe to all sub-topic of the topic define in weewx.conf self.topic = stn_dict.get('topic', 'weather') + "/#" self.username = stn_dict.get('username', 'XXX') self.password = stn_dict.get('password', 'password') # how often to poll the weather data file, seconds self.poll_interval = float(stn_dict.get('poll_interval', 5.0)) # mapping from variable names to weewx names self.label_map = stn_dict.get('label_map', {}) loginf("host is %s" % self.host) loginf("topic is %s" % self.topic) loginf("polling interval is %s" % self.poll_interval) loginf('label map is %s' % self.label_map) self.payload = "Empty" self.receive_buffer = {} #self.payloadList = [payload] self.client = mqtt.Client(client_id="XXX", protocol=mqtt.MQTTv31) #self.client.on_connect = self.on_connect self.client.on_message = self.on_message self.client.username_pw_set(self.username, self.password) self.client.connect(self.host, 1883, 60) self.client.subscribe(self.topic, qos=0) # The callback for when a PUBLISH message is received from the server. def on_message(self, client, userdata, msg): self.payload = str(msg.payload) string_topic = str(msg.topic) key = string_topic.split('/')[-1] self.receive_buffer[key] = str(msg.payload) def closePort(self): self.client.disconnect() def genLoopPackets(self): self.client.loop_start() # enable to receive ('on_message') in background while True: time.sleep(self.poll_interval) # wait for some MQTT data to be published data = self.receive_buffer if data: # if data is not empty then prepare loop packet _packet = {'dateTime': int(time.time() + 0.5),'usUnits': weewx.METRIC} logdbg("dateTime %s" % _packet["dateTime"]) for vname in data: _packet[self.label_map.get(vname, vname)] = _get_as_float(data, vname) logdbg("buffer content for key: %s = %s" %(vname, data[vname])) yield _packet self.receive_buffer.clear() data.clear() @property def hardware_name(self): return "wxMesh" ---------------- pi@raspberrypi ~ $ mosquitto_sub -d -u Axxxx -P yyyyyy -t "weewx/#" Client mosqsub|1024-raspberryp sending CONNECT Client mosqsub|1024-raspberryp received CONNACK Client mosqsub|1024-raspberryp sending SUBSCRIBE (Mid: 1, Topic: weewx/#, QoS: 0) Client mosqsub|1024-raspberryp received SUBACK Subscribed (mid: 1): 0 Client mosqsub|1024-raspberryp sending PINGREQ Client mosqsub|1024-raspberryp received PINGRESP Client mosqsub|1024-raspberryp received PUBLISH (d0, q0, r0, m0, 'weewx/outHumidity', ... (5 bytes)) 48.80 Client mosqsub|1024-raspberryp received PUBLISH (d0, q0, r0, m0, 'weewx/outTemp', ... (5 bytes)) 18.90 Client mosqsub|1024-raspberryp received PUBLISH (d0, q0, r0, m0, 'weewx/outHumidity', ... (5 bytes)) 48.80 Client mosqsub|1024-raspberryp received PUBLISH (d0, q0, r0, m0, 'weewx/outTemp', ... (5 bytes)) 18.90 =================== Am Freitag, 13. April 2018 10:53:56 UTC+2 schrieb Neil Trimboy: > > Ralph/Bill > > Good job you pinged me privately as I was not subscribed to this thread > for some reason.....anyway.... > > My MQTT fork has been running as a service along with OWFS driver for > about 6 months with no big issues. Most of my problems are due to my > One-wire system hanging. > > I have a separate piece of code polling a yoctopuce > <https://www.yoctopuce.com/EN/products/usb-electrical-sensors/yocto-millivolt-rx>USB > > analogue to digital converter connected to my apogee SP-110 > <https://www.apogeeinstruments.com/sp-110-ss-self-powered-pyranometer/> > pyranometer, this posts to mosquito. > My mqtt service pulls in this data from mosquito > The rest of my system is one-wire running as a driver > > I have not dabbled with my MQTT code since I shoe stringed it together so > what I have is dirty and not really something I'd want to brag about, it > still needs work to be robust and kind to failures/exceptions. > The code is on github, the commit comment is "Messy but functional. No > exception catches" > > https://github.com/nellyt/ESP8266-MQTT-WEEWX/commit/26127c5da24fd814b5871640e2c896a395fee0a0 > > This was I recall a fork of bonjour81's fork of Bill's. > I recall I was using this because of the way i structured my MQTT data > > I think if you look at some of the standard include drivers/services from > Mwalll/Tom you will see that they support both driver or service. By > looking at a simple one of these you can understand how to convert your > code to a service....I think I based mine on the deltas between > service/driver in Mwalls OWFS plugin. There may well be more elegant coding > ways of achieving this now...i was running code from at least a year ago > when i did this. > Also relevant is a read of > http://www.weewx.com/docs/customizing.htm#service_engine > and http://www.weewx.com/docs/customizing.htm#Adding_2nd_source > > I would suggest looking at what I did and then incorporating similar > changes in your code (Bill), certainly dont fork mine as I think you are > multiple bug fixes ahead of me! > > I will keep track of thread but afraid I have little time available at > present. :( > > Neil > > On Thursday, 12 April 2018 06:14:28 UTC+12, Bill Morrow wrote: >> >> On Monday, 21 August 2017 08:25:59 UTC-3, Neil Trimboy wrote: >>> >>> Correct.... The standard OWFS running as a driver for my main inputs and >>> my modified version of yours that now runs as a service to obtain >>> additional data from some other sensors >> >> >> Neil, is your re-implementation as a service available anywhere? It seems >> like the preferred way to go. Most people have a driver to commercial >> weather instrument, and want to augment it with the odd datum from some >> other sensor. So it the MQTT subscriber driver (wxMesh is not a good name >> anymore) turns in to a service, that becomes possible. >> >> >
