Hi, thanks for your answer! Yes, this is a very good insight and suggestion. Before, I was using Apache Pulsar as a broker, but somehow the people working on the backend have decided to move to a REST service.
Anyhow, I think my issue is strictly related to logging.. if you have any suggestions on how to solve it, please let me know! :) Thank you very much, Saverio On Saturday, 12 March 2022 at 14:22:25 UTC+1 [email protected] wrote: > Saverio: > > Reading your post, have you considered using mqtt for each station node > publishing it's archive data to a single broker? > > I wonder if you could do the following to achieve the same goal: > > 1. Set each station node archive interval to 1 minute. > 2. Install weewx mqtt extension on each node > 3. Configure each node to publish to a different topic, for example > weather-n1, n2,etc. > 4. Configure the mqtt driver to just publish archive data. > 5. Setup a mosquito mqtt broker on the same network to collect all the > data. It can be setup in a container. > > This assumes all of your nodes are on the same network, although you could > setup the mqtt broker in a cloud instance if the nodes are not on the same > network. > > That way you will get the full archive record published from each station. > You would just need to focus on consuming the published data from the > broker. At least all the data will be in a single place. > > Just a thought. > > On Sat, Mar 12, 2022, 6:50 AM Saverio Guzzo <[email protected]> wrote: > >> PS. One detail that I forgot to mention: Balena uses docker images, for >> running the program I'm using >> balenalib/raspberrypi4-64-python:3.8-stretch-run >> <https://hub.docker.com/layers/balenalib/raspberrypi4-64-python/3.8-stretch-run/images/sha256-30dc1cfee0d09004a70c3663aaae763df6ac6c2167d2e8cd9cbb929312ebe960?context=explore> >> >> On Saturday, 12 March 2022 at 11:17:13 UTC+1 Saverio Guzzo wrote: >> >>> Hey community, >>> >>> I have a fleet of Davis weather stations connected to as many >>> RaspberryPis, on which I am running the vantage drivers as a standalone >>> program on docker container, deployed using Balena >>> <https://www.balena.io/open/>. >>> I'd like my program to send data every minute to a REST API and I've >>> been looking into some way to get archive data from a in loop, but I'm not >>> sure I'm doing it the right way. >>> >>> What I did was defining in the driver's main method something like: >>> >>> *while True:* >>> * since_ts = datetime.datetime.timestamp(datetime.datetime.now() - >>> datetime.timedelta(minutes=1))* >>> * for packet in vantage.genDavisArchiveRecords(since_ts):* >>> * try:* >>> * new_packet = weewx.units.to_METRICWX(packet)* >>> * davispusher.send_message(payload=new_packet)* >>> * log.debug(davispusher.payload)* >>> * except Exception as e:* >>> * log.debug("Found an exception: %s" % e)* >>> * time.sleep(1.2)* >>> * continue* >>> * time.sleep(60)* >>> >>> Where davispusher is an instance of a simple class that is needed to add >>> some values to the archive record and send the message and is defined as: >>> *class DavisPusher:* >>> * def __init__(self, host='https://mywebsite.somewhere >>> <https://mywebsite.somewhere>, port=8080, endpoint='davis'):* >>> * self._endpoint = "{host}:{port}/{endpoint}".format(host=host, * >>> * endpoint=endpoint,* >>> * port=port)* >>> * self.payload = {}* >>> >>> * def format_message(self,* >>> * payload: dict,* >>> * sensor_id: str = environ['SENSOR_ID'],* >>> * latitude: str = environ['LATITUDE'],* >>> * longitude: str = environ['LONGITUDE'],* >>> * altitude: str = environ['ALTITUDE'],* >>> * ) -> dict:* >>> * """Formats message for backend."""* >>> >>> * payload_copy = payload.copy()* >>> * try:* >>> * self.payload = payload_copy* >>> * self.payload['sensor_id'] = sensor_id* >>> * self.payload['altitude'] = float(altitude)* >>> * self.payload['longitude'] = float(longitude)* >>> * self.payload['latitude'] = float(latitude)* >>> >>> * except Exception:* >>> * self.payload = {}* >>> * def send_message(self, payload):* >>> * '''formats and sends message to backend'''* >>> * self.format_message(payload)* >>> * resp = requests.post <http://requests.post>(self._endpoint, data = >>> json.dumps(self.payload), allow_redirects = True)* >>> * log.debug("got HTTP statuscode %s", resp.status_code)* >>> >>> However, my logs are full of logging errors ("Bad file descriptor", >>> "File "/usr/local/lib/python3.8/logging/handlers.py" No such file or >>> directory") >>> I guess those are errors due to logging configuration, but I'm not sure >>> how to address them. I created a GitHub Gist >>> <https://gist.github.com/saveriogzz/c624c03e63a3be3d66daf57b25337b7e> >>> in order not to pollute this post! >>> >>> Also, what I would like to achieve is the possibility to configure the >>> various consoles (set time, set coordinates) at startup using environment >>> variables. How could I do this using the VantageConfigurator class? >>> >>> Thanks a lot in advance for your help, I'm kind of alone in my work and >>> I'd really appreciate your help. >>> >>> Friendly greetings, >>> Saverio >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "weewx-user" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/weewx-user/55d40e74-aa67-40c0-bcb6-8b8a03c9a665n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/weewx-user/55d40e74-aa67-40c0-bcb6-8b8a03c9a665n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/ff9d13fe-1e6f-4d18-be5f-1acf802aca17n%40googlegroups.com.
