Regardless of the errors. Do you think that my code is acceptable to get archive data every minute?
Thanks, Saverio On Saturday, 12 March 2022 at 19:30:55 UTC+1 Saverio Guzzo wrote: > Thanks for you answer Vince. > True, the message is clear! However, the file > "/usr/local/lib/python3.8/logging/handlers.py" > is there as expected! That's why I'm surprised by the logging errors. > It's already a few weeks I'm chasing this error :( > > If you guys have additional suggestions, please let me know > > On Saturday, 12 March 2022 at 18:48:35 UTC+1 vince wrote: > >> "Bad file descriptor", "File >> "/usr/local/lib/python3.8/logging/handlers.py" No such file or directory" >> >> Seems pretty clear to me. Your docker image doesn't appear to have the >> logging stuff built into it at the location python expects. >> >> Poke around in a shell to see what's in there with something like "docker >> run --rm -it yourimagename bash" >> >> >> On Saturday, March 12, 2022 at 9:41:38 AM UTC-8 Saverio Guzzo wrote: >> >>> Hi, thanks for your answer! >>> Yes, this is a very good insight and suggestion. Before, I was using >>> Apache Pulsar as a broker, but somehow the people working on the backend >>> have decided to move to a REST service. >>> >>> Anyhow, I think my issue is strictly related to logging.. if you have >>> any suggestions on how to solve it, please let me know! :) >>> >>> Thank you very much, >>> Saverio >>> >>> On Saturday, 12 March 2022 at 14:22:25 UTC+1 [email protected] >>> wrote: >>> >>>> Saverio: >>>> >>>> Reading your post, have you considered using mqtt for each station node >>>> publishing it's archive data to a single broker? >>>> >>>> I wonder if you could do the following to achieve the same goal: >>>> >>>> 1. Set each station node archive interval to 1 minute. >>>> 2. Install weewx mqtt extension on each node >>>> 3. Configure each node to publish to a different topic, for example >>>> weather-n1, n2,etc. >>>> 4. Configure the mqtt driver to just publish archive data. >>>> 5. Setup a mosquito mqtt broker on the same network to collect all the >>>> data. It can be setup in a container. >>>> >>>> This assumes all of your nodes are on the same network, although you >>>> could setup the mqtt broker in a cloud instance if the nodes are not on >>>> the >>>> same network. >>>> >>>> That way you will get the full archive record published from each >>>> station. You would just need to focus on consuming the published data from >>>> the broker. At least all the data will be in a single place. >>>> >>>> Just a thought. >>>> >>>> On Sat, Mar 12, 2022, 6:50 AM Saverio Guzzo <[email protected]> wrote: >>>> >>>>> PS. One detail that I forgot to mention: Balena uses docker images, >>>>> for running the program I'm using >>>>> balenalib/raspberrypi4-64-python:3.8-stretch-run >>>>> <https://hub.docker.com/layers/balenalib/raspberrypi4-64-python/3.8-stretch-run/images/sha256-30dc1cfee0d09004a70c3663aaae763df6ac6c2167d2e8cd9cbb929312ebe960?context=explore> >>>>> >>>>> On Saturday, 12 March 2022 at 11:17:13 UTC+1 Saverio Guzzo wrote: >>>>> >>>>>> Hey community, >>>>>> >>>>>> I have a fleet of Davis weather stations connected to as many >>>>>> RaspberryPis, on which I am running the vantage drivers as a standalone >>>>>> program on docker container, deployed using Balena >>>>>> <https://www.balena.io/open/>. >>>>>> I'd like my program to send data every minute to a REST API and I've >>>>>> been looking into some way to get archive data from a in loop, but I'm >>>>>> not >>>>>> sure I'm doing it the right way. >>>>>> >>>>>> What I did was defining in the driver's main method something like: >>>>>> >>>>>> *while True:* >>>>>> * since_ts = datetime.datetime.timestamp(datetime.datetime.now() - >>>>>> datetime.timedelta(minutes=1))* >>>>>> * for packet in vantage.genDavisArchiveRecords(since_ts):* >>>>>> * try:* >>>>>> * new_packet = weewx.units.to_METRICWX(packet)* >>>>>> * davispusher.send_message(payload=new_packet)* >>>>>> * log.debug(davispusher.payload)* >>>>>> * except Exception as e:* >>>>>> * log.debug("Found an exception: %s" % e)* >>>>>> * time.sleep(1.2)* >>>>>> * continue* >>>>>> * time.sleep(60)* >>>>>> >>>>>> Where davispusher is an instance of a simple class that is needed to >>>>>> add some values to the archive record and send the message and is >>>>>> defined >>>>>> as: >>>>>> *class DavisPusher:* >>>>>> * def __init__(self, host='https://mywebsite.somewhere >>>>>> <https://mywebsite.somewhere>, port=8080, endpoint='davis'):* >>>>>> * self._endpoint = "{host}:{port}/{endpoint}".format(host=host, * >>>>>> * endpoint=endpoint,* >>>>>> * port=port)* >>>>>> * self.payload = {}* >>>>>> >>>>>> * def format_message(self,* >>>>>> * payload: dict,* >>>>>> * sensor_id: str = environ['SENSOR_ID'],* >>>>>> * latitude: str = environ['LATITUDE'],* >>>>>> * longitude: str = environ['LONGITUDE'],* >>>>>> * altitude: str = environ['ALTITUDE'],* >>>>>> * ) -> dict:* >>>>>> * """Formats message for backend."""* >>>>>> >>>>>> * payload_copy = payload.copy()* >>>>>> * try:* >>>>>> * self.payload = payload_copy* >>>>>> * self.payload['sensor_id'] = sensor_id* >>>>>> * self.payload['altitude'] = float(altitude)* >>>>>> * self.payload['longitude'] = float(longitude)* >>>>>> * self.payload['latitude'] = float(latitude)* >>>>>> >>>>>> * except Exception:* >>>>>> * self.payload = {}* >>>>>> * def send_message(self, payload):* >>>>>> * '''formats and sends message to backend'''* >>>>>> * self.format_message(payload)* >>>>>> * resp = requests.post <http://requests.post>(self._endpoint, data = >>>>>> json.dumps(self.payload), allow_redirects = True)* >>>>>> * log.debug("got HTTP statuscode %s", resp.status_code)* >>>>>> >>>>>> However, my logs are full of logging errors ("Bad file descriptor", >>>>>> "File "/usr/local/lib/python3.8/logging/handlers.py" No such file or >>>>>> directory") >>>>>> I guess those are errors due to logging configuration, but I'm not >>>>>> sure how to address them. I created a GitHub Gist >>>>>> <https://gist.github.com/saveriogzz/c624c03e63a3be3d66daf57b25337b7e> >>>>>> in order not to pollute this post! >>>>>> >>>>>> Also, what I would like to achieve is the possibility to configure >>>>>> the various consoles (set time, set coordinates) at startup using >>>>>> environment variables. How could I do this using the VantageConfigurator >>>>>> class? >>>>>> >>>>>> Thanks a lot in advance for your help, I'm kind of alone in my work >>>>>> and I'd really appreciate your help. >>>>>> >>>>>> Friendly greetings, >>>>>> Saverio >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "weewx-user" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/weewx-user/55d40e74-aa67-40c0-bcb6-8b8a03c9a665n%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/weewx-user/55d40e74-aa67-40c0-bcb6-8b8a03c9a665n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/0e5378ac-6477-4e76-b67d-8c8247258865n%40googlegroups.com.
