Re: [weewx-user] MQTTSubscribe and paho mqtt heads up

[Graham Eddy]

i wrapped paho mqtt a few years ago, but not implementing whole interface.
i wote a bunch of apps on top of my wrapper, one being my own weewx mqtt 
service/driver, other apps are various mqtt producers and consumers

thanks for the heads-up on the mqtt interface change.
it prompted me to polish my wrapper a bit, with things i learnt over last few 
years.
one is to put json encoding/decoding at the heart of the wrapper - i expose 
python objects rather than mqtt messages, and the python objects are converted 
to json (strings) in the wrapper to/from the mqtt wire

i’ve yet to find a use for clean_session=false.
cheers
⊣GE⊢

> On 15 Mar 2024, at 10:57 am, bell...@gmail.com  wrote:
> 
> Just a quick update.
> I’m happy with this approach, but eventually V1 API support will be removed. 
> Ideally this will be long after V1 clients are gone. But, I’ve been burnt 
> once…. So, I’m implementing the approach below.
> I’m going to use the factory pattern. I already had a class ‘wrapping’ paho 
> mqtt. I will now have a base class that implements all of the logic. I will 
> have subclasses as necessary for the different versions of paho mqtt. These 
> subclasses will interact with paho mqtt and pass data to the base class to do 
> the real work. I’ll have a factory to provide the appropriate class to 
> interact with paho mqtt.
> Hopefully this will insulate a bit from breaking changes, or at least make it 
> easier to react to them.
> rich
> 
> On Sunday 11 February 2024 at 16:49:25 UTC-5 bell...@gmail.com wrote:
>> I've pretty much settled on this approach. Minimally tested
>> 
>> try:
>> callback_api_version = mqtt.CallbackAPIVersion.VERSION1
>> self.client = 
>> mqtt.Client(callback_api_version=callback_api_version,
>> client_id=mqtt_options['clientid'],
>> userdata=self.userdata,
>> 
>> clean_session=mqtt_options['clean_session'])
>> except AttributeError:
>> self.client = mqtt.Client(client_id=mqtt_options['clientid'],
>> userdata=self.userdata,
>> 
>> clean_session=mqtt_options['clean_session'])
>> 
>> rich
>> On Sunday 11 February 2024 at 15:21:58 UTC-5 matthew wall wrote:
>>> On Sunday, February 11, 2024 at 1:37:20 PM UTC-5 bell...@gmail.com <> wrote:
>>> It currently will not work with paho mqtt 2.0.0 or higher.
>>> 
>>> rich, i too am interested in what you learn, as i will probably have to 
>>> update weewx-mqtt as well. m
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to weewx-user+unsubscr...@googlegroups.com 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/weewx-user/8b3fba7b-dfae-4105-8138-7381e5697017n%40googlegroups.com
>  
> .

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/96563991-9D76-4812-B101-FF1FB2A29BA6%40geddy.au.

Re: [weewx-user] MQTTSubscribe and paho mqtt heads up

[bell...@gmail.com]

Just a quick update.
I’m happy with this approach, but eventually V1 API support will be 
removed. Ideally this will be long after V1 clients are gone. But, I’ve 
been burnt once…. So, I’m implementing the approach below.
I’m going to use the factory pattern. I already had a class ‘wrapping’ paho 
mqtt. I will now have a base class that implements all of the logic. I will 
have subclasses as necessary for the different versions of paho mqtt. These 
subclasses will interact with paho mqtt and pass data to the base class to 
do the real work. I’ll have a factory to provide the appropriate class to 
interact with paho mqtt.
Hopefully this will insulate a bit from breaking changes, or at least make 
it easier to react to them.
rich

On Sunday 11 February 2024 at 16:49:25 UTC-5 bell...@gmail.com wrote:

> I've pretty much settled on this approach. Minimally tested
>
> try:
> callback_api_version = mqtt.CallbackAPIVersion.VERSION1
> self.client = 
> mqtt.Client(callback_api_version=callback_api_version,
> client_id=mqtt_options['clientid'],
> userdata=self.userdata,
> 
> clean_session=mqtt_options['clean_session'])
> except AttributeError:
> self.client = mqtt.Client(client_id=mqtt_options['clientid'],
> userdata=self.userdata,
> 
> clean_session=mqtt_options['clean_session'])
>
> rich
> On Sunday 11 February 2024 at 15:21:58 UTC-5 matthew wall wrote:
>
>> On Sunday, February 11, 2024 at 1:37:20 PM UTC-5 bell...@gmail.com wrote:
>>
>> It currently will not work with paho mqtt 2.0.0 or higher.
>>
>>
>> rich, i too am interested in what you learn, as i will probably have to 
>> update weewx-mqtt as well. m
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/8b3fba7b-dfae-4105-8138-7381e5697017n%40googlegroups.com.

Re: [weewx-user] MQTTSubscribe and paho mqtt heads up

[Graham Knights]

I just migrated my install to a new machine and was pulling my hair out 
trying to figure this out until I saw this today, thanks @gary for figuring 
this out.  I've edited the weewx wiki to direct new setups to install the 
v1.6.1 paho-mqtt until the original mqtt extension is updated.  

On Sunday, February 11, 2024 at 5:04:01 PM UTC-8 bell...@gmail.com wrote:

> FWIW 
> Successfully ran integration tests on Ubuntu 22, python 3.12 with 
> paho-mqtt v2.0.0 and v1.6.1.
>
> On Sunday 11 February 2024 at 16:49:25 UTC-5 bell...@gmail.com wrote:
>
>> I've pretty much settled on this approach. Minimally tested
>>
>> try:
>> callback_api_version = mqtt.CallbackAPIVersion.VERSION1
>> self.client = 
>> mqtt.Client(callback_api_version=callback_api_version,
>> client_id=mqtt_options['clientid'],
>> userdata=self.userdata,
>> 
>> clean_session=mqtt_options['clean_session'])
>> except AttributeError:
>> self.client = mqtt.Client(client_id=mqtt_options['clientid'],
>> userdata=self.userdata,
>> 
>> clean_session=mqtt_options['clean_session'])
>>
>> rich
>> On Sunday 11 February 2024 at 15:21:58 UTC-5 matthew wall wrote:
>>
>>> On Sunday, February 11, 2024 at 1:37:20 PM UTC-5 bell...@gmail.com 
>>> wrote:
>>>
>>> It currently will not work with paho mqtt 2.0.0 or higher.
>>>
>>>
>>> rich, i too am interested in what you learn, as i will probably have to 
>>> update weewx-mqtt as well. m
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/12548e6a-ba98-4d59-9b57-8ed9bb1f860en%40googlegroups.com.

Re: [weewx-user] MQTTSubscribe and paho mqtt heads up

[bell...@gmail.com]

FWIW 
Successfully ran integration tests on Ubuntu 22, python 3.12 with paho-mqtt 
v2.0.0 and v1.6.1.

On Sunday 11 February 2024 at 16:49:25 UTC-5 bell...@gmail.com wrote:

> I've pretty much settled on this approach. Minimally tested
>
> try:
> callback_api_version = mqtt.CallbackAPIVersion.VERSION1
> self.client = 
> mqtt.Client(callback_api_version=callback_api_version,
> client_id=mqtt_options['clientid'],
> userdata=self.userdata,
> 
> clean_session=mqtt_options['clean_session'])
> except AttributeError:
> self.client = mqtt.Client(client_id=mqtt_options['clientid'],
> userdata=self.userdata,
> 
> clean_session=mqtt_options['clean_session'])
>
> rich
> On Sunday 11 February 2024 at 15:21:58 UTC-5 matthew wall wrote:
>
>> On Sunday, February 11, 2024 at 1:37:20 PM UTC-5 bell...@gmail.com wrote:
>>
>> It currently will not work with paho mqtt 2.0.0 or higher.
>>
>>
>> rich, i too am interested in what you learn, as i will probably have to 
>> update weewx-mqtt as well. m
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/9a43f501-7476-4065-be9d-05e1ba6c2a03n%40googlegroups.com.

Re: [weewx-user] MQTTSubscribe and paho mqtt heads up

[bell...@gmail.com]

I've pretty much settled on this approach. Minimally tested

try:
callback_api_version = mqtt.CallbackAPIVersion.VERSION1
self.client = 
mqtt.Client(callback_api_version=callback_api_version,
client_id=mqtt_options['clientid'],
userdata=self.userdata,

clean_session=mqtt_options['clean_session'])
except AttributeError:
self.client = mqtt.Client(client_id=mqtt_options['clientid'],
userdata=self.userdata,

clean_session=mqtt_options['clean_session'])

rich
On Sunday 11 February 2024 at 15:21:58 UTC-5 matthew wall wrote:

> On Sunday, February 11, 2024 at 1:37:20 PM UTC-5 bell...@gmail.com wrote:
>
> It currently will not work with paho mqtt 2.0.0 or higher.
>
>
> rich, i too am interested in what you learn, as i will probably have to 
> update weewx-mqtt as well. m
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/90111f2c-4885-4ef9-bf61-52dd0abc02d2n%40googlegroups.com.

Re: [weewx-user] MQTTSubscribe and paho mqtt heads up

[matthew wall]

On Sunday, February 11, 2024 at 1:37:20 PM UTC-5 bell...@gmail.com wrote:

It currently will not work with paho mqtt 2.0.0 or higher.


rich, i too am interested in what you learn, as i will probably have to 
update weewx-mqtt as well. m

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/c1bd436a-3d01-4ef8-bb53-74ea72bc7646n%40googlegroups.com.

Re: [weewx-user] MQTTSubscribe and paho mqtt heads up

[bell...@gmail.com]

Because (1) its inevitable and (2) I’ll get tired of typing the wiki url 
explaining this.
I just hope they wait long enough (that 1.x is gone) before removing the 
other signatures that have changed…

On Sunday 11 February 2024 at 14:23:07 UTC-5 vince wrote:

> Why not stick with 1.6.1 indefinitely and the matching version of the 
> extension and not sweat it ?
>
> On Sunday, February 11, 2024 at 11:12:32 AM UTC-8 Greg Troxel wrote:
>
>> "bell...@gmail.com"  writes: 
>>
>> > It currently will not work with paho mqtt 2.0.0 or higher. 
>>
>> Sorry, I understood that. I have looked briefly at the migration which 
>> says "change your code because a function signature changed", and have I 
>> have been holding off because I'll need to fully absorb it, and then 
>> update rtl_433_mqtt_relay.pt (for which I am somewhat the maintainer) 
>> and a program (that I haven't published yet) that polls ups-nut and 
>> reports status over mqtt. 
>>
>> What I meant, is, do you expect that when you figure out how to deal 
>> with the incompatible changes, you'll end up with a MQTTSubscribe that 
>> can function with either, or if we'll need to match versions. 
>>
>> If there's a way to do it (try/catch and figure out the signature?) then 
>> presumaly the MQTT publish extension, rtl_433_mqtt_relay and everything 
>> else can use that technique. 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/e1c26ae9-7e8a-4a89-bb81-d74e2a510306n%40googlegroups.com.

Re: [weewx-user] MQTTSubscribe and paho mqtt heads up

['michael.k...@gmx.at' via weewx-user]

It did a fresh pip install on bookworm and ran into this for weewx-mqtt. It 
will be an issue people ask about in the near future, that's for sure.

vince schrieb am Sonntag, 11. Februar 2024 um 20:32:48 UTC+1:

> No reason to upgrade if it works for you already and if there are no 
> security risks since then.
>
> To list all versions available "pip list paho-mqtt=="
> To install a specified version "pip install paho-mqtt==1.6.1"
>
> Or for apt...
>
> List all versions "apt-cache policy python3-paho-mqtt"
> Install one specified version "apt-get install python3-paho-mqtt=1.5.1-1"
>
>
> Note that for deb12 (raspi) the latest package is 1.5.1 which is a year 
> older than 1.6.1, so they're not upgrading either.
>
>
> On Sunday, February 11, 2024 at 11:27:54 AM UTC-8 Greg Troxel wrote:
>
>> vince  writes: 
>>
>> > Why not stick with 1.6.1 indefinitely and the matching version of the 
>> > extension and not sweat it ? 
>>
>> Because bugs are not going to be fixed, and packaging systems are going 
>> to have 2.0.0, and in general trying to stay in the past causes too much 
>> trouble eventually. The same reason weewx had to cope with python 
>> instead of just telling people to use 2.7 :) 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/1305fec1-cdb8-4ebd-8c67-0caede21e73dn%40googlegroups.com.

Re: [weewx-user] MQTTSubscribe and paho mqtt heads up

[vince]

No reason to upgrade if it works for you already and if there are no 
security risks since then.

To list all versions available "pip list paho-mqtt=="
To install a specified version "pip install paho-mqtt==1.6.1"

Or for apt...

List all versions "apt-cache policy python3-paho-mqtt"
Install one specified version "apt-get install python3-paho-mqtt=1.5.1-1"


Note that for deb12 (raspi) the latest package is 1.5.1 which is a year 
older than 1.6.1, so they're not upgrading either.


On Sunday, February 11, 2024 at 11:27:54 AM UTC-8 Greg Troxel wrote:

> vince  writes:
>
> > Why not stick with 1.6.1 indefinitely and the matching version of the 
> > extension and not sweat it ?
>
> Because bugs are not going to be fixed, and packaging systems are going
> to have 2.0.0, and in general trying to stay in the past causes too much
> trouble eventually. The same reason weewx had to cope with python
> instead of just telling people to use 2.7 :)
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/af716446-6b2b-485b-859f-b6d81bd1a0b1n%40googlegroups.com.

Re: [weewx-user] MQTTSubscribe and paho mqtt heads up

[Greg Troxel]

vince  writes:

> Why not stick with 1.6.1 indefinitely and the matching version of the 
> extension and not sweat it ?

Because bugs are not going to be fixed, and packaging systems are going
to have 2.0.0, and in general trying to stay in the past causes too much
trouble eventually.  The same reason weewx had to cope with python
instead of just telling people to use 2.7 :)

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/rmieddi243w.fsf%40s1.lexort.com.

Re: [weewx-user] MQTTSubscribe and paho mqtt heads up

[bell...@gmail.com]

My plan is to try the 2.0 signature. I think it will throw a TypeError (but 
what ever it throws) and catch that and use the 1.6 signature.
Unless a Python expert has a better idea.
The alternative is use an if to check the version and call the appropriate 
signature.
Make sure to a read the upgrade notes. You will want to pass in the value 
to say to use the old signatures for other methods. Looks like a bunch 
changed.
I hope to start working on it soon. The biggest pain will be updating my 
CI/CD to (1) deal with multiple versions and (2) decide how much to test 
against each version… 
rich

On Sunday 11 February 2024 at 14:12:32 UTC-5 Greg Troxel wrote:

> "bell...@gmail.com"  writes:
>
> > It currently will not work with paho mqtt 2.0.0 or higher.
>
> Sorry, I understood that. I have looked briefly at the migration which
> says "change your code because a function signature changed", and have I
> have been holding off because I'll need to fully absorb it, and then
> update rtl_433_mqtt_relay.pt (for which I am somewhat the maintainer)
> and a program (that I haven't published yet) that polls ups-nut and
> reports status over mqtt.
>
> What I meant, is, do you expect that when you figure out how to deal
> with the incompatible changes, you'll end up with a MQTTSubscribe that
> can function with either, or if we'll need to match versions.
>
> If there's a way to do it (try/catch and figure out the signature?) then
> presumaly the MQTT publish extension, rtl_433_mqtt_relay and everything
> else can use that technique.
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/6cfef91c-5d81-42ec-be12-ce14a25be384n%40googlegroups.com.

Re: [weewx-user] MQTTSubscribe and paho mqtt heads up

[vince]

Why not stick with 1.6.1 indefinitely and the matching version of the 
extension and not sweat it ?

On Sunday, February 11, 2024 at 11:12:32 AM UTC-8 Greg Troxel wrote:

> "bell...@gmail.com"  writes:
>
> > It currently will not work with paho mqtt 2.0.0 or higher.
>
> Sorry, I understood that. I have looked briefly at the migration which
> says "change your code because a function signature changed", and have I
> have been holding off because I'll need to fully absorb it, and then
> update rtl_433_mqtt_relay.pt (for which I am somewhat the maintainer)
> and a program (that I haven't published yet) that polls ups-nut and
> reports status over mqtt.
>
> What I meant, is, do you expect that when you figure out how to deal
> with the incompatible changes, you'll end up with a MQTTSubscribe that
> can function with either, or if we'll need to match versions.
>
> If there's a way to do it (try/catch and figure out the signature?) then
> presumaly the MQTT publish extension, rtl_433_mqtt_relay and everything
> else can use that technique.
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/faa7dc2d-c59c-4f98-8ce1-b657a6fc8d62n%40googlegroups.com.

Re: [weewx-user] MQTTSubscribe and paho mqtt heads up

[Greg Troxel]

"bell...@gmail.com"  writes:

> It currently will not work with paho mqtt 2.0.0 or higher.

Sorry, I understood that.  I have looked briefly at the migration which
says "change your code because a function signature changed", and have I
have been holding off because I'll need to fully absorb it, and then
update rtl_433_mqtt_relay.pt (for which I am somewhat the maintainer)
and a program (that I haven't published yet) that polls ups-nut and
reports status over mqtt.

What I meant, is, do you expect that when you figure out how to deal
with the incompatible changes, you'll end up with a MQTTSubscribe that
can function with either, or if we'll need to match versions.

If there's a way to do it (try/catch and figure out the signature?) then
presumaly the MQTT publish extension, rtl_433_mqtt_relay and everything
else can use that technique.

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/rmiil2u24ti.fsf%40s1.lexort.com.

Re: [weewx-user] MQTTSubscribe and paho mqtt heads up

[bell...@gmail.com]

It currently will not work with paho mqtt 2.0.0 or higher.

On Sunday 11 February 2024 at 13:24:17 UTC-5 Greg Troxel wrote:

> "bell...@gmail.com"  writes:
>
> > There is new release of paho mqtt, 2.0.0. This has a breaking change for 
> > all clients using this library. I'll let you know when MQTTSubscribe 
> > supports it.
>
> Do you expect that MQTTSubscribe can work ok if either 1.6.x or 2.0.0
> is installed?
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/98d40882-21b1-4298-95d7-113ae4b22f43n%40googlegroups.com.

Re: [weewx-user] MQTTSubscribe and paho mqtt heads up

[Greg Troxel]

"bell...@gmail.com"  writes:

> There is new release of paho mqtt, 2.0.0. This has a breaking change for 
> all clients using this library. I'll let you know when MQTTSubscribe 
> supports it.

Do you expect that MQTTSubscribe can work ok if either  1.6.x or 2.0.0
is installed?

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/rmizfw6271v.fsf%40s1.lexort.com.

[weewx-user] MQTTSubscribe and paho mqtt heads up

[bell...@gmail.com]

There is new release of paho mqtt, 2.0.0. This has a breaking change for 
all clients using this library. I'll let you know when MQTTSubscribe 
supports it.
Sigh, backwards compatibility is overrated.
rich

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/e6edeb57-d4ac-49ba-acad-875642e14d46n%40googlegroups.com.