Ok, progress with the locally hosted MQTT server.
I changed my DNS of mqtt.beldenserver.com to point to my firewall instead
of the cloud instance I had setup and added the port forwarding rules to
OPNSense. I can get the mqtt websocket real time updates to work internally
on any client by accessing the site by ip address of the weewx/mqtt device
with the following config:
*weewx.config*
[[MQTT]]
server_url = mqtt://usr:[email protected]:8883/
topic = weather
unit_system = US
binding = archive, loop
aggregation = aggregate
*skin.conf*
# MQTT Websockets defaults
mqtt_websockets_enabled = 1
mqtt_websockets_host = "mqtt.beldenserver.com"
mqtt_websockets_port = 8083
mqtt_websockets_ssl = 0
mqtt_websockets_topic = "weather/loop"
disconnect_live_website_visitor = 1800000
This says to me that I successfully have the mqtt.beldenserver.com DNS
setup and forwarded in my router. I still can't see it externally nor
locally with the secure website https://shakerweather.com. Perhaps I was
hoping the NGINX reverse proxy that is handling the SSL cert would just
return it all over SSL, so I think working on that setup is next.
I initially tried this config but no luck, and it broke everything locally
accessed by IP as well.
*weewx.config*
[[MQTT]]
server_url = mqtt://usr:[email protected]:8883/
topic = weather
unit_system = US
binding = archive, loop
aggregation = aggregate
[[[tls]]]
tls_version = tlsv1
ca_certs = /etc/ssl/certs/ca-certificates.crt
*skin.conf*
# MQTT Websockets defaults
mqtt_websockets_enabled = 1
mqtt_websockets_host = "mqtt.beldenserver.com"
mqtt_websockets_port = 8083
mqtt_websockets_ssl = 1
mqtt_websockets_topic = "weather/loop"
disconnect_live_website_visitor = 1800000
What next to check?
On Saturday, July 24, 2021 at 8:07:23 AM UTC-4 Matt Johnson wrote:
> Thanks Les. I think you have helped clear some things up. For the sake of
> clarity and getting to the end state I desire and your suggestion let's
> focus on locally hosting (my first setup) over the cloud broker. I have a
> robust homelab and really like to keep things in house.
>
> *think the first configuration, with the local mqtt broker isn’t going to
> work because mqtt_websockets_host is set to localhost, which will only
> resolve to the weewx/mqtt server when the web browser is running on that
> server. You need to set something here that will resolve to the weewx/mqtt
> server from any client that you want to get realtime updates. *
>
> This makes sense. I initially was on this path and changed the
> mqtt_websockets host to the ip address of the weewx/mqtt server. This did
> not work for other clients on the LAN which seemed strange given how I have
> lots of locally running services on a few different servers in my home lab
> and I access them all via device IP and port, with a few that are
> accessible externally via reverse proxy.
>
>
> *You need a DNS name that will resolve to your firewall and get
> port-forwarded (for port 8083) to the weewx/mqtt server,
> for mqtt_websockets_host. That should enable external access. *
>
> I can use my mqtt.beldenserver.com DNS name I have setup at cloudflare
> for this. I am running OPNSense for my firewall so I should be able to do
> anything. Right now I have a firewall rule setup to send port 80/443
> traffic to my NGINX Reverse Proxy where I have several DNS addresses
> pointing to different services on different servers. One of these takes
> shakerweather.com through 80/443 and through the reverse proxy and points
> to the weewx/mqtt server to return the webpage (which is working fine)
>
> So you are saying I just need to add a rule to forward external port 8083
> requests to the weewx/mqtt server IP and port?
>
> *And, if your firewall will do hairpinning, it should work internally as
> well. It may take some magic with forwarding/masquerading rules on the
> firewall to get hairpinning to work. (The alternative for internal access
> is to have an internal DNS server that resolves that hostname directly to
> the internal IP of the weewx/mqtt server for clients on the internal
> network.)*
>
> My OPNSense firewall should be able to do hairpinning. I read on that
> briefly as I have only heard of the term and not too familiar with it.
>
>
>
>
> On Saturday, July 24, 2021 at 2:24:32 AM UTC-4 ln77 wrote:
>
>> I think the first configuration, with the local mqtt broker isn’t going
>> to work because mqtt_websockets_host is set to localhost, which will only
>> resolve to the weewx/mqtt server when the web browser is running on that
>> server. You need to set something here that will resolve to the weewx/mqtt
>> server from any client that you want to get realtime updates.
>>
>> Not sure why the second config, with the cloud mqtt broker, isn’t
>> working. Are you sure the mqtt broker is configured for SSL on port 8883?
>> You might put “log_success = true” in the weewx [[MQTT]] config and see if
>> the log messages tell you anything useful.
>>
>> Or forget about the cloud server and go back to getting the first config
>> working. You need a DNS name that will resolve to your firewall and get
>> port-forwarded (for port 8083) to the weewx/mqtt server,
>> for mqtt_websockets_host. That should enable external access. And, if your
>> firewall will do hairpinning, it should work internally as well. It may
>> take some magic with forwarding/masquerading rules on the firewall to get
>> hairpinning to work. (The alternative for internal access is to have an
>> internal DNS server that resolves that hostname directly to the internal IP
>> of the weewx/mqtt server for clients on the internal network.)
>>
>> -Les
>>
>>
>>
>> On 23 Jul 2021, at 21:05, Matt Johnson <[email protected]> wrote:
>>
>> I've been trouble shooting getting the Belchertown skin MQTT Websocket
>> real time updates to work on my site shakerweather.com for a lot of this
>> week.
>>
>> I have WeeWx installed on a dedicated thin client on Ubuntu 20.04 LTS and
>> have Mosquitto and NGINX installed on the same machine. Running bare metal,
>> no docker or VMs here.
>>
>> External access to WeeWx website is handled via NGINX reverse proxy
>> manager with SSL certs on a different server via docker. Requests to
>> shakerweather.com are sent to the proxy server and then to the WeeWx
>> machine.
>>
>> With this setup, the site is served up fine internally and externally
>> with the updates at archive intervals every 5 minutes.
>>
>> I know that the weewx-mqtt extension is installed correctly as I have
>> been able to test it locally and get the websocket updates to work
>> perfectly with the following configs:
>>
>> *weewx.conf*
>> [[MQTT]]
>> server_url = mqtt://user:pw@localhost:8883/
>> topic = weather
>> unit_system = US
>> binding = archive, loop
>> aggregation = aggregate
>>
>> *skin.conf*
>> # MQTT Websockets defaults
>> mqtt_websockets_enabled = 1
>> mqtt_websockets_host = "localhost"
>> mqtt_websockets_port = 8083
>> mqtt_websockets_ssl = 0
>> mqtt_websockets_topic = "weather/loop"
>> disconnect_live_website_visitor = 1800000
>>
>> I am only able to see the the real time updates on the local machine only
>> with WeeWx and Mosquitto. If I try to access it by IP address elsewhere on
>> my LAN on other clients it does not connect and eventually fails. No luck
>> externally either - despite my NGINX Reverse Proxy Manger handling serving
>> the page and SSL certs the websocket real time updates don't pass through.
>> That was my original thought of how it would work.
>>
>> After much trial and error, and reading every thread imaginable on this
>> along with many messages and some correspondence with Pat O'Brien I decided
>> to go ahead and setup a Digital Ocean Ubuntu VM and install Mosquito there
>> to serve as a cloud broker. I followed Pat's instructions exactly as he
>> outlines in setting up the cloud broker:
>> https://obrienlabs.net/how-to-setup-your-own-mqtt-broker/
>>
>> I have the cloud MQTT broker installed correctly at Digital Ocean with
>> Let's Encrypt, and ran tests on it. Messages can be sent when
>> authenticated, ports are open, etc. However, I can get no further with the
>> websockets real time updates than "Connected. Waiting for data". If I
>> reboot the cloud MQTT broker I immediately get a disconnected message on
>> the website so it does appear to be connecting and waiting for data.
>> Somehow the data is simply not transferring from my WeeWx client to the
>> cloud MQTT broker at Digital Ocean. The other weird thing is if I try to
>> access shakerweather.com or the website by local IP address on the
>> machine that hosts WeeWx I always get a failed message, won't even connect
>> to the server. However, any other client on my LAN and external on WAN does
>> not have this issue.
>>
>> Here are my current configs:
>>
>> *weewx.conf*
>> [[MQTT]]
>> server_url = mqtt://user:[email protected]:8883/
>> topic = weather
>> unit_system = US
>> binding = archive, loop
>> aggregation = aggregate
>> [[[tls]]]
>> tls_version = tlsv1
>> ca_certs = /etc/ssl/certs/ca-certificates.crt
>>
>> *skin.conf*
>> # MQTT Websockets defaults
>> mqtt_websockets_enabled = 1
>> mqtt_websockets_host = "mqtt.beldenserver.com"
>> mqtt_websockets_port = 8083
>> mqtt_websockets_ssl = 1
>> mqtt_websockets_topic = "weather/loop"
>> disconnect_live_website_visitor = 1800000
>>
>> At this point, I have spent 20+ hours on this and hoping someone here can
>> point me in the right direction, it seems data is just not feeding the MQTT
>> topic. I'm fine with using Digital Ocean as a cloud MQTT server just to get
>> it up and running. My preferred state is eventually to selfhost it all.
>>
>> Thanks in advance for anything I may be overlooking, advice or possible
>> solutions.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "weewx-user" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/weewx-user/38183ff6-d5d5-4151-a1b3-93fd618aef5cn%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/weewx-user/38183ff6-d5d5-4151-a1b3-93fd618aef5cn%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>>
>>
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/7439fe67-f3f0-407f-8227-d497a5fe5152n%40googlegroups.com.
