Re: MQTT CONNECT parsing in Lua

2018-12-11 Thread Baptiste
>
> > At last AWS conference, I met with a engineer who was using HAProxy to
> > load-balance IoT devices through HAProxy using MQTT protocol and he was
> > complaining about the poor performance of the server with 10k of devices
> just
> > get reconnecting.
>
> Have you any chance to aks the engineer if your solution have better
> performance
> then his?
>

Not yet. it's the next step on my side.



> > He pointed SSL performance but also authentication (validation of
> username /
> > password).
>
> Do you have some more details about his SSL/TLS performance problem stuff?
>

Nope. Just that when HAProxy is configured in TCP mode, the SSL is done by
the server.
And when there is a network outage, the server gets up to 30K connections
coming back where the server has to manage TLS + authentication validation.
And the engineer said me this kills his server. We first looked at a
solution to limit the incoming rate, but then I realized this would not be
complicated enable SSLon HAProxy, then parse the CONNECT message (first
message sent when the client opens the connection), and check available
info and call a remote (web) service where I can validate the creds
provided by the client.

As I said above, next step is to engage the engineer and make him first
validate the code, second benchmark the whole solution.

We could even go further later and analyse the CONNACK message sent back by
the server to count errors / failure per client and why not block them at
the LB layer.

Baptiste


Re: MQTT CONNECT parsing in Lua

2018-12-11 Thread Aleksandar Lazic
Hi Baptiste.

Am 11.12.2018 um 03:29 schrieb Baptiste:
> Hi guys,
> 
> At last AWS conference, I met with a engineer who was using HAProxy to
> load-balance IoT devices through HAProxy using MQTT protocol and he was
> complaining about the poor performance of the server with 10k of devices just
> get reconnecting.

Have you any chance to aks the engineer if your solution have better performance
then his?

> He pointed SSL performance but also authentication (validation of username /
> password).

Do you have some more details about his SSL/TLS performance problem stuff?

> So I wrote a small MQTT library for HAProxy which allows parsing the MQTT
> CONNECT message, the very first one being sent by a client.
> The library allows the following:
> * validation of the message (through a converter)
> * fetch any field from the connect message (client id, username, password,
> etc...) for fun and profit (routing, persistence, rate or concurrent 
> connection
> enforcement, etc...)
> * write your own authentication validation module on top of HAProxy
> 
> The code is there, including some HAProxy configuration examples:
> https://github.com/bedis/haproxy_mqtt_lua
> 
> I hope this will be useful to some of you.
> I am planning to write in native C the converter and the fetch above.

In general , cool ;-)

> Baptiste

Regards
Aleks



MQTT CONNECT parsing in Lua

2018-12-10 Thread Baptiste
Hi guys,

At last AWS conference, I met with a engineer who was using HAProxy to
load-balance IoT devices through HAProxy using MQTT protocol and he was
complaining about the poor performance of the server with 10k of devices
just get reconnecting.
He pointed SSL performance but also authentication (validation of username
/ password).

So I wrote a small MQTT library for HAProxy which allows parsing the MQTT
CONNECT message, the very first one being sent by a client.
The library allows the following:
* validation of the message (through a converter)
* fetch any field from the connect message (client id, username, password,
etc...) for fun and profit (routing, persistence, rate or concurrent
connection enforcement, etc...)
* write your own authentication validation module on top of HAProxy

The code is there, including some HAProxy configuration examples:
https://github.com/bedis/haproxy_mqtt_lua

I hope this will be useful to some of you.
I am planning to write in native C the converter and the fetch above.

Baptiste