Re: Authentication/authorization implementation in haproxy, possibly with Redis

2025-01-13 Thread Christopher Faulet

Hi Lucas,

Le 10/01/2025 à 2:20 PM, Lucas Rolff a écrit :

I ended up trying to give SPOE a try, since it seemed like a possible contender 
for this use case.

I tried to give the LUA, Python, Rust and Go examples a go, and it seems like I 
need as many workers/threads on the agents, as I have concurrently processed 
requests in haproxy.


I cannot say for Go and Rust SPOA, but I've checked the LUA/Python SPOA 
(https://github.com/haproxy/spoa-server) and indeed, a worker is only able to 
process a message at a time. In fact, there is exactly one connection per worker 
that processes one message at a time. So it does not scale at all. It is more a 
proof of concept than a production ready agent.


Whatever. To make it work, you must set a maxconn on your SPOA servers to not 
open more connections than the agent can accept. In fact, I suggest to set a 
slightly lower maxconn. For instance a maxconn set to 5 for 8 or 10 workers. 
Because the maxconn is unfortunately not a so strict value than it should be.


Note that the SPOE was rewritten in 3.1. On older versions, the SPOP connexion 
management is far from perfect. So if you really want to go with the SPOE, I 
suggest to use the 3.1.




Is the expectation of SPOE, that the agents are fully event-driven as well, to 
be able to handle the AGENT-HELLO frames coming from haproxy, and then get them 
processed (within the processing timeout).
If that's the case, that obviously complicates the usage of SPOE a lot more


The SPOE does not expect anything special about the agents. But they must be 
sized to handle the load. In this case, it is obvious the spoa-server will never 
be able to handle many concurrent connections. And because it does not support 
the "pipelining" mode, there is no way to multiplex the requests.


--
Christopher Faulet





Re: Authentication/authorization implementation in haproxy, possibly with Redis

2025-01-11 Thread Tim Düsterhus

Hi

On 1/10/25 00:11, Lucas Rolff wrote:

- I can do it the ugly way, and proxy the request from haproxy to a small 
Golang app or similar, and then let the Go application talk to S3 backend 
directly


This is what I do with my haproxy-auth-request Lua script, it's working 
well for my use case: https://github.com/TimWolla/haproxy-auth-request


Best regards
Tim Düsterhus




Re: Authentication/authorization implementation in haproxy, possibly with Redis

2025-01-10 Thread Lucas Rolff
I ended up trying to give SPOE a try, since it seemed like a possible contender 
for this use case.

I tried to give the LUA, Python, Rust and Go examples a go, and it seems like I 
need as many workers/threads on the agents, as I have concurrently processed 
requests in haproxy.

Effectively I spawned the ps_lua.loa example:
===
./spoa -f ps_lua.lua
SPOA is listening on port 12345
SPOA worker 01 started
SPOA worker 02 started
SPOA worker 03 started
SPOA worker 04 started
SPOA worker 05 started
(string) "Load lua message processors"
(string) "Load lua message processors"
(string) "Load lua message processors"
(string) "Load lua message processors"
(string) "Load lua message processors"
===

If I do an "h2load -t 1 -n 10 -c 5 http://localhost/test.txt";, all requests 
succeed. The average response time is 262us:
===
finished in 3.83ms, 2610.97 req/s, 6.87MB/s
requests: 10 total, 10 started, 10 done, 10 succeeded, 0 failed, 0 errored, 0 
timeout
status codes: 10 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 26.96KB (27610) total, 7.07KB (7240) headers (space savings 9.16%), 
19.54KB (20010) data
 min max mean sd+/- sd
time for request:  307us   699us   470us   137us80.00%
time for connect: 1.08ms  2.34ms  1.77ms   480us60.00%
time to 1st byte: 3.00ms  3.14ms  3.05ms57us80.00%
req/s   : 554.62  586.64  571.08   11.4660.00%
===

Now, the issue starts if I increase clients from 5 to 10. Exactly half the 
requests will reach the 1000ms processing timeout configured.

I'm assuming this is because when haproxy tries to call the SPOE filter with 
the requests, the agent (being it the LUA, Python and Go examples) are not able 
to accept the request the millisecond haproxy tries to send it to the agent.
On spoe-agent, I have `timeout hello 5s` configured.

e.g. if I simply make the ps_lua.lua output "Hello World" in the 
spoa.register_message function, I get exactly 5 "hello world", instead of 10, 
and eventually after the 5 second timeout has been reached for `hello`, I get 
the various "failed to read/write" frames:
===
/spoa -f ps_lua.lua
SPOA is listening on port 12345
SPOA worker 01 started
SPOA worker 02 started
SPOA worker 03 started
SPOA worker 04 started
SPOA worker 05 started
(string) "Load lua message processors"
(string) "Load lua message processors"
(string) "Load lua message processors"
(string) "Load lua message processors"
(string) "Load lua message processors"
hello world
hello world
hello world
hello world
hello world


1736514434.980989 [01] Failed to read Haproxy NOTIFY frame
1736514434.980992 [03] Failed to read Haproxy NOTIFY frame
1736514434.980992 [04] Failed to read Haproxy NOTIFY frame
1736514434.980992 [02] Failed to read Haproxy NOTIFY frame
1736514434.980991 [05] Failed to read Haproxy NOTIFY frame
1736514434.981003 [01] Close the client socket because of I/O errors
1736514434.981004 [03] Close the client socket because of I/O errors
1736514434.981011 [04] Close the client socket because of I/O errors
1736514434.981015 [02] Close the client socket because of I/O errors
1736514434.981016 [05] Close the client socket because of I/O errors
1736514434.981045 [01] Failed to write Agent frame
1736514434.981046 [03] Failed to write Agent frame
1736514434.981049 [01] Close the client socket because of I/O errors
1736514434.981049 [03] Close the client socket because of I/O errors
1736514434.981058 [04] Failed to write Agent frame
1736514434.981061 [02] Failed to write Agent frame
1736514434.981061 [04] Close the client socket because of I/O errors
1736514434.981061 [05] Failed to write Agent frame
1736514434.981062 [02] Close the client socket because of I/O errors
1736514434.981066 [05] Close the client socket because of I/O errors
===

Is the expectation of SPOE, that the agents are fully event-driven as well, to 
be able to handle the AGENT-HELLO frames coming from haproxy, and then get them 
processed (within the processing timeout).
If that's the case, that obviously complicates the usage of SPOE a lot more

Best Regards,
Lucas Rolff


> On 10 Jan 2025, at 00:11, Lucas Rolff  wrote:
> 
> Hello,
> 
> I'm looking into possibilities to implement some slightly more complex logic 
> into haproxy that's being used when talking to origins.
> Looking through the documentation, I see I can obviously use Lua, which 
> offers great flexibility.
> 
> I'm looking to implement a few additional checks and balances before I 
> forward requests to the origin/backend for being processed.
> 
> Two of these things are:
> - validation of bearer tokens (API tokens) on haproxy level
> - S3 signing
> 
> Bearer tokens are stored in a Redis instance, and I can obviously write Lua 
> that's ran for a given http-request, but from what I understand, the way the 
> Lua is ran, means I obviously have to establish the whole Redis 
> instance/connection every time the code is executed, and I ca