Re: Lua + shared memory segment

[Thierry Fournier]

Hi,

You can’t use socket in sample-fetches for HAProxy internal reason.
If you want to take decision using socket+redis, you must set your
Redis request in “action”. according with the redis response, your
action set the result in shared context, and a sample fetche can
return the result. Something like that:


   http-request lua.do_my_redis_stuff
   use_backend bck_a if { lua.get_redis_result bck_a }
   use_backend bck_b if { lua.get_redis_result bck_b }

   core.register_action(“do_my_redis_stuff”, fucntion (txn)
  -- perform redis request
  network = core.tcp()
  ...
  -- get response
  response = network:read()
  -- store in context
  ctx = {}
  ctx[‘message’] = response
  txn:set_ctx(ctx)
   end)

   core.register_fetches("get_redis_result", function(txn)
  a = txn:get_ctx()
  return a[‘message']
   end)


Thierry


> On 7 Nov 2019, at 11:56, Kalantari  wrote:
> 
> 
> 
> 
> Hi Thierry,
> 
> I know this an old thread but I'm having a similar issue where HA Proxy 
> doesn't allow me to use Redis (not allowed to use socket in fetches). My 
> scenario is as below:
> 
> one frontend and multiple backends (proxies). (the frontend is just one IP 
> and always sends requests to the same url) 
> I use Lua script SmartRouting to:
> A. Look at the body of the body of the message and decide which back end to 
> use
> B. If the backend servers being used are too busy (active sessions > 
> threshold) then it will choose to send the requests to the offlineAgent
> 
> frontend http-in
> bind *:80 
> use_backend %[lua.SmartRouting(txn)]
> default_backend OnlineChargers
>  
> backend Onlineagent
> server testServer1 ${Server_1_IP}:${Server_1_Port} check
> backend ServersForRequestTypeX
> server testServer1 ${Server_2_IP}:${Server_2_Port} check
> backend offlineAgents
> server testServer1 ${Server_3_IP}:${Server_3_Port} check
> http-response lua.add_DegradedSession
> 
> Straight forward up to this point and no need for Redis, however if a message 
> is sent to offlineAgent proxy, then I want all the rest of the requests to be 
> sent to the offline agent. (each message can have a sessionID 
> inside the payload). I tried to add the sessionID for the messages inside my 
> SmartRouting to Redis as it was explained in your blogpost but HAProxy throws 
> an error and doesn't allow use of a socket in sample fetch. 
> below is my Lua script:
> 
> -- if number of active sessions goes higher than below then degraded mode is 
> detected
> DEGRADED_THRESHOLD = 10
> -- global object which will be used for routing of requests based on request 
> type xml tag
> -- if the request type is not found then HA proxy's default backend will be 
> used
> routingTable = {
> [""] = "Onlineagent",
> [""] = "ServersForRequestTypeX"
> }
> 
> local function SmartRouting(txn)
> --print_r(txn)
> local payload = txn.sf:req_body()
> -- extract request tag (final occurence of  local requestTag = string.match(payload, " local selectedBackend = routingTable[requestTag]
> -- check if there's enough capacity (degraded mode)
> -- this check is only neccessary if the
> if (selectedBackend == "Onlineagent") then
> local sessionCount = getTotalSessions()
> if (sessionCount > DEGRADED_THRESHOLD) then
> core.Debug("shit is hiting the fan! Sending requests to degraded 
> agent")
> --TODO:  Add the sessionID to Redis and check for future requests 
> if it's in Redis then redirect to Offline Agent
> return "offlineAgents"
> end
> end
> return selectedBackend
> end
> 
> function getTotalSessions()
> local total_sessions = 0
> for _, backend in pairs(core.backends) do
> if backend.name == "Onlineagent" then
> for _, server in pairs(backend.servers) do
> -- Get server's stats
> local stats = server:get_stats()
> 
> -- Get the backend's total number of current sessions
> if stats["status"] == "UP" then
> total_sessions = total_sessions + stats["scur"]
> end
> end
> end
> end
> return total_sessions
> end
> 
> -- register HAProxy "fetch"
> core.register_fetches("SmartRouting", dreSmartRouting)

Fwd: Lua + shared memory segment

[Kalantari]

Hi Thierry,

I know this an old thread but I'm having a similar issue where HA Proxy
doesn't allow me to use Redis (not allowed to use socket in fetches). My
scenario is as below:

one frontend and multiple backends (proxies). (the frontend is just one IP
and always sends requests to the same url)
I use Lua script SmartRouting to:
A. Look at the body of the body of the message and decide which back end to
use
B. If the backend servers being used are too busy (active sessions >
threshold) then it will choose to send the requests to the offlineAgent

frontend http-in
bind *:80
use_backend %[lua.SmartRouting(txn)]
default_backend OnlineChargers

backend Onlineagent
server testServer1 ${Server_1_IP}:${Server_1_Port} check
backend ServersForRequestTypeX
server testServer1 ${Server_2_IP}:${Server_2_Port} check
backend offlineAgents
server testServer1 ${Server_3_IP}:${Server_3_Port} check
http-response lua.add_DegradedSession

Straight forward up to this point and no need for Redis, however if a
message is sent to offlineAgent proxy, then I want all the rest of the
requests to be sent to the offline agent. (each message can have a
sessionID
inside the payload). I tried to add the sessionID for the messages inside
my SmartRouting to Redis as it was explained in your blogpost but HAProxy
throws an error and doesn't allow use of a socket in sample fetch.
below is my Lua script:

-- if number of active sessions goes higher than below then degraded
mode is detected
DEGRADED_THRESHOLD = 10
-- global object which will be used for routing of requests based on
request type xml tag
-- if the request type is not found then HA proxy's default backend will be used
routingTable = {
[""] = "Onlineagent",
[""] = "ServersForRequestTypeX"
}

local function SmartRouting(txn)
--print_r(txn)
local payload = txn.sf:req_body()
-- extract request tag (final occurence of  DEGRADED_THRESHOLD) then
core.Debug(
"shit is hiting the fan! Sending requests to degraded agent")

--TODO:  Add the sessionID to Redis and check for future requests if
it's in Redis then redirect to Offline Agent
return "offlineAgents"
end
end
return selectedBackend
end

function getTotalSessions()
local total_sessions = 0
for _, backend in pairs(core.backends) do
if backend.name == "Onlineagent" then
for _, server in pairs(backend.servers) do
-- Get server's stats
local stats = server:get_stats()

-- Get the backend's total number of current sessions
if stats["status"] == "UP" then
total_sessions = total_sessions + stats["scur"]
end
end
end
end
return total_sessions
end

-- register HAProxy "fetch"
core.register_fetches("SmartRouting", dreSmartRouting)

Re: Lua + shared memory segment

[bjun...@gmail.com]

2017-08-01 10:47 GMT+02:00 Thierry Fournier :
>
>> On 31 Jul 2017, at 22:41, bjun...@gmail.com wrote:
>>
>> Hi,
>>
>> i'm experimenting with some Lua code in HAProxy where i need a simple 
>> key/value store (not persistent). I want to avoid Redis or other external 
>> dependency.
>>
>> Is there some sort of shared memory segment in HAProxy Lua integration that 
>> can be used? (or is it possible to access HAProxy stick-tables from Lua?)
>
>
> Adding shared memory segment in Lua is an interesting way. Adding a key/value 
> system with
> Lua using shared memory seems very smart, but the development of this kind of 
> function
> may be complicated and needs a little bit of brainstorm.
>
> As the shared memory is local on the system, the information is not shared 
> between nodes
> distributed on servers.
>
> Also I have some doubts about the right usage of these system because shared 
> memory requires
> lock (sem) and haproxy doesn’t like locks.
>
>
>
>
> Its not possible today to access to the stick tables with other method than 
> the sample
> fetches and converters bindings. In other way, peer protocol requires full 
> mesh connections
> (I’m using 12 HAProxy, so 66 connections)
>
> I prefer sharing data with stick tables, mechanism are already written and 
> are reliable,
> but the content stored is limited. Maybe a peer protocol hierarchical 
> concentrator will be
> welcome.
>
>
>
>
> Note that, for my own usage I’m brainstorming with Redis. This software is 
> very light
> and fast. My goal is tracking sessions on many servers.
> So I think to this behaviour:
>
>  - Each haproxy reads the local redis,
>
>  - HAProxy send updates message in MULTICAST/UDP and all nodes receives the 
> update
>(this way may loose messages)
>
>
>
> Sharing data between HAProxy is not so easy :-)
>
> Thierry
>
>
>
>> --
>> Best Regards
>>
>> Bjoern
>

Hi Thierry,

thanks for your detailed answer.

Due to your remarks, i will go with Redis.


---
Best Regards

Bjoern

Lua + shared memory segment

[bjun...@gmail.com]

Hi,

i'm experimenting with some Lua code in HAProxy where i need a simple
key/value store (not persistent). I want to avoid Redis or other external
dependency.

Is there some sort of shared memory segment in HAProxy Lua integration that
can be used? (or is it possible to access HAProxy stick-tables from Lua?)


--
Best Regards

Bjoern