Re: appsession replacement in 1.6

[Aleksandar Lazic]

Hi Sylvain & Willy.

Am 19-11-2015 19:41, schrieb Willy Tarreau:

Hi Sylvain,

On Thu, Nov 19, 2015 at 05:04:40PM +0100, Sylvain Faivre wrote:

>What Aleks meant is that you don't need http-buffer-request as it's
>only used to process POST data which isn't your case.

Oh OK. Well, I did not talk about http-buffer-request in the 
beginning,

so I thought Aleks recommended to use it.
I can see that it is not useful in my case, so I removed it again.


I think he wanted to be on the safest side before knowing well all the
details of your setup. Now that it works, time for cleanup has come :-)


You got me Willy ;-)

For future reference, here is my latest config. It has been running 
for

a few hours on our staging servers without a problem :

backend front
[stick-table for another purpose]
[other stuff]
# sticky session on JSESSIONID
stick on urlp(jsessionid) table front_jsessionid
stick on urlp(jsessionid,;) table front_jsessionid
stick on cookie(JSESSIONID) table front_jsessionid
stick store-response cookie(JSESSIONID) table front_jsessionid

backend front_jsessionid
stick-table type string len 24 size 10m expire 1h peers prod


Thanks that's much appreciated. I wanted to work on a mini-doc to
explain how to replace appsession with stick tables but unfortunately I
failed to find the time to do it before the release. So I appreciate
that you share the result of your findings, it will definitely help
other users.

Thanks to you and Aleks for this :-)


Also thanks from me ;-)

Cheers
Aleks

Re: appsession replacement in 1.6

[Willy Tarreau]

Hi Sylvain,

On Thu, Nov 19, 2015 at 05:04:40PM +0100, Sylvain Faivre wrote:
> >What Aleks meant is that you don't need http-buffer-request as it's
> >only used to process POST data which isn't your case.
> 
> Oh OK. Well, I did not talk about http-buffer-request in the beginning, 
> so I thought Aleks recommended to use it.
> I can see that it is not useful in my case, so I removed it again.

I think he wanted to be on the safest side before knowing well all the
details of your setup. Now that it works, time for cleanup has come :-)

> For future reference, here is my latest config. It has been running for 
> a few hours on our staging servers without a problem :
> 
> backend front
> [stick-table for another purpose]
> [other stuff]
> # sticky session on JSESSIONID
> stick on urlp(jsessionid) table front_jsessionid
> stick on urlp(jsessionid,;) table front_jsessionid
> stick on cookie(JSESSIONID) table front_jsessionid
> stick store-response cookie(JSESSIONID) table front_jsessionid
> 
> backend front_jsessionid
> stick-table type string len 24 size 10m expire 1h peers prod

Thanks that's much appreciated. I wanted to work on a mini-doc to
explain how to replace appsession with stick tables but unfortunately I
failed to find the time to do it before the release. So I appreciate
that you share the result of your findings, it will definitely help
other users.

Thanks to you and Aleks for this :-)
Willy

Re: appsession replacement in 1.6

[Sylvain Faivre]

Hi Willy,

On 11/18/2015 09:31 PM, Willy Tarreau wrote:

Hi Sylvain,

On Tue, Nov 17, 2015 at 03:09:49PM +0100, Sylvain Faivre wrote:

option http-buffer-request

maybe you should stick on the header ;-)


OK I added "option http-buffer-request", it will help for sure !

What do you mean "stick on the header" ?


Oh well, it seems like I forgot to stick on the request URL parameter,
when passed with a question mark.


What Aleks meant is that you don't need http-buffer-request as it's
only used to process POST data which isn't your case.


Oh OK. Well, I did not talk about http-buffer-request in the beginning, 
so I thought Aleks recommended to use it.

I can see that it is not useful in my case, so I removed it again.




So I just added this to my config :
  stick on urlp(JSESSIONID) table back_jsessionid
  stick store-request urlp(JSESSIONID) table back_jsessionid

It should cover all cases now.


These two are partially redundant, since "stick on" is already a
shortcut for "stick match" + "stick store-request". So you end up
having this :

   stick match urlp(JSESSIONID) table back_jsessionid
   stick store-request urlp(JSESSIONID) table back_jsessionid
   stick store-request urlp(JSESSIONID) table back_jsessionid

Thus you can safely remove the "stick store-request" rule, you won't
notice the difference.



OK, I did that, and also changed urlp(jsessionid) to lower case, since 
our application server passes it in lower case when in URL parameters.


For future reference, here is my latest config. It has been running for 
a few hours on our staging servers without a problem :


backend front
[stick-table for another purpose]
[other stuff]
# sticky session on JSESSIONID
stick on urlp(jsessionid) table front_jsessionid
stick on urlp(jsessionid,;) table front_jsessionid
stick on cookie(JSESSIONID) table front_jsessionid
stick store-response cookie(JSESSIONID) table front_jsessionid

backend front_jsessionid
stick-table type string len 24 size 10m expire 1h peers prod


Best regards,
Sylvain

Re: appsession replacement in 1.6

[Willy Tarreau]

Hi Sylvain,

On Tue, Nov 17, 2015 at 03:09:49PM +0100, Sylvain Faivre wrote:
> >>option http-buffer-request
> >>
> >>maybe you should stick on the header ;-)
> >
> >OK I added "option http-buffer-request", it will help for sure !
> >
> >What do you mean "stick on the header" ?
> 
> Oh well, it seems like I forgot to stick on the request URL parameter, 
> when passed with a question mark.

What Aleks meant is that you don't need http-buffer-request as it's
only used to process POST data which isn't your case.

> So I just added this to my config :
>  stick on urlp(JSESSIONID) table back_jsessionid
>  stick store-request urlp(JSESSIONID) table back_jsessionid
> 
> It should cover all cases now.

These two are partially redundant, since "stick on" is already a
shortcut for "stick match" + "stick store-request". So you end up
having this :

  stick match urlp(JSESSIONID) table back_jsessionid
  stick store-request urlp(JSESSIONID) table back_jsessionid
  stick store-request urlp(JSESSIONID) table back_jsessionid

Thus you can safely remove the "stick store-request" rule, you won't
notice the difference.

Regards,
willy

Re: appsession replacement in 1.6

[Sylvain Faivre]

Replying to myself :

On 11/17/2015 10:58 AM, Sylvain Faivre wrote:

Hi Aleks, and thanks again for your help.

Concerning this point :
On 11/16/2015 11:16 PM, Aleksandar Lazic wrote:


As described here

http://git.haproxy.org/?p=haproxy-1.6.git;a=blob;f=doc/configuration.txt;h=45d1aacfbe0d2d53193f7956a0dd03e5f8151ea6;hb=HEAD#l5043



option http-buffer-request

maybe you should stick on the header ;-)


OK I added "option http-buffer-request", it will help for sure !

What do you mean "stick on the header" ?


Oh well, it seems like I forgot to stick on the request URL parameter, 
when passed with a question mark.


So I just added this to my config :
 stick on urlp(JSESSIONID) table back_jsessionid
 stick store-request urlp(JSESSIONID) table back_jsessionid

It should cover all cases now.


I currently have the following config, I think it will stick on
JSESSIONID, either in the cookie (so request header, also capturing the
cookie in reply header) or as a request URL parameter. Am I missing
something ?

backend front
 [stick-table for another purpose]
 [other stuff]
 # sticky session on JSESSIONID
 option http-buffer-request
 stick on urlp(JSESSIONID,;) table front_jsessionid
 stick on cookie(JSESSIONID) table front_jsessionid
 stick store-response cookie(JSESSIONID) table front_jsessionid
 stick store-request cookie(JSESSIONID) table front_jsessionid
 stick store-request urlp(JSESSIONID,;) table front_jsessionid

backend front_jsessionid
 stick-table type string len 24 size 10m expire 1h peers prod

Re: appsession replacement in 1.6

[Sylvain Faivre]

Hi Aleks, and thanks again for your help.

Concerning this point :
On 11/16/2015 11:16 PM, Aleksandar Lazic wrote:


As described here

http://git.haproxy.org/?p=haproxy-1.6.git;a=blob;f=doc/configuration.txt;h=45d1aacfbe0d2d53193f7956a0dd03e5f8151ea6;hb=HEAD#l5043


option http-buffer-request

maybe you should stick on the header ;-)


OK I added "option http-buffer-request", it will help for sure !

What do you mean "stick on the header" ?
I currently have the following config, I think it will stick on 
JSESSIONID, either in the cookie (so request header, also capturing the 
cookie in reply header) or as a request URL parameter. Am I missing 
something ?


backend front
[stick-table for another purpose]
[other stuff]
# sticky session on JSESSIONID
option http-buffer-request
stick on urlp(JSESSIONID,;) table front_jsessionid
stick on cookie(JSESSIONID) table front_jsessionid
stick store-response cookie(JSESSIONID) table front_jsessionid
stick store-request cookie(JSESSIONID) table front_jsessionid
stick store-request urlp(JSESSIONID,;) table front_jsessionid

backend front_jsessionid
stick-table type string len 24 size 10m expire 1h peers prod

Re: appsession replacement in 1.6

[Sylvain Faivre]

Hi Aleks,

On 11/10/2015 10:56 PM, Aleksandar Lazic wrote:

Dear Sylvain Faivre,

Am 10-11-2015 12:48, schrieb Sylvain Faivre:

On 11/10/2015 12:00 AM, Aleksandar Lazic wrote:

Hi Sylvain Faivre.

Am 09-11-2015 17:31, schrieb Sylvain Faivre:


[snipp]


So, I've got this so far :

backend http

  stick-table type string len 24 size 10m expire 1h peers prod

  stick on urlp(JSESSIONID,;)
  stick on cookie(JSESSIONID)


Does this seem right ?
The help for "stick on" tells it defines a request pattern, so I guess
this would not match JSESSIONID cookie ou url parameter set in the
reply ?


I have no java server here to test this commands but with this commands
haproxy does not warn you about some config errors ;-).

###
backend dest01
   mode http

   stick-table type string len 24 size 10m expire 1h peers prod

   stick on urlp(JSESSIONID,;)
   stick on cookie(JSESSIONID)

   stick store-response cookie(JSESSIONID)
#  stick store-response res.hdr(JSESSIONID,;)

   stick store-request cookie(JSESSIONID)
   stick store-request urlp(JSESSIONID,;)

   server srv_dest01 dest01.com:80
###

I have not seen a good option to read the JSESSIONID from the response
header in case it is not in a cookie.
Have anyone I idea?!

Please can you post a full response header which was created from the
app or appserver when the app or appserver have detected that the client
does not allow cookies.


[snipp]


In fact, the server sends the JSESSIONID as a cookie even if the
client does not support cookies, *and* it adds the JSESSIONID as a URL
parameter in all links, so this should be all right.


This would be helpfully to see the full response.
Maybe some appserver behaves different.


As far as I know, there is no way for the server to detect if the client 
has cookies enabled, by looking only at the first request from that client.


According to a quick Google search, the most common ways to detect 
cookies support are either to use Javascript (so client-side check) or 
to reply with a redirect response with the cookie set, then when 
processing the redirected URL, check if the client sent the cookie along 
with the request (so this case will be covered by the proposed HAproxy 
settings).


I don't feel comfortable giving our application server version on a 
public list, but I will send it to you in private.


Here are the headers from a client request and server reply, with a 
brand new profile on the client (with cookies disabled) :


- request :
GET /front/url1.do?m=booking=FR HTTP/1.1
Host: redacted.host.domain
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:42.0) 
Gecko/20100101 Firefox/42.0

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive

- reply :
HTTP/1.1 200 OK
Date: Mon, 16 Nov 2015 15:25:41 GMT
Content-Type: text/html;charset=ISO-8859-15
Set-Cookie: JSESSIONID=uNmYNvgUME5-8LYPzimsCg__.8d15fc; Path=/front
Vary: Accept-Encoding
Content-Encoding: gzip
Transfer-Encoding: chunked

And here is a URL sample from the body of the reply. You will notice 
that the jsessionid is there twice, first one after a semicolon and 
second one after a question mark. I am not sure if this comes from the 
application server of from our custom code.


src="https://redacted.host.domain/front/url2.do;jsessionid=uNmYNvgUME5-8LYPzimsCg__.8d15fc?jsessionid=uNmYNvgUME5-8LYPzimsCg__.8d15fc&langcode=FR"; 
language="JavaScript">






I just tried your config on a test server, and the session IDs are
rightly recorded in the table, whether the client accepts cookies or
not.


Perfect.


I still have some test cases to run, I will check this next week and
report back if needed.


Oh yes please tell us the results so that we can add this as migration
example for appsession.


OK, I will. This will not go into production yet, we still need to run 
it on a test environment for at least 3 weeks...


Best regards.
Sylvain