Re: appsession replacement in 1.6
Hi Sylvain & Willy. Am 19-11-2015 19:41, schrieb Willy Tarreau: Hi Sylvain, On Thu, Nov 19, 2015 at 05:04:40PM +0100, Sylvain Faivre wrote: >What Aleks meant is that you don't need http-buffer-request as it's >only used to process POST data which isn't your case. Oh OK. Well, I did not talk about http-buffer-request in the beginning, so I thought Aleks recommended to use it. I can see that it is not useful in my case, so I removed it again. I think he wanted to be on the safest side before knowing well all the details of your setup. Now that it works, time for cleanup has come :-) You got me Willy ;-) For future reference, here is my latest config. It has been running for a few hours on our staging servers without a problem : backend front [stick-table for another purpose] [other stuff] # sticky session on JSESSIONID stick on urlp(jsessionid) table front_jsessionid stick on urlp(jsessionid,;) table front_jsessionid stick on cookie(JSESSIONID) table front_jsessionid stick store-response cookie(JSESSIONID) table front_jsessionid backend front_jsessionid stick-table type string len 24 size 10m expire 1h peers prod Thanks that's much appreciated. I wanted to work on a mini-doc to explain how to replace appsession with stick tables but unfortunately I failed to find the time to do it before the release. So I appreciate that you share the result of your findings, it will definitely help other users. Thanks to you and Aleks for this :-) Also thanks from me ;-) Cheers Aleks
Re: appsession replacement in 1.6
Hi Sylvain, On Thu, Nov 19, 2015 at 05:04:40PM +0100, Sylvain Faivre wrote: > >What Aleks meant is that you don't need http-buffer-request as it's > >only used to process POST data which isn't your case. > > Oh OK. Well, I did not talk about http-buffer-request in the beginning, > so I thought Aleks recommended to use it. > I can see that it is not useful in my case, so I removed it again. I think he wanted to be on the safest side before knowing well all the details of your setup. Now that it works, time for cleanup has come :-) > For future reference, here is my latest config. It has been running for > a few hours on our staging servers without a problem : > > backend front > [stick-table for another purpose] > [other stuff] > # sticky session on JSESSIONID > stick on urlp(jsessionid) table front_jsessionid > stick on urlp(jsessionid,;) table front_jsessionid > stick on cookie(JSESSIONID) table front_jsessionid > stick store-response cookie(JSESSIONID) table front_jsessionid > > backend front_jsessionid > stick-table type string len 24 size 10m expire 1h peers prod Thanks that's much appreciated. I wanted to work on a mini-doc to explain how to replace appsession with stick tables but unfortunately I failed to find the time to do it before the release. So I appreciate that you share the result of your findings, it will definitely help other users. Thanks to you and Aleks for this :-) Willy
Re: appsession replacement in 1.6
Hi Willy, On 11/18/2015 09:31 PM, Willy Tarreau wrote: Hi Sylvain, On Tue, Nov 17, 2015 at 03:09:49PM +0100, Sylvain Faivre wrote: option http-buffer-request maybe you should stick on the header ;-) OK I added "option http-buffer-request", it will help for sure ! What do you mean "stick on the header" ? Oh well, it seems like I forgot to stick on the request URL parameter, when passed with a question mark. What Aleks meant is that you don't need http-buffer-request as it's only used to process POST data which isn't your case. Oh OK. Well, I did not talk about http-buffer-request in the beginning, so I thought Aleks recommended to use it. I can see that it is not useful in my case, so I removed it again. So I just added this to my config : stick on urlp(JSESSIONID) table back_jsessionid stick store-request urlp(JSESSIONID) table back_jsessionid It should cover all cases now. These two are partially redundant, since "stick on" is already a shortcut for "stick match" + "stick store-request". So you end up having this : stick match urlp(JSESSIONID) table back_jsessionid stick store-request urlp(JSESSIONID) table back_jsessionid stick store-request urlp(JSESSIONID) table back_jsessionid Thus you can safely remove the "stick store-request" rule, you won't notice the difference. OK, I did that, and also changed urlp(jsessionid) to lower case, since our application server passes it in lower case when in URL parameters. For future reference, here is my latest config. It has been running for a few hours on our staging servers without a problem : backend front [stick-table for another purpose] [other stuff] # sticky session on JSESSIONID stick on urlp(jsessionid) table front_jsessionid stick on urlp(jsessionid,;) table front_jsessionid stick on cookie(JSESSIONID) table front_jsessionid stick store-response cookie(JSESSIONID) table front_jsessionid backend front_jsessionid stick-table type string len 24 size 10m expire 1h peers prod Best regards, Sylvain
Re: appsession replacement in 1.6
Hi Sylvain, On Tue, Nov 17, 2015 at 03:09:49PM +0100, Sylvain Faivre wrote: > >>option http-buffer-request > >> > >>maybe you should stick on the header ;-) > > > >OK I added "option http-buffer-request", it will help for sure ! > > > >What do you mean "stick on the header" ? > > Oh well, it seems like I forgot to stick on the request URL parameter, > when passed with a question mark. What Aleks meant is that you don't need http-buffer-request as it's only used to process POST data which isn't your case. > So I just added this to my config : > stick on urlp(JSESSIONID) table back_jsessionid > stick store-request urlp(JSESSIONID) table back_jsessionid > > It should cover all cases now. These two are partially redundant, since "stick on" is already a shortcut for "stick match" + "stick store-request". So you end up having this : stick match urlp(JSESSIONID) table back_jsessionid stick store-request urlp(JSESSIONID) table back_jsessionid stick store-request urlp(JSESSIONID) table back_jsessionid Thus you can safely remove the "stick store-request" rule, you won't notice the difference. Regards, willy
Re: appsession replacement in 1.6
Replying to myself : On 11/17/2015 10:58 AM, Sylvain Faivre wrote: Hi Aleks, and thanks again for your help. Concerning this point : On 11/16/2015 11:16 PM, Aleksandar Lazic wrote: As described here http://git.haproxy.org/?p=haproxy-1.6.git;a=blob;f=doc/configuration.txt;h=45d1aacfbe0d2d53193f7956a0dd03e5f8151ea6;hb=HEAD#l5043 option http-buffer-request maybe you should stick on the header ;-) OK I added "option http-buffer-request", it will help for sure ! What do you mean "stick on the header" ? Oh well, it seems like I forgot to stick on the request URL parameter, when passed with a question mark. So I just added this to my config : stick on urlp(JSESSIONID) table back_jsessionid stick store-request urlp(JSESSIONID) table back_jsessionid It should cover all cases now. I currently have the following config, I think it will stick on JSESSIONID, either in the cookie (so request header, also capturing the cookie in reply header) or as a request URL parameter. Am I missing something ? backend front [stick-table for another purpose] [other stuff] # sticky session on JSESSIONID option http-buffer-request stick on urlp(JSESSIONID,;) table front_jsessionid stick on cookie(JSESSIONID) table front_jsessionid stick store-response cookie(JSESSIONID) table front_jsessionid stick store-request cookie(JSESSIONID) table front_jsessionid stick store-request urlp(JSESSIONID,;) table front_jsessionid backend front_jsessionid stick-table type string len 24 size 10m expire 1h peers prod
Re: appsession replacement in 1.6
Hi Aleks, and thanks again for your help. Concerning this point : On 11/16/2015 11:16 PM, Aleksandar Lazic wrote: As described here http://git.haproxy.org/?p=haproxy-1.6.git;a=blob;f=doc/configuration.txt;h=45d1aacfbe0d2d53193f7956a0dd03e5f8151ea6;hb=HEAD#l5043 option http-buffer-request maybe you should stick on the header ;-) OK I added "option http-buffer-request", it will help for sure ! What do you mean "stick on the header" ? I currently have the following config, I think it will stick on JSESSIONID, either in the cookie (so request header, also capturing the cookie in reply header) or as a request URL parameter. Am I missing something ? backend front [stick-table for another purpose] [other stuff] # sticky session on JSESSIONID option http-buffer-request stick on urlp(JSESSIONID,;) table front_jsessionid stick on cookie(JSESSIONID) table front_jsessionid stick store-response cookie(JSESSIONID) table front_jsessionid stick store-request cookie(JSESSIONID) table front_jsessionid stick store-request urlp(JSESSIONID,;) table front_jsessionid backend front_jsessionid stick-table type string len 24 size 10m expire 1h peers prod
Re: appsession replacement in 1.6
Hi Sylvain. Am 16-11-2015 17:06, schrieb Sylvain Faivre: Hi Aleks, On 11/10/2015 10:56 PM, Aleksandar Lazic wrote: Dear Sylvain Faivre, [snipp] This would be helpfully to see the full response. Maybe some appserver behaves different. As far as I know, there is no way for the server to detect if the client has cookies enabled, by looking only at the first request from that client. According to a quick Google search, the most common ways to detect cookies support are either to use Javascript (so client-side check) or to reply with a redirect response with the cookie set, then when processing the redirected URL, check if the client sent the cookie along with the request (so this case will be covered by the proposed HAproxy settings). Yes. That's also my experience. I don't feel comfortable giving our application server version on a public list, but I will send it to you in private. thanks. Here are the headers from a client request and server reply, with a brand new profile on the client (with cookies disabled) : - request : GET /front/url1.do?m=booking&langcode=FR HTTP/1.1 Host: redacted.host.domain User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:42.0) Gecko/20100101 Firefox/42.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Connection: keep-alive - reply : HTTP/1.1 200 OK Date: Mon, 16 Nov 2015 15:25:41 GMT Content-Type: text/html;charset=ISO-8859-15 Set-Cookie: JSESSIONID=uNmYNvgUME5-8LYPzimsCg__.8d15fc; Path=/front Vary: Accept-Encoding Content-Encoding: gzip Transfer-Encoding: chunked And here is a URL sample from the body of the reply. You will notice that the jsessionid is there twice, first one after a semicolon and second one after a question mark. I am not sure if this comes from the application server of from our custom code. https://redacted.host.domain/front/url2.do;jsessionid=uNmYNvgUME5-8LYPzimsCg__.8d15fc?jsessionid=uNmYNvgUME5-8LYPzimsCg__.8d15fc&langcode=FR"; language="JavaScript"> thanks. As described here http://git.haproxy.org/?p=haproxy-1.6.git;a=blob;f=doc/configuration.txt;h=45d1aacfbe0d2d53193f7956a0dd03e5f8151ea6;hb=HEAD#l5043 option http-buffer-request maybe you should stick on the header ;-) [snipp] Oh yes please tell us the results so that we can add this as migration example for appsession. OK, I will. This will not go into production yet, we still need to run it on a test environment for at least 3 weeks... Thanks. Best regards Aleks
Re: appsession replacement in 1.6
Hi Aleks, On 11/10/2015 10:56 PM, Aleksandar Lazic wrote: Dear Sylvain Faivre, Am 10-11-2015 12:48, schrieb Sylvain Faivre: On 11/10/2015 12:00 AM, Aleksandar Lazic wrote: Hi Sylvain Faivre. Am 09-11-2015 17:31, schrieb Sylvain Faivre: [snipp] So, I've got this so far : backend http stick-table type string len 24 size 10m expire 1h peers prod stick on urlp(JSESSIONID,;) stick on cookie(JSESSIONID) Does this seem right ? The help for "stick on" tells it defines a request pattern, so I guess this would not match JSESSIONID cookie ou url parameter set in the reply ? I have no java server here to test this commands but with this commands haproxy does not warn you about some config errors ;-). ### backend dest01 mode http stick-table type string len 24 size 10m expire 1h peers prod stick on urlp(JSESSIONID,;) stick on cookie(JSESSIONID) stick store-response cookie(JSESSIONID) # stick store-response res.hdr(JSESSIONID,;) stick store-request cookie(JSESSIONID) stick store-request urlp(JSESSIONID,;) server srv_dest01 dest01.com:80 ### I have not seen a good option to read the JSESSIONID from the response header in case it is not in a cookie. Have anyone I idea?! Please can you post a full response header which was created from the app or appserver when the app or appserver have detected that the client does not allow cookies. [snipp] In fact, the server sends the JSESSIONID as a cookie even if the client does not support cookies, *and* it adds the JSESSIONID as a URL parameter in all links, so this should be all right. This would be helpfully to see the full response. Maybe some appserver behaves different. As far as I know, there is no way for the server to detect if the client has cookies enabled, by looking only at the first request from that client. According to a quick Google search, the most common ways to detect cookies support are either to use Javascript (so client-side check) or to reply with a redirect response with the cookie set, then when processing the redirected URL, check if the client sent the cookie along with the request (so this case will be covered by the proposed HAproxy settings). I don't feel comfortable giving our application server version on a public list, but I will send it to you in private. Here are the headers from a client request and server reply, with a brand new profile on the client (with cookies disabled) : - request : GET /front/url1.do?m=booking&langcode=FR HTTP/1.1 Host: redacted.host.domain User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:42.0) Gecko/20100101 Firefox/42.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Connection: keep-alive - reply : HTTP/1.1 200 OK Date: Mon, 16 Nov 2015 15:25:41 GMT Content-Type: text/html;charset=ISO-8859-15 Set-Cookie: JSESSIONID=uNmYNvgUME5-8LYPzimsCg__.8d15fc; Path=/front Vary: Accept-Encoding Content-Encoding: gzip Transfer-Encoding: chunked And here is a URL sample from the body of the reply. You will notice that the jsessionid is there twice, first one after a semicolon and second one after a question mark. I am not sure if this comes from the application server of from our custom code. src="https://redacted.host.domain/front/url2.do;jsessionid=uNmYNvgUME5-8LYPzimsCg__.8d15fc?jsessionid=uNmYNvgUME5-8LYPzimsCg__.8d15fc&langcode=FR"; language="JavaScript"> I just tried your config on a test server, and the session IDs are rightly recorded in the table, whether the client accepts cookies or not. Perfect. I still have some test cases to run, I will check this next week and report back if needed. Oh yes please tell us the results so that we can add this as migration example for appsession. OK, I will. This will not go into production yet, we still need to run it on a test environment for at least 3 weeks... Best regards. Sylvain
Re: appsession replacement in 1.6
Dear Sylvain Faivre, Am 10-11-2015 12:48, schrieb Sylvain Faivre: On 11/10/2015 12:00 AM, Aleksandar Lazic wrote: Hi Sylvain Faivre. Am 09-11-2015 17:31, schrieb Sylvain Faivre: [snipp] So, I've got this so far : backend http stick-table type string len 24 size 10m expire 1h peers prod stick on urlp(JSESSIONID,;) stick on cookie(JSESSIONID) Does this seem right ? The help for "stick on" tells it defines a request pattern, so I guess this would not match JSESSIONID cookie ou url parameter set in the reply ? I have no java server here to test this commands but with this commands haproxy does not warn you about some config errors ;-). ### backend dest01 mode http stick-table type string len 24 size 10m expire 1h peers prod stick on urlp(JSESSIONID,;) stick on cookie(JSESSIONID) stick store-response cookie(JSESSIONID) # stick store-response res.hdr(JSESSIONID,;) stick store-request cookie(JSESSIONID) stick store-request urlp(JSESSIONID,;) server srv_dest01 dest01.com:80 ### I have not seen a good option to read the JSESSIONID from the response header in case it is not in a cookie. Have anyone I idea?! Please can you post a full response header which was created from the app or appserver when the app or appserver have detected that the client does not allow cookies. [snipp] In fact, the server sends the JSESSIONID as a cookie even if the client does not support cookies, *and* it adds the JSESSIONID as a URL parameter in all links, so this should be all right. This would be helpfully to see the full response. Maybe some appserver behaves different. I just tried your config on a test server, and the session IDs are rightly recorded in the table, whether the client accepts cookies or not. Perfect. I still have some test cases to run, I will check this next week and report back if needed. Oh yes please tell us the results so that we can add this as migration example for appsession. Thanks Aleks
Re: appsession replacement in 1.6
On 11/10/2015 12:00 AM, Aleksandar Lazic wrote: Hi Sylvain Faivre. Am 09-11-2015 17:31, schrieb Sylvain Faivre: Hi, Sorry I'm late on this discussion, following this thread : https://marc.info/?l=haproxy&m=143345620219498&w=2 We are using appsession with HAproxy 1.5 like this : Thanks ;-) backend http appsession JSESSIONID len 24 timeout 1h request-learn We would like to be able to do the same thing with HAproxy 1.6. If it is possible, we'd like to catch the JSESSIONID in cookies and URL parameters, either in the request or in the response. I tried to use the info posted previously by Aleksandar, but I encountered several problems : - using "stick on" in frontend section fails with : 'stick' ignored because frontend 'web' has no backend capability. - using "stick store-response urlp(JSESSIONID,;)" in backend section fails with : 'stick': fetch method 'urlp' extracts information from 'HTTP request headers', none of which is available for 'store-response'. So, I've got this so far : backend http stick-table type string len 24 size 10m expire 1h peers prod stick on urlp(JSESSIONID,;) stick on cookie(JSESSIONID) Does this seem right ? The help for "stick on" tells it defines a request pattern, so I guess this would not match JSESSIONID cookie ou url parameter set in the reply ? I have no java server here to test this commands but with this commands haproxy does not warn you about some config errors ;-). ### backend dest01 mode http stick-table type string len 24 size 10m expire 1h peers prod stick on urlp(JSESSIONID,;) stick on cookie(JSESSIONID) stick store-response cookie(JSESSIONID) # stick store-response res.hdr(JSESSIONID,;) stick store-request cookie(JSESSIONID) stick store-request urlp(JSESSIONID,;) server srv_dest01 dest01.com:80 ### I have not seen a good option to read the JSESSIONID from the response header in case it is not in a cookie. Have anyone I idea?! Please can you post a full response header which was created from the app or appserver when the app or appserver have detected that the client does not allow cookies. cheers Aleks Thank you Aleks. In fact, the server sends the JSESSIONID as a cookie even if the client does not support cookies, *and* it adds the JSESSIONID as a URL parameter in all links, so this should be all right. I just tried your config on a test server, and the session IDs are rightly recorded in the table, whether the client accepts cookies or not. I still have some test cases to run, I will check this next week and report back if needed. Regards, Sylvain
Re: appsession replacement in 1.6
Hi Sylvain Faivre. Am 09-11-2015 17:31, schrieb Sylvain Faivre: Hi, Sorry I'm late on this discussion, following this thread : https://marc.info/?l=haproxy&m=143345620219498&w=2 We are using appsession with HAproxy 1.5 like this : Thanks ;-) backend http appsession JSESSIONID len 24 timeout 1h request-learn We would like to be able to do the same thing with HAproxy 1.6. If it is possible, we'd like to catch the JSESSIONID in cookies and URL parameters, either in the request or in the response. I tried to use the info posted previously by Aleksandar, but I encountered several problems : - using "stick on" in frontend section fails with : 'stick' ignored because frontend 'web' has no backend capability. - using "stick store-response urlp(JSESSIONID,;)" in backend section fails with : 'stick': fetch method 'urlp' extracts information from 'HTTP request headers', none of which is available for 'store-response'. So, I've got this so far : backend http stick-table type string len 24 size 10m expire 1h peers prod stick on urlp(JSESSIONID,;) stick on cookie(JSESSIONID) Does this seem right ? The help for "stick on" tells it defines a request pattern, so I guess this would not match JSESSIONID cookie ou url parameter set in the reply ? I have no java server here to test this commands but with this commands haproxy does not warn you about some config errors ;-). ### backend dest01 mode http stick-table type string len 24 size 10m expire 1h peers prod stick on urlp(JSESSIONID,;) stick on cookie(JSESSIONID) stick store-response cookie(JSESSIONID) # stick store-response res.hdr(JSESSIONID,;) stick store-request cookie(JSESSIONID) stick store-request urlp(JSESSIONID,;) server srv_dest01 dest01.com:80 ### I have not seen a good option to read the JSESSIONID from the response header in case it is not in a cookie. Have anyone I idea?! Please can you post a full response header which was created from the app or appserver when the app or appserver have detected that the client does not allow cookies. cheers Aleks