Re: File uploads (multipart/form-data POST ) and transparent mode fail
Yee! THANK YOU!!! Works perfectly :-D I guess I can set the floating rule Destination: Server-IP to a range of ip numbers? Or should I make one floating rule per server ip? /Magnus PS: For your information the config below does not contain the Transparent ClientIP Yup., Was turned off so visitors could upload :-) On Sat, Jan 18, 2014 at 11:51 PM, PiBa-NL piba.nl@gmail.com wrote: Hi Magnus, I have integrated that 'transparent' option into the pfSense(FreeBSD8.3) haproxy-devel package. And can confirm that there is an issue when sending a large POST. For your information the config below does not contain the Transparent ClientIP option.. Which would read source 0.0.0.0 usesrc clientip.. Also on pfSense the main firewall is 'pf' , but to get 'transparent' traffic working it was needed to in the background also load and configure part of ipfw.. (this is also done for captive portal..) This so HAProxy gets to see the tcp traffic, and prevent replies from being routed out the wan interface.. The solution is to configure a floating rule like this: Action: Pass Quick: YES Interface: DMZ (the one pointing to your server..) Direction: Out Protocol: TCP Source: ANY Destination: Server-IP Destination: Server-PORT State Type: sloppy state Ill try and see if i can automate that, and if not at least put a warning note that this kind of rule must be added for it to work properly. Can you confirm this indeed solves the issue? Thanks PiBa-NL Magnus Thomé schreef op 18-1-2014 9:32: Last night during a couple of hours I took the time to read through the whole documentation from start to finish (instead of just doing keyword searches in it). But I really can't find anything. I set up option forceclose (and also option forwardfor which is unrelated) just to see if anything happened but nope. I wild guess from me as a total noob is that something is divided into 64kbyte chunks,being that a buffer, a particular set of packets or whatnot, and the first time 64kbytes is sent it goes through ok but the second and further chunks go astray. I've scratched my head wondering if the webserveror or possibly the pfsense box has anything set in connection with 64kB and of course also looked for anything like that in the haproxy documentation. There are no problems sending items larger than 64kB in the other normal direction, to the browsers. Will setting a cookie help? ANY help or pointers in some direction would be deeply appreciated /Magnus Thomé On Fri, Jan 17, 2014 at 4:50 PM, Magnus Thomé magn...@gmail.com wrote: I've really really searched for answers, both in the mailing list archives and google but haven't been able to find anything. Would deeply appreciate any help! I'm running pfsense 2.1 with the only extra package installed being haproxy-devel 1.5-dev19 pkg v 0.6 EVERYTHING works great but one single thing: When doing a HTTP file upload with a FORM multipart/form-data POST to any server behind the firewall it only works with very small files, aprox max 60kbyte. With slightly larger files I get a timeout page after a while and with even larger files I get nothing at all. It seems that when Transparent ClientIP is enabled and set to DMZ the file uploads fail and with Transparent ClientIP disabled all works perfectly as it should. I do need the transparent mode though. Is there a setting somewhere I've missed? Thanks in advance for any possible help /Magnus -- The config created by pfsense GUI looks like this: global stats socket /tmp/haproxy.socket level admin uid 80 gid 80 nbproc 1 chroot /var/empty daemon frontend SRV-WEB1-merged bind83.250.27.152:80 default_backend SRV-WEB1_http modehttp log global option dontlognull timeout client 3 acl 0_rejsa.nu hdr_end(host) -i rejsa.nu use_backend SRV-WEB1_http if 0_rejsa.nu acl 1_rejsa.se hdr_end(host) -i rejsa.se use_backend SRV-WEB1_http if 1_rejsa.se acl 2_tystpc.nu hdr_end(host) -i tystpc.nu use_backend SRV-WEB2_http if 2_tystpc.nu acl 3_tystpc.se hdr_end(host) -i tystpc.se use_backend SRV-WEB2_http if 3_tystpc.se backend SRV-WEB1_http modehttp balance roundrobin timeout connect 3 timeout server 3 retries 3
Re: File uploads (multipart/form-data POST ) and transparent mode fail
Hi Magnus, I'm currently in the process of automating the creation this rule. Needs a little more testing and together with some other new features a was already busy with i think it will be ready in a week or so it will be part of the pfSense package version 1.5-dev21 pkg v 0.7. As for the current 'workaround', you can probably make an alias with all ip's you want to affect and use that in the floating-rule. I haven't tested it but cant think of a reason why that wouldn't work.. Greets PiBa-NL Magnus Thomé schreef op 19-1-2014 9:45: Yee! THANK YOU!!! Works perfectly :-D I guess I can set the floating rule Destination: Server-IP to a range of ip numbers? Or should I make one floating rule per server ip? /Magnus PS: For your information the config below does not contain the Transparent ClientIP Yup., Was turned off so visitors could upload :-) On Sat, Jan 18, 2014 at 11:51 PM, PiBa-NL piba.nl@gmail.com mailto:piba.nl@gmail.com wrote: Hi Magnus, I have integrated that 'transparent' option into the pfSense(FreeBSD8.3) haproxy-devel package. And can confirm that there is an issue when sending a large POST. For your information the config below does not contain the Transparent ClientIP option.. Which would read source 0.0.0.0 usesrc clientip.. Also on pfSense the main firewall is 'pf' , but to get 'transparent' traffic working it was needed to in the background also load and configure part of ipfw.. (this is also done for captive portal..) This so HAProxy gets to see the tcp traffic, and prevent replies from being routed out the wan interface.. The solution is to configure a floating rule like this: Action: Pass Quick: YES Interface: DMZ (the one pointing to your server..) Direction: Out Protocol: TCP Source: ANY Destination: Server-IP Destination: Server-PORT State Type: sloppy state Ill try and see if i can automate that, and if not at least put a warning note that this kind of rule must be added for it to work properly. Can you confirm this indeed solves the issue? Thanks PiBa-NL Magnus Thomé schreef op 18-1-2014 9:32: Last night during a couple of hours I took the time to read through the whole documentation from start to finish (instead of just doing keyword searches in it). But I really can't find anything. I set up option forceclose (and also option forwardfor which is unrelated) just to see if anything happened but nope. I wild guess from me as a total noob is that something is divided into 64kbyte chunks,being that a buffer, a particular set of packets or whatnot, and the first time 64kbytes is sent it goes through ok but the second and further chunks go astray. I've scratched my head wondering if the webserveror or possibly the pfsense box has anything set in connection with 64kB and of course also looked for anything like that in the haproxy documentation. There are no problems sending items larger than 64kB in the other normal direction, to the browsers. Will setting a cookie help? ANY help or pointers in some direction would be deeply appreciated /Magnus Thomé On Fri, Jan 17, 2014 at 4:50 PM, Magnus Thomé magn...@gmail.com mailto:magn...@gmail.com wrote: I've really really searched for answers, both in the mailing list archives and google but haven't been able to find anything. Would deeply appreciate any help! I'm running pfsense 2.1 with the only extra package installed being haproxy-devel 1.5-dev19 pkg v 0.6 EVERYTHING works great but one single thing: When doing a HTTP file upload with a FORM multipart/form-data POST to any server behind the firewall it only works with very small files, aprox max 60kbyte. With slightly larger files I get a timeout page after a while and with even larger files I get nothing at all. It seems that when Transparent ClientIP is enabled and set to DMZ the file uploads fail and with Transparent ClientIP disabled all works perfectly as it should. I do need the transparent mode though. Is there a setting somewhere I've missed? Thanks in advance for any possible help /Magnus -- The config created by pfsense GUI looks like this: global stats socket /tmp/haproxy.socket level admin uid 80 gid 80 nbproc 1 chroot /var/empty daemon frontend SRV-WEB1-merged bind 83.250.27.152:80 http://83.250.27.152:80
Re: File uploads (multipart/form-data POST ) and transparent mode fail
Last night during a couple of hours I took the time to read through the whole documentation from start to finish (instead of just doing keyword searches in it). But I really can't find anything. I set up option forceclose (and also option forwardfor which is unrelated) just to see if anything happened but nope. I wild guess from me as a total noob is that something is divided into 64kbyte chunks,being that a buffer, a particular set of packets or whatnot, and the first time 64kbytes is sent it goes through ok but the second and further chunks go astray. I've scratched my head wondering if the webserveror or possibly the pfsense box has anything set in connection with 64kB and of course also looked for anything like that in the haproxy documentation. There are no problems sending items larger than 64kB in the other normal direction, to the browsers. Will setting a cookie help? ANY help or pointers in some direction would be deeply appreciated /Magnus Thomé On Fri, Jan 17, 2014 at 4:50 PM, Magnus Thomé magn...@gmail.com wrote: I've really really searched for answers, both in the mailing list archives and google but haven't been able to find anything. Would deeply appreciate any help! I'm running pfsense 2.1 with the only extra package installed being haproxy-devel 1.5-dev19 pkg v 0.6 EVERYTHING works great but one single thing: When doing a HTTP file upload with a FORM multipart/form-data POST to any server behind the firewall it only works with very small files, aprox max 60kbyte. With slightly larger files I get a timeout page after a while and with even larger files I get nothing at all. It seems that when Transparent ClientIP is enabled and set to DMZ the file uploads fail and with Transparent ClientIP disabled all works perfectly as it should. I do need the transparent mode though. Is there a setting somewhere I've missed? Thanks in advance for any possible help /Magnus -- The config created by pfsense GUI looks like this: global stats socket /tmp/haproxy.socket level admin uid 80 gid 80 nbproc 1 chroot /var/empty daemon frontend SRV-WEB1-merged bind83.250.27.152:80 default_backend SRV-WEB1_http modehttp log global option dontlognull timeout client 3 acl 0_rejsa.nu hdr_end(host) -i rejsa.nu use_backend SRV-WEB1_http if 0_rejsa.nu acl 1_rejsa.se hdr_end(host) -i rejsa.se use_backend SRV-WEB1_http if 1_rejsa.se acl 2_tystpc.nu hdr_end(host) -i tystpc.nu use_backend SRV-WEB2_http if 2_tystpc.nu acl 3_tystpc.se hdr_end(host) -i tystpc.se use_backend SRV-WEB2_http if 3_tystpc.se backend SRV-WEB1_http modehttp balance roundrobin timeout connect 3 timeout server 3 retries 3 option httpchk server SRV-WEB1 192.168.2.2:80weight 100 backend SRV-WEB2_http modehttp balance roundrobin timeout connect 3 timeout server 3 retries 3 option httpchk server SRV-WEB2 192.168.2.3:80weight 100 --
Re: File uploads (multipart/form-data POST ) and transparent mode fail
Hi Magnus, I have integrated that 'transparent' option into the pfSense(FreeBSD8.3) haproxy-devel package. And can confirm that there is an issue when sending a large POST. For your information the config below does not contain the Transparent ClientIP option.. Which would read source 0.0.0.0 usesrc clientip.. Also on pfSense the main firewall is 'pf' , but to get 'transparent' traffic working it was needed to in the background also load and configure part of ipfw.. (this is also done for captive portal..) This so HAProxy gets to see the tcp traffic, and prevent replies from being routed out the wan interface.. The solution is to configure a floating rule like this: Action: Pass Quick: YES Interface: DMZ (the one pointing to your server..) Direction: Out Protocol: TCP Source: ANY Destination: Server-IP Destination: Server-PORT State Type: sloppy state Ill try and see if i can automate that, and if not at least put a warning note that this kind of rule must be added for it to work properly. Can you confirm this indeed solves the issue? Thanks PiBa-NL Magnus Thomé schreef op 18-1-2014 9:32: Last night during a couple of hours I took the time to read through the whole documentation from start to finish (instead of just doing keyword searches in it). But I really can't find anything. I set up option forceclose (and also option forwardfor which is unrelated) just to see if anything happened but nope. I wild guess from me as a total noob is that something is divided into 64kbyte chunks,being that a buffer, a particular set of packets or whatnot, and the first time 64kbytes is sent it goes through ok but the second and further chunks go astray. I've scratched my head wondering if the webserveror or possibly the pfsense box has anything set in connection with 64kB and of course also looked for anything like that in the haproxy documentation. There are no problems sending items larger than 64kB in the other normal direction, to the browsers. Will setting a cookie help? ANY help or pointers in some direction would be deeply appreciated /Magnus Thomé On Fri, Jan 17, 2014 at 4:50 PM, Magnus Thomé magn...@gmail.com mailto:magn...@gmail.com wrote: I've really really searched for answers, both in the mailing list archives and google but haven't been able to find anything. Would deeply appreciate any help! I'm running pfsense 2.1 with the only extra package installed being haproxy-devel 1.5-dev19 pkg v 0.6 EVERYTHING works great but one single thing: When doing a HTTP file upload with a FORM multipart/form-data POST to any server behind the firewall it only works with very small files, aprox max 60kbyte. With slightly larger files I get a timeout page after a while and with even larger files I get nothing at all. It seems that when Transparent ClientIP is enabled and set to DMZ the file uploads fail and with Transparent ClientIP disabled all works perfectly as it should. I do need the transparent mode though. Is there a setting somewhere I've missed? Thanks in advance for any possible help /Magnus -- The config created by pfsense GUI looks like this: global stats socket /tmp/haproxy.socket level admin uid 80 gid 80 nbproc 1 chroot /var/empty daemon frontend SRV-WEB1-merged bind 83.250.27.152:80 http://83.250.27.152:80 default_backend SRV-WEB1_http modehttp log global option dontlognull timeout client 3 acl 0_rejsa.nu http://0_rejsa.nuhdr_end(host) -i rejsa.nu http://rejsa.nu use_backend SRV-WEB1_http if 0_rejsa.nu http://0_rejsa.nu acl 1_rejsa.se http://1_rejsa.sehdr_end(host) -i rejsa.se http://rejsa.se use_backend SRV-WEB1_http if 1_rejsa.se http://1_rejsa.se acl 2_tystpc.nu http://2_tystpc.nu hdr_end(host) -i tystpc.nu http://tystpc.nu use_backend SRV-WEB2_http if 2_tystpc.nu http://2_tystpc.nu acl 3_tystpc.se http://3_tystpc.se hdr_end(host) -i tystpc.se http://tystpc.se use_backend SRV-WEB2_http if 3_tystpc.se http://3_tystpc.se backend SRV-WEB1_http modehttp balance roundrobin timeout connect 3 timeout server 3 retries 3 option httpchk server SRV-WEB1 192.168.2.2:80
