Re: Request for feedback: Add fetcher for getting all cookie names
Hi Ruei-Bang, On Fri, Oct 13, 2023 at 08:59:24PM +, Ruei-Bang Chen wrote: > Hi team, > > > This is related to my previous email regarding adding a fetcher for all > headers excluding cookies, but I think it might be helpful to open a separate > thread to discuss this alone as cookie is a special header. > > We have a use-case where we need only cookie names but not their values. > Currently, we have similar fetchers "req.hdr_names" / "res.hdr_names" for > returning all headers, but no such fetchers for cookies. > > Since we are only interested in cookie names and the cookie values can be > pretty long sometimes, we can avoid unnecessary overhead when we want to > store / send / parse the string variable if such fetchers are available for > cookies. > > The proposal is to add a new fetcher for extracting cookie names to reduce > overhead in high QPS. > > Let me know what you think. I don't have any particular opinion on this one. I agree that avoiding to retrieve the values can be useful. However, please keep in mind that the request and response code might differ due to the different syntax in both directions. Anyway, as long as you think you'll be having use cases for that and you're careful about performance and making it generic enough, I don't see any reason for objecting to this ;-) Thanks, Willy
Re: Request for feedback: Add fetcher for all headers excluding cookies
Hi Ruei-Bang, On Fri, Oct 13, 2023 at 08:54:24PM +, Ruei-Bang Chen wrote: > Hi Willy, > > Thank you for the great suggestion! I agree that it would be more general for > other use cases to have support for only including or excluding certain > headers. > > I can look into the implementation and send a patch for that if no other > concerns are there. OK! > We did try the lua implementation, but we observed there would be a lot more > malloc/free that impacted performance and took a lot of CPUs when QPS was > high (100k+). Ah yes, Lua is fine for low to moderate loads, but 100k+ starts to be a lot for it indeed! And having to manipulate lists and concatenate values definitely involves a lot of memory allocations so I can see how that is not suitable for your workload. Cheers, Willy
Request for feedback: Add fetcher for getting all cookie names
Hi team, This is related to my previous email regarding adding a fetcher for all headers excluding cookies, but I think it might be helpful to open a separate thread to discuss this alone as cookie is a special header. We have a use-case where we need only cookie names but not their values. Currently, we have similar fetchers "req.hdr_names" / "res.hdr_names" for returning all headers, but no such fetchers for cookies. Since we are only interested in cookie names and the cookie values can be pretty long sometimes, we can avoid unnecessary overhead when we want to store / send / parse the string variable if such fetchers are available for cookies. The proposal is to add a new fetcher for extracting cookie names to reduce overhead in high QPS. Let me know what you think. Thanks, Ruei-Bang
Re: Request for feedback: Add fetcher for all headers excluding cookies
Hi Willy, Thank you for the great suggestion! I agree that it would be more general for other use cases to have support for only including or excluding certain headers. I can look into the implementation and send a patch for that if no other concerns are there. We did try the lua implementation, but we observed there would be a lot more malloc/free that impacted performance and took a lot of CPUs when QPS was high (100k+). Ruei-Bang From: Willy Tarreau Sent: Thursday, October 12, 2023 8:56 PM To: Ruei-Bang Chen Cc: haproxy@formilux.org Subject: Re: Request for feedback: Add fetcher for all headers excluding cookies Hi Ruei-Bang, On Thu, Oct 12, 2023 at 09:44:22PM +, Ruei-Bang Chen wrote: > Hi team, > > I am writing to gather feedback on an idea before doing the implementation. > > > We have a use-case where we need all headers except for cookies. Currently, > the fetcher "req.hdrs" / "res.hdrs" returns all headers including cookie / > set-cookie headers. > > However, the cookie header itself can get pretty huge and cause unnecessary > overhead when we want to store / send / parse the string variable. > > In a high qps environment, this will cause big degradation to the latency of > the requests, so the proposal is to add a new fetcher for sending the headers > excluding cookies. > > Let me know what you think. If there are no concerns, I can send a patch. I'm not necessarily against it but I fear that it opens a can of worms. Others will say that User-Agent is often too verbose and will want to skip it, then it will be Referer, then X-Pad, then all Connection and Cache-control headers etc. So if we go in that direction, we'd need to keep the exclusion list configurable. For example we could imagine that req.hdrs takes an optional list of arguments that are either a restricted list of headers to extract, or to exclude depending on the syntax. I think we can support up to about 10 arguments so maybe that could do the job. For example: req.hdrs() => all req.hdrs(host,referer,x-forwarded-for) => only these req.hdrs(!cookie,referer,connection)=> all but these Alternately, have you tried to make your own sample-fetch function in Lua to deal with the headers you want ? I'm pretty sure that it was earlier used like this before becoming a native function. Thanks, Willy
Need a vacation? Let me write for you
Greetings, Starting a business is exciting, but there are several crucial considerations that budding entrepreneurs should keep in mind. Which gets me to a guest article proposal I have for your site: could I write for your readers about how to get rid of the not-so-fun aspects of beginning a business—for example, figuring out what to do with the money? obtaining funding, forming a limited liability company, establishing appropriate accounting standards, and so on? Please let me know your thoughts. Thank you very much! Katie Conroy / AdviceMine.com ~ By the way, if you’d like for me to write on a topic of your choice, please send your idea my way! Or, if you don’t want to hear from me again, just let me know.
Re: FYI: OpenWrt switches from wolfssl to mbedtls as default
On Fri, Oct 13, 2023 at 11:39:59AM +, Lukas Tribus wrote: > Hello, > > > an interesting move from the OpenWRT project: > > > > Switch from wolfssl to mbedtls as default > > = > > > > OpenWrt has transitioned its default cryptographic library from wolfssl > > to mbedtls. This shift brings several changes and implications: > > > > * Size Efficiency: mbedtls is considerably smaller, making it an > > optimal choice for systems where storage space is paramount. > > * LTS and ABI Stability: mbedtls consistently provides updates via its > > Long Term Support (LTS) branch, ensuring both security and a stable > > application binary interface (ABI). In contrast, wolfssl does not > > offer an LTS release, and its stable ABI is limited to a specific set > > of functions. > > * TLS 1.3 Support: Users should be aware that mbedtls 2.28 no longer > > supports TLS 1.3. > > > > While mbedtls is now the default, users who have specific needs or > > preferences can still manually switch back to wolfssl or choose openssl. > > As per: > http://lists.openwrt.org/pipermail/openwrt-announce/2023-October/47.html > > Size Efficiency does not matter a lot in the context of haproxy, It depends, OpenSSL uses a lot of memory and a smaller footprint is interesting when you want more simultaneous sessions. > and TLSv1.3 is a must-have, I don't get this point, mbedtls supports TLS 1.3 so I don't really understand what they are talking about. Maybe they disable TLS 1.3 on the 2.28 version for some rease but I couldn't find any detail. > but I'm surprisedFYI about the point about LTS > and ABI Stability in wolfssl and I'm wondering if this is really the > case? > To be honest I don't know well enough the wolfssl release cycle, but they don't announce publicly LTS versions for their opensource versions, if you want that you must have commercial contract. Regarding the ABI, it seems to me that are a lot of changes, but it makes sense since there is no LTS> We are using the openssl compatibility API for HAProxy, the API is not supposed to change a lot, but the ABI change fromt time to time. -- William Lallemand
FYI: OpenWrt switches from wolfssl to mbedtls as default
Hello, an interesting move from the OpenWRT project: > Switch from wolfssl to mbedtls as default > = > > OpenWrt has transitioned its default cryptographic library from wolfssl > to mbedtls. This shift brings several changes and implications: > > * Size Efficiency: mbedtls is considerably smaller, making it an > optimal choice for systems where storage space is paramount. > * LTS and ABI Stability: mbedtls consistently provides updates via its > Long Term Support (LTS) branch, ensuring both security and a stable > application binary interface (ABI). In contrast, wolfssl does not > offer an LTS release, and its stable ABI is limited to a specific set > of functions. > * TLS 1.3 Support: Users should be aware that mbedtls 2.28 no longer > supports TLS 1.3. > > While mbedtls is now the default, users who have specific needs or > preferences can still manually switch back to wolfssl or choose openssl. As per: http://lists.openwrt.org/pipermail/openwrt-announce/2023-October/47.html Size Efficiency does not matter a lot in the context of haproxy, and TLSv1.3 is a must-have, but I'm surprisedFYI about the point about LTS and ABI Stability in wolfssl and I'm wondering if this is really the case? cheers, lukas
Re: HA Proxy
Hi Mohammed. Yes HAProxy supports all of the requested capacity and features from below. For a nice example what HAProxy is able to handle can you read this Blog post. https://www.haproxy.com/blog/haproxy-forwards-over-2-million-http-requests-per-second-on-a-single-aws-arm-instance The very detailed Documentation can be found in the Web https://docs.haproxy.org/ or in the source repository under the doc directory https://git.haproxy.org/?p=haproxy.git;a=tree;f=doc;h=9a53977a683fd7e80f23fff2a18ef192ca908636;hb=HEAD There are very good examples and explanations for HAProxy features on the HAProxy com Blog page https://www.haproxy.com/blog and you can also find some examples with your favorite Search engine. Please take care that some search results refer to some previous HAPRoxy Versions which are not maintained anymore, this means that the founded solution could work or need some rework for the current versions. HAProxy have two versions the Opensource one and the Enterprise one. If your company want support and is willing to pay for that can you get in touch with HAProxy Sales via the contact from https://www.haproxy.com/contact-us for the HAProxy Enterprise version https://www.haproxy.com/products/haproxy-enterprise. Hth with best regards Alex On 2023-10-13 (Fr.) 09:41, Mohammed Anees A wrote: Hi Team We have a requirement to for a Software based NLB to Load Balance an enterprise application. Following are the Capacity and Features of NLB required. Please confirm, does HA Proxy supports the below capacity and features ?. let us know the licensing model and Support structure. Capacity : * Requests per Second = 5000 RPS * Concurrent Connections = 5000 Concurrent Sessions. * Throughput = 40 Mbps Features : 1. *Routing Profile * Routing profile can be TCP based (layer 4) or HTTP based (layer 7). ** 2. *Load Balancing Method* All load balancing methods are supported. It is recommended to use Least Connections or Round Robin load balancing methods, for better distribution between Application servers. 3. *Session persistence (stickiness)* The LB must be configured with session persistence to enable a session connection with the same application server instance. Configure session persistence in all levels of load balancing (for example, if there is a global load balancer in front of a few local load balancers). To achieve session persistence, configure the LB with one of the following persistence profiles: * HTTP Cookie * Client IP (Source address) 4. *Health monitoring* ** An important property of an LB is the ability to perform health monitoring checks (heartbeats) on each Application server. By using health monitors, the LB verifies the server response or checks for any network problems that can prevent a client from reaching a server. By doing so, the LB can place the server in or out of service and can make reliable load-balancing and high availability decisions.** A common and recommended health monitor is *HTTP GET Request.*** 5. *Idle (execution) timeout* Setting the execution timeout controls termination of idle connections. Configure an execution timeout of at least 4 hours. 6. *HTTPS Configuration* The load balancer supports several HTTPS configuration methods. These include: * SSL bridging * SSL offload * SSL pass-through SSL bridging and SSL offload are supported in HTTP based routing (layer 7), and require deploying TLS certificate on the LB. SSL pass-through is supported in TCP based routing (layer 4), and does not require deploying a certificate on the LB. Regards Mohammed Anees +91 9944170656
HA Proxy
Hi Team We have a requirement to for a Software based NLB to Load Balance an enterprise application. Following are the Capacity and Features of NLB required. Please confirm, does HA Proxy supports the below capacity and features ?. let us know the licensing model and Support structure. Capacity : - Requests per Second = 5000 RPS - Concurrent Connections = 5000 Concurrent Sessions. - Throughput = 40 Mbps Features : 1. *Routing Profile * Routing profile can be TCP based (layer 4) or HTTP based (layer 7). 1. *Load Balancing Method* All load balancing methods are supported. It is recommended to use Least Connections or Round Robin load balancing methods, for better distribution between Application servers. 1. *Session persistence (stickiness)* The LB must be configured with session persistence to enable a session connection with the same application server instance. Configure session persistence in all levels of load balancing (for example, if there is a global load balancer in front of a few local load balancers). To achieve session persistence, configure the LB with one of the following persistence profiles: - HTTP Cookie - Client IP (Source address) 1. *Health monitoring* An important property of an LB is the ability to perform health monitoring checks (heartbeats) on each Application server. By using health monitors, the LB verifies the server response or checks for any network problems that can prevent a client from reaching a server. By doing so, the LB can place the server in or out of service and can make reliable load-balancing and high availability decisions. A common and recommended health monitor is *HTTP GET Request.* 1. *Idle (execution) timeout* Setting the execution timeout controls termination of idle connections. Configure an execution timeout of at least 4 hours. 1. *HTTPS Configuration* The load balancer supports several HTTPS configuration methods. These include: - SSL bridging - SSL offload - SSL pass-through SSL bridging and SSL offload are supported in HTTP based routing (layer 7), and require deploying TLS certificate on the LB. SSL pass-through is supported in TCP based routing (layer 4), and does not require deploying a certificate on the LB. Regards Mohammed Anees +91 9944170656
