On Wed, Sep 29, 2021 at 02:17:59PM +1000, Andrew Lemin wrote: > I see this question died on its arse! :) > > This is still an issue for outbound load-balancing over multiple internet > links. > > PF's 'sticky-address' parameter only works on source IPs (because it was > originally designed for use when hosting your own server pools - inbound > load balancing). > I.e. There is no way to configure 'sticky-address' to consider destination > IPs for outbound load balancing, so all subsequent outbound connections to > the same target IP originate from the same internet connection. > > The reason why this is desirable is because an increasing number of > websites use single sign on mechanisms (quite a few different architectures > expose the issue described here). After a users outbound connection is > initially randomly load balanced onto an internet connection, their browser > is redirected into opening multiple additional sockets towards the > website's load balancers / cloud gateways, which redirect the connections > to different internal servers for different parts of the site/page, and the > SSO authentication/cookies passed on the additional sockets must to > originate from the same IP as the original socket. As a result outbound > load-balancing does not work for these sites. > > The ideal functionality would be for 'sticky-address' to consider both > source IP and destination IP after initially being load balanced by > round-robin or random.
Just use multipath routing, it will make sure that selected default routes are sticky to source/dest pairs. You may want the states to be interface bound if you need to nat-to on those links. On rerouting the multipath code reshuffles the selected routes in a way to minimize the affected sessions. All this is done without any extra memory usage since the hashing function is smart. -- :wq Claudio > Thanks again, Andy. > > On Sat, Apr 3, 2021 at 12:40 PM Andy Lemin <andrew.le...@gmail.com> wrote: > > > Hi smart people :) > > > > The current implementation of ‘sticky-address‘ relates only to a sticky > > source IP. > > https://www.openbsd.org/faq/pf/pools.html > > > > This is used for inbound server load balancing, by ensuring that all > > socket connections from the same client/user/IP on the internet goes to the > > same server on your local server pool. > > > > This works great for ensuring simplified memory management of session > > artefacts on the application being hosted (the servers do not have to > > synchronise the users session data as extra sockets from that user will > > always connect to the same local server) > > > > However sticky-address does not have an equivalent for sticky destination > > IPs. For example when doing outbound load balancing over multiple ISP > > links, every single socket is load balanced randomly. This causes many > > websites to break (especially cookie login and single-sign-on style > > enterprise services), as the first outbound socket will originate randomly > > from one of the local ISP IPs, and the users login session/SSO (on the > > server side) will belong to that first random IP. > > > > When the user then browses to or uses another part of that same website > > which requires additional sockets, the additional sockets will pass the SSO > > credentials from the first socket, but the extra socket connection will > > again be randomly load-balanced, and so the remote server will reject the > > connection as it is originating from the wrong source IP etc. > > > > Therefore can I please propose a “sticky-address for destination IPs” as > > an analogue to the existing sticky-address for source IPs? > > > > This is now such a problem that we have to use sticky-address even on > > outbound load-balancing connections, which causes internal user1 to always > > use the same ISP for _everthing_ etc. While this does stop the breakage, it > > does not result in evenly distributed balancing of traffic, as users are > > locked to one single transit, for all their web browsing for the rest of > > the day after being randomly balanced once first-thing in the morning, > > rather than all users balancing over all transits throughout the day. > > > > Another pain; using the current source-ip sticky-address for outbound > > balancing, makes it hard to drain transits for maintenance. For example > > without source sticky-address balancing, you can just remove the transit > > from the Pf rule, and after some time, all traffic will eventually move > > over to the other transits, allowing the first to be shut down for whatever > > needs. But with the current source-ip sticky-address, that first transit > > will take months to drain in a real-world situations.. > > > > lastly just as a nice-to-have, how feasible would a deterministic load > > balancing algorithm be? So that balancing selection is done based on the > > “least utilised” path? > > > > Thanks for your time and consideration, > > Kindest regards Andy > > > > > > > > Sent from a teeny tiny keyboard, so please excuse typos. > >
