Hi Jinho, On Mon, Oct 12, 2026 at 11:25:50AM +0900, Zino wrote: > Hi Willy, > > Thanks a lot for taking the time, and sorry for my own slow reply. > > On Thu, Sep 17, 2026, Willy Tarreau wrote: > > However I'm wondering how long it will survive the "exact match" of > > the selector. Some users will come asking for network match, others > > with case insensitive match etc. > > I thought about this a lot, and I think the exact match is the right > thing for the selector, because the selector is not a lookup key: it > names the scope an entry belongs to (a site, a backend, a tenant). > Scopes are discrete.
I was having other use cases in mind that are relatively common, which consist in setting access restrictions (auth or rate) per (ip, url). Also, even for backends, when you're running with hundreds to thousands of them, it's frequent to want to group them by categories (e.g. those from customer A, those from customer B, then those for customers of class C etc) and that's where matching by begging of key (for example) could make sense. > Matching on a scope by prefix or by network would > make an entry belong to several scopes at once, depending on the > request, and that is hard to reason about. Usually for such things, it works by longest match, exaactly like IP addresses, with a longest match (i.e. map_beg). You could name the selector with "<class>_<customer>_" and look your class+customer up there. If the pair is found it's precise; if not found you at least match the class. (...) > We used to run one set of map files per backend: a bit more than 100 > backends with around 8 maps each, so roughly 800 map files, each with > its own rules in the configuration. Every one of these files is > opened, parsed and indexed when the new process starts, and at that > count a reload alone took 2 to 3 minutes. We believe the cost was > mostly per file rather than per entry, and that is what made us look > for a way to fold them into a few files. Wow, this is huge! How is this possible ? Were your map files large ? And if so, what did result in that time shrinking once you moved all the keys into a same file ? Because that's even more data to parse, there's no grouping at all! > Chained maps would bring this back: every map that can be named at > run time must already be loaded at boot with the right type, so > "one map per selector" still means as many files to load on every > reload as there are selectors. The lookups would be smaller, but the > reload cost is paid on the number of files, not on their size. I understand but I would like to be certain we're attacking the right problem. 3 minutes for 800 files is not expected, especially if the same number of keys into a single file is faster. It's possible that we're having some linear map name lookups that could possibly be changed, I don't know, but if you could at least run perf record on "haproxy -c -f config.cfg" then share the output of "perf report" so that we figure where that time is spent, it would be nice. > > And when two input keys result in the same mappings, you wouldn't > > even have to duplicate them. > > Agreed, this is the real advantage of chaining. In our case the shared > part was usually one list used by all scopes (e.g. known abusive > addresses), and putting it under its own scope and looking it up as a > second step was enough. I don't claim this covers every case though. Of course, and my goal is to make sure that either very similar cases are properly handled, or that they arenot too hard to later add when someone else requests them. > To sum up: I still believe the grouped map is the simplest answer to > the "one file, several independent lists" problem, and that it > composes with the existing converters for the non-exact cases. I'd be > glad to hear if you see a case where it falls short. Yes but I tend to be cautious about simplest solutions, as they were often at the root of multi-year problems that we can no longer fix because they still address some use cases. That's why I need to go down to the details about it (it's not to annoy you, I'd really like such a feature to get merged). thanks! Willy
