Hello Uday, Quick follow-up as I realize that templating can be a bit scary when confronted for the first time, and you are far from the first one to be curious about, so I've committed this: https://github.com/varnish/toolbox/tree/master/gotemplate-example It probably won't get you very far, but it should at least get you started, and help understand how templating can make things a tiny be simpler but splitting data from business logic, for example to add more IPs/ACLs or source without edit the VCL manually.
Hope that helps. -- Guillaume Quintard On Thu, Oct 12, 2023 at 12:36 PM Uday Kumar <[email protected]> wrote: > > That's mainly how computers work, processing will be linear. You *could* > create a vmod that packs ACLs into a hashmap to simplify the apparent > logic, but you will pay that price developing the vmod, and for a very > modest performance gain. If you have less than 50 sources, or even less > than 100, I don't think it's worth agonizing over that kind of optimization > (unless you've actually measured and you did see a performance drop). > > Okay, Thanks for your suggestion! > > > I assume that the VCL is currently committed in a repo somewhere and > gets edited every time you need to add a new IP or source. If so, it's not > great because editing such repetitive code is error-prone, and therefore > you should use templating to create the VCL from a simpler, more > maintainable source. > > Sure, will definitely explore! > > Thanks & Regards > Uday Kumar > > > On Fri, Oct 13, 2023 at 12:35 AM Guillaume Quintard < > [email protected]> wrote: > >> > In the above example, if the request URL is source=tablet [for which >> condition is present at the end], still I have to check all the above >> conditions. >> >> That's mainly how computers work, processing will be linear. You *could* >> create a vmod that packs ACLs into a hashmap to simplify the apparent >> logic, but you will pay that price developing the vmod, and for a very >> modest performance gain. If you have less than 50 sources, or even less >> than a 100, I don't think it's worth agonizing over that kind of >> optimization (unless you've actually measured and you did see a >> performance drop). >> >> > One thing I would do though is to generate the VCL from a source file, >> like a YAML one: >> >> All I'm saying is that you should focus on increasing the maintainability >> of the project before worrying about performance. I assume that the VCL is >> currently committed in a repo somewhere and gets edited every time you need >> to add a new IP or source. If so, it's not great because editing such >> repetitive code is error-prone, and therefore you should use templating to >> create the VCL from a simpler, more maintainable source. >> >> Tools like go templates or jinja can provide that feature and save you >> from repeating yourself when writing configuration. >> >> -- >> Guillaume Quintard >> >> >> On Thu, Oct 12, 2023 at 11:46 AM Uday Kumar <[email protected]> >> wrote: >> >>> Hi Guillaume, >>> >>> I don't think those are redundant checks, from what you are showing, >>> they are all justified. Sure, there may be a bunch of them, but you have to >>> go through to them. >>> >>> By redundant I meant, I have to write multiple checks for each source >>> and list of IPs associated with it. [which would be *worse *if the >>> number of sources are huge] >>> >>> *Example:* >>> >>> If( >>> >>> (req.url ~ "source=mobile" && client.ip != mobile_source) || >>> >>> (req.url ~ "source=desktop" && client.ip != desktop_source) || >>> >>> (req.url ~ "source=laptop" && client.ip != laptop_source) || >>> >>> (req.url ~ "source=tablet" && client.ip != tablet_source) >>> >>> ){ >>> >>> return(Synth(403, "access denied!")) >>> >>> } >>> >>> >>> In the above example, if the request URL is source=tablet *[for which >>> condition is present at the end]*, still I have to check all the above >>> conditions. >>> >>> >>> >>> >>> >>> One thing I would do though is to generate the VCL from a source file, >>> like a YAML one: >>> >>> Didn't understand, can you please elaborate? >>> >>> Thanks & Regards >>> Uday Kumar >>> >>> >>> On Thu, Oct 12, 2023 at 11:11 PM Guillaume Quintard < >>> [email protected]> wrote: >>> >>>> Hi Uday, >>>> >>>> I don't think those are redundant checks, from what you are showing, >>>> they are all justified. Sure, there may be a bunch of them, but you have to >>>> go through to them. >>>> >>>> One thing I would do though is to generate the VCL from a source file, >>>> like a YAML one: >>>> >>>> mobile: >>>> - IP1 >>>> - IP2 >>>> - IP3 >>>> desktop: >>>> - IP4 >>>> - IP5 >>>> - IP6 >>>> >>>> >>>> From that, you can build the VCL without having to manually write >>>> "client.ip" or "(req.url ~ "source=" every time. >>>> >>>> -- >>>> Guillaume Quintard >>>> >>>> >>>> On Thu, Oct 12, 2023 at 10:17 AM Uday Kumar <[email protected]> >>>> wrote: >>>> >>>>> Hello everyone, >>>>> >>>>> We use varnish in our production environment for caching content. >>>>> >>>>> Our Requirement: >>>>> >>>>> We are trying to block unauthorized requests at varnish based on the >>>>> source parameter in the URL and the client IP in the request header. >>>>> >>>>> For example: >>>>> >>>>> Sample URL: >>>>> >>>>> www.hostname:port/path?source=mobile&keyword= bags >>>>> >>>>> Let's assume there are 3 IPs [which are allowed to access varnish] >>>>> associated with the above request of mobile source. >>>>> >>>>> i.e *IP1, IP2, IP3* >>>>> >>>>> So if any request comes with the source as *mobile *and client-ip as >>>>> *IP4*, it's treated as an unauthorized request and should be blocked >>>>> at varnish. >>>>> >>>>> >>>>> What we have done for blocking? >>>>> >>>>> *Sample URL:* >>>>> www.hostname:port/path?source=mobile&keyword= bags >>>>> >>>>> Created a map using ACL as below: >>>>> >>>>> acl mobile_source{ >>>>> >>>>> "IP1"; >>>>> >>>>> "IP2"; >>>>> >>>>> "IP3"; >>>>> >>>>> } >>>>> >>>>> If(req.url ~ "source=mobile" && client.ip !~ mobile_source) { >>>>> >>>>> return(Synth(403, "varnish access denied!")) >>>>> >>>>> } >>>>> >>>>> >>>>> The problem we are facing: >>>>> >>>>> The source parameter can have different values like mobile, desktop, >>>>> laptop, tablet, etc. and each value can have different IPs associated with >>>>> it. >>>>> >>>>> ACL Rules will be as below: >>>>> >>>>> acl mobile_source{ >>>>> >>>>> "IP1"; >>>>> >>>>> "IP2"; >>>>> >>>>> "IP3"; >>>>> >>>>> } >>>>> >>>>> acl desktop_source{ >>>>> >>>>> "IP4"; >>>>> >>>>> "IP5"; >>>>> >>>>> "IP6"; >>>>> >>>>> } >>>>> >>>>> and so on, >>>>> >>>>> >>>>> If we wanted to block unauthorized access from different source vs IP >>>>> combinations, we would have to add that many conditions as below. >>>>> >>>>> If( >>>>> >>>>> (req.url ~ "source=mobile" && client.ip != mobile_source) || >>>>> >>>>> (req.url ~ "source=desktop" && client.ip != desktop_source) || >>>>> >>>>> (req.url ~ "source=laptop" && client.ip != laptop_source) || >>>>> >>>>> (req.url ~ "source=tablet" && client.ip != tablet_source) >>>>> >>>>> ){ >>>>> >>>>> return(Synth(403, "access denied!")) >>>>> >>>>> } >>>>> >>>>> This becomes worse, if we have 10's or 20's of source values. >>>>> >>>>> Our question: >>>>> >>>>> We would like to know if there is any way to optimize the code by >>>>> removing redundant checks so that we can scale it even if we have many >>>>> sources vs IP combinations. >>>>> >>>>> >>>>> Thanks & Regards >>>>> Uday Kumar >>>>> _______________________________________________ >>>>> varnish-misc mailing list >>>>> [email protected] >>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>>>> >>>>
_______________________________________________ varnish-misc mailing list [email protected] https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
