Hey folks, I'm currently working on this TO API routing blacklist feature, and while I've identified which routes should be on the whitelist of "routes that have been rewritten to Go but that are still safe to fall back to Perl for", the tediousness of copying route regular expressions for the whitelist has given me another idea.
Rather than have the configuration be based on regular expressions that are meant to match the actual routes in the code, I was thinking of giving each route an ID, including the ID as part of the Route struct, and making the configuration based on these IDs instead of trying to mirror the regex. That way, you can't accidentally disable routes or have Perl handle routes you didn't mean to from writing a bad regular expression. Essentially, every route would get a unique ID that can be referenced in the config for either disabling it or routing it to Perl. Whether or not a Go route would be routable to Perl would also become part of the Route struct. TO would print the route IDs on startup (so you can easily find them and match them to the routes you're trying to disable or fall back to Perl) and verify that the actual given route IDs are unique (to ensure that IDs stay unique as routes are moved around or new routes are added). What do you think? Stick to regular expressions, or go with this IDea instead (see what I did there)? - Rawlin On Fri, Nov 1, 2019 at 4:01 PM Gray, Jonathan <[email protected]> wrote: > > I largely don't care about the blacklisted routes for this purpose. I really > care about a conclusive list of whitelisted routes (for which the example > json payload could be expanded to carry). It seems like we're solving the > exact same issue from two directions. It permits each native client library > to assert that the routes it expects and needs to exist, exist on the other > side. I have no desire to actively modify the runtime routes (for security I > don't think we every should), just to get the list of what it had at startup. > Having the override config file on disk to switch on/off independent > route/methods is something I'd expect to have to restart TO for (no different > than changes in the cdn.conf). I do also agree with proper 503 handling, but > it allows us to perform a basic sanity check to prevent half-completed > workflows necessitating complex recovery paths. For applications that use > the client SDK, it gives an easy handle to know if every single upgrade > necessitates recompiling and deploying 3rd party applications, such as a CZF > File generator. > > Jonathan G > > > On 11/1/19, 1:49 PM, "Rawlin Peters" <[email protected]> wrote: > > > Not trying to sideswipe, but could we expose that as an endpoint with a > Golang list as well to solve: > https://protect2.fireeye.com/url?k=5d57b4c9-01b3ba02-5d57937d-000babff3540-17d7cedf2908de8b&u=https://github.com/apache/trafficcontrol/issues/2872 > > While I do agree with the request for an API endpoint that tells the > client what API versions are supported, I wouldn't want to > overcomplicate *this* particular feature with an API endpoint to > expose the information that is in the config file. > > If we implement that kind of "API information" API endpoint, I > wouldn't be opposed to including the currently blacklisted routes in > its response as a minor goal, but I don't really think it's warranted > by this routing blacklist feature alone. You should have a really, > really good reason to blacklist a route or bypass a TO-Go route for > the Perl, so this should be a (hopefully) relatively rare operation to > begin with. I don't think it would be all that useful for API clients > to be able to see the list of currently blacklisted APIs. The API > client should be written to properly handle 503s whenever they occur, > and to the client it shouldn't matter if the 503 is from the database > being overloaded at the time or if the route is blacklisted. > > - Rawlin > >
