clintropolis opened a new pull request #7222: refactor lookups to be more chill to router URL: https://github.com/apache/incubator-druid/pull/7222 This PR introduces `RouterLookupModule` which allows the router to perform query serde but without having to load lookups as happens using the stock `LookupModule`. To achieve this, an interface has been extracted from `LookupReferencesManager`, the very java named `LookupExtractorFactoryContainerProvider`, which is now used everywhere `LookupReferencesManager` was previously used, _except_ for `LookupListeningResource`. I think this is a bit nicer, because now all query processing stuff no longer has obvious access to methods to load and drop lookups, only the method to get the `LookupExtractorFactoryContainer` needed to do the transforms. An integration test has been added to the wiki tests, which loads a simple json lookup, and tests queries against brokers and routers. This also means there is now an integration test that is testing lookups! The router will proxy requests to the lookup introspection API it was previously serving, so if the router was being queried to introspect lookups things should still function, it will just be proxied to a default broker now. I suspect it wasn't _too much_ of an issue, because that API didn't appear functional because didn't have auth checks performed, and would throw an exception of the form ``` 2019-03-10T00:15:20,298 WARN [qtp2080864181-118] org.eclipse.jetty.server.HttpChannel - /druid/v1/lookups/introspect/simple-wiki org.apache.druid.java.util.common.ISE: Request did not have an authorization check performed. at org.apache.druid.server.security.PreResponseAuthorizationCheckFilter.handleAuthorizationCheckError(PreResponseAuthorizationCheckFilter.java:156) ~[classes/:?] at org.apache.druid.server.security.PreResponseAuthorizationCheckFilter.doFilter(PreResponseAuthorizationCheckFilter.java:89) ~[classes/:?] ... ``` So I've included a fix for that as well. I also made some minor adjustments to the integration tests configurations, switching gc from cms to g1, and lowering some thread pool sizes to maybe result in a slightly lighter footprint.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
