I would vote for option1. I think also you’ll get better feedback as the Ranger folks know Ranger and we know Drill.
> On Aug 30, 2026, at 20:42, huafengshi <[email protected]> wrote: > > hi Charles, drill devs: > I’d like to share the conclusion from my discussion with the Apache Ranger > community for Drill‑Ranger integration, together with our restructured code > modules. I also have one key question to seek your advice. > Conclusion from Ranger community > Following the modern integration pattern adopted by Ranger for new components > (Trino, Polaris), the runtime Java authorization plugin shall be maintained > within the upstream project’s own repository. Only the service‑definition > JSON file will be hosted in the Apache Ranger repo. This avoids cross‑project > dependency pain, JDK version conflicts and release‑cycle coupling, compared > with the legacy Presto model where the full plugin resided inside Ranger > repository. > Accordingly, the Drill‑Ranger runtime plugin will live entirely in the Apache > Drill codebase and be maintained by the Drill community. > Restructured code module layout > drill ├── exec │ ├── java‑exec # Drill authorization invocation point in > query execution flow │ └── security‑spi # New generic authorization SPI > contract. │ It can be implemented by Ranger or other security frameworks. └── > drill‑ranger [drill‑ranger‑parent] # Ranger‑specific modules, isolated with > Ranger‑only dependencies ├── drill‑ranger‑plugin # Main runtime plugin, > implements security‑spi: authorization, resource mapping, auditing ├── > drill‑ranger‑plugin‑shim # Shim layer for compatibility across different > Ranger client versions └── drill‑ranger‑service # Temporary home for > ranger‑servicedef‑drill.json service‑definition file └── pom.xml > Design note: All modules under drill‑ranger carry only Ranger‑related > dependencies. Once we move the service‑definition file to Ranger repository > later, we only need to remove the drill‑ranger‑service module with minimal > code changes. > Key question for PMC feedback > For phase‑1 implementation: > Only basic access types are supported: CREATE, SELECT, DROP. > Row‑level filter and data masking are not yet implemented. > We have two options regarding where to place the ranger‑servicedef‑drill.json > service‑definition file. Please advise which approach we should take. > Option 1: Submit the current partial service‑definition directly to Apache > Ranger repo now > Pros: Fully follows Ranger’s official recommended integration pattern. Ranger > Admin can recognize Drill service and allow policy configuration > out‑of‑the‑box. > Cons: The service‑definition will lack entries for row‑level filter and > masking. Later we need to submit follow‑up PRs to Ranger to extend the > service definition when those features are ready. > Option 2: Keep service‑definition inside Drill repo in drill‑ranger‑service > for now > Pros: We can contribute a complete service‑definition with all access types, > row‑level filter and masking in one‑shot when fully implemented. No multiple > subsequent PRs against Ranger repo. > Cons: Does not align with Ranger’s best practice. End‑users will need to > manually load the Drill service‑definition file. > Important note: Whichever option we choose for service‑definition, the Ranger > runtime Java plugin will always stay within Drill repository and owned by > Drill community, which has been confirmed with Ranger community. > The drill‑security‑spi is designed as a generic authorization contract, so > other security frameworks can implement this SPI besides Ranger in the future. > Looking forward to your suggestions. > Best regards, Huafeng Shi > ---- Replied Message ---- > FromMadhan Neethiraj<[email protected]>Date8/30/2026 > 23:[email protected]<[email protected]>[email protected]<[email protected]>SubjectRe: > Subject: [DISCUSS] Integrate Apache Drill with Apache Ranger via > authorization SPI > Hi Huafeng Shi, > Thank you for reaching out to Ranger community and the providing appropriate > details. > Here is the preferred integration approach from Ranger community: > 1. Service-definition, describing the resources and access-types, is > committed in > Apache Ranger git repo, so that it is automatically registered during Ranger > admin server bootstrap. > 2. Plugin implementation itself remains in Apache Drill git repo > With this approach, Apache Drill community will avoid dependency on Apache > Ranger > releases for any enhancements in authorizer implementation - for example > changes in > authorization SPI or fixes. This will also help avoid having to deal with JDK > version > differences across the projects. > Here are few references for this integration model: > - Trino: https://github.com/trinodb/trino/tree/master/plugin/trino-ranger > - Polaris: https://github.com/apache/polaris/tree/main/extensions/auth/ranger > Blog on "Integrating Applications with Apache Ranger" can be a useful > reference as you > look into integration: > https://ranger.apache.org/blogs/integrating_applications.html. > Thanks, > Madhan > On 8/30/26, 4:55 AM, "huafengshi" <[email protected] > <mailto:[email protected]>> wrote: > Hi Ranger Devs, > I'm Huafeng Shi, a Committer of Apache Drill. I’m working on authorization > integration between Apache Drill and Apache Ranger, and would like to start a > discussion with the community. > Here is my proposed design: > Apache Drill will expose a generic authorization SPI (contract) for access > control. This SPI is not yet released, I am preparing to submit the > implementation to Drill. > Based on this SPI, we can build a Ranger‑specific authorization plugin for > Drill. > Two possible options for hosting this Ranger‑Drill plugin: > Option A: Contribute the Drill Ranger plugin into Apache Ranger codebase > (target Ranger 3.0). > Option B: Maintain the plugin within Apache Drill repository, once the Drill > authorization SPI is finalized and released. > I would like to clarify several key questions with the Ranger community: > Does the Ranger community accept this overall integration approach: Drill > provides authorization SPI contract, Ranger supplies the plugin > implementation to enforce access control for Drill? > If Ranger community is willing to host this Drill plugin inside Ranger repo > for Ranger 3.0: > I will collaborate with Drill community to finalize and formalize the > authorization SPI in Drill first. > Then I will contribute the Drill‑Ranger plugin code to Ranger. > If the plugin is accepted for Ranger 3.0, are there any requirements for SPI > stability, testing scope, documentation, compatibility we need to follow? > If Ranger prefers not to host Drill plugin inside Ranger repository, is there > any guidance for maintaining it in Drill side as a third‑party Ranger plugin? > Looking forward to your feedback. Thanks a lot. > Best regards, > Huafeng Shi > ---- Replied Message ---- > FromCharles Givre<[email protected] <mailto:[email protected]>>Date8/29/2026 > 05:30To<[email protected] <mailto:[email protected]>>SubjectRe: > [DISCUSS] Package namespace and module ownership for Drill-Ranger integration > (DRILL-8548) > Huafeng, Thanks for this. Have you contacted the Ranger community and asked > this question of them? Are they willing to accept your plugin? It would be > helpful to know this. > For the record, I think this integration will be very valuable to Drill once > complete. > Best, > — Charles > On Aug 27, 2026, at 09:40, huafengshi <[email protected] > <mailto:[email protected]>> wrote: > Dear Drill developers, > First of all, thank you for the detailed review and for raising the > governance‑level issue regarding the Ranger integration PR. I would like to > open a discussion on this matter with the community. > Background The PR currently adds two modules – drill-ranger-plugin and > drill-ranger-service – and all Java code within them resides under the > org.apache.ranger.* package namespace. This potentially raises two issues: > Placing Drill‑specific code under org.apache.ranger.* means that Drill > releases artifacts occupying another ASF project’s namespace. This brings > questions such as: > Who owns, versions, and applies CVE patches to these classes? > Is this acceptable from an ASF policy perspective? > This practice deviates from the established convention in the Apache Ranger > ecosystem. All similar service plugins (e.g., ranger-hive-plugin, > ranger-hbase-plugin, ranger-kafka-plugin, ranger-presto-plugin) are > maintained and released directly by the Ranger PMC, which keeps the package > namespace and release ownership unified within a single project. > Current proposals At this point, we see two high‑level approaches: > Option A: Keep the modules as an independent plugin, rename the package to a > Drill‑owned namespace, and have Drill manage the plugin entirely without > being constrained by Ranger’s release cycle. > Option B: Wait for official feedback from the Ranger community to decide > whether they are willing to accept this plugin as part of their project. > Detailed analysis > Issue 1 (namespace ownership) The solution is straightforward: keep both > modules in Drill but rename the package to > org.apache.drill.exec.security.ranger (or similar), making it clear that this > code belongs to Drill and is maintained by the Drill community. > Issue 2 (alignment with Ranger ecosystem) Moving the drill‑ranger code to > Ranger, where it would be maintained by the Ranger PMC, depends critically on > the attitude of the dev@ranger community. I would like to discuss a few > possible directions with everyone. > (1) What parts of this PR truly belong to Drill? > Sub‑option 1 – Recommended: Drill retains only the generic authorization > contract, and the Ranger‑specific implementation is moved out. Specifically, > the following should stay in the Drill repository: > AccessAuthorizer SPI – the interface contract that any authorization engine > can implement (not biased toward Ranger) > AccessTypes – access type constants > NoOpAccessAuthorizer – the default no‑op implementation, which also > demonstrates the neutrality of the SPI > AccessAuthorizerFactory – the standard ServiceLoader‑based discovery > mechanism (fail‑closed if no provider is found) > ColumnAccessChecker call sites – the integration points within the execution > framework > The parts to be moved out of Drill include: policy engine integration, > RangerAdmin client, auditing, and RangerServiceDrill (the admin‑side service > definition). > With this split, Drill core has zero org.apache.ranger dependencies and can > be released independently. > However, this sub‑option raises several practical concerns that need to be > addressed: > SPI dependency for Ranger: The Ranger plugin would depend on Drill’s > AccessAuthorizer SPI. Drill would only need a one‑time bootstrapping release: > first release a version containing the SPI, then Ranger follows up with the > plugin release. Documentation would clearly state that the actual > authorization capability is provided by the Ranger plugin at installation > time. Is this release ordering feasible? > JDK version mismatch: Ranger currently runs on JDK 8, while Drill uses JDK > 17+. > Approach a (recommended): Extract a minimal, independent module > drill‑auth‑spi from java‑exec, compile it with --release 8, and ship it with > Drill. This is exactly what Presto does – presto‑spi is a small, separate > artifact published to Central, and the Ranger Presto shim depends only on > that, not on the entire Presto engine. A side benefit: future implementors > for OPA or other authorization engines would also only need to depend on this > small JAR. > Approach b: On the Ranger side, use Maven profiles + toolchains to compile > against Drill’s java‑exec. This is possible but pulls in a much larger > dependency footprint for Ranger, making it less clean than approach (a). > Approach c: Wait for Ranger 3.0 (which will support newer JDK versions). > Summary and next steps I would like the community’s guidance on which > direction we should take. At the same time, if we decide that the > service‑plugin part should move to Ranger, I think we should proactively > reach out to the [email protected] <mailto:[email protected]> > mailing list to understand their willingness to host that code. > Please share your thoughts. Once we reach consensus, I will update the PR > accordingly. > Thanks, > [huafeng.shi]
signature.asc
Description: Message signed with OpenPGP
