hi Charles, drill devs:
I’d like to share the conclusion from my discussion with the Apache Ranger
community for Drill‑Ranger integration, together with our restructured code
modules. I also have one key question to seek your advice.
Conclusion from Ranger community
Following the modern integration pattern adopted by Ranger for new components
(Trino, Polaris), the runtime Java authorization plugin shall be maintained
within the upstream project’s own repository. Only the service‑definition JSON
file will be hosted in the Apache Ranger repo. This avoids cross‑project
dependency pain, JDK version conflicts and release‑cycle coupling, compared
with the legacy Presto model where the full plugin resided inside Ranger
repository.
Accordingly, the Drill‑Ranger runtime plugin will live entirely in the Apache
Drill codebase and be maintained by the Drill community.
Restructured code module layout
drill ├── exec │ ├── java‑exec # Drill authorization invocation point in query
execution flow │ └── security‑spi # New generic authorization SPI contract. │
It can be implemented by Ranger or other security frameworks. └── drill‑ranger
[drill‑ranger‑parent] # Ranger‑specific modules, isolated with Ranger‑only
dependencies ├── drill‑ranger‑plugin # Main runtime plugin, implements
security‑spi: authorization, resource mapping, auditing ├──
drill‑ranger‑plugin‑shim # Shim layer for compatibility across different Ranger
client versions └── drill‑ranger‑service # Temporary home for
ranger‑servicedef‑drill.json service‑definition file └── pom.xml
Design note: All modules under drill‑ranger carry only Ranger‑related
dependencies. Once we move the service‑definition file to Ranger repository
later, we only need to remove the drill‑ranger‑service module with minimal code
changes.
Key question for PMC feedback
For phase‑1 implementation:
Only basic access types are supported: CREATE, SELECT, DROP.
Row‑level filter and data masking are not yet implemented.
We have two options regarding where to place the ranger‑servicedef‑drill.json
service‑definition file. Please advise which approach we should take.
Option 1: Submit the current partial service‑definition directly to Apache
Ranger repo now
Pros: Fully follows Ranger’s official recommended integration pattern. Ranger
Admin can recognize Drill service and allow policy configuration out‑of‑the‑box.
Cons: The service‑definition will lack entries for row‑level filter and
masking. Later we need to submit follow‑up PRs to Ranger to extend the service
definition when those features are ready.
Option 2: Keep service‑definition inside Drill repo in drill‑ranger‑service for
now
Pros: We can contribute a complete service‑definition with all access types,
row‑level filter and masking in one‑shot when fully implemented. No multiple
subsequent PRs against Ranger repo.
Cons: Does not align with Ranger’s best practice. End‑users will need to
manually load the Drill service‑definition file.
Important note: Whichever option we choose for service‑definition, the Ranger
runtime Java plugin will always stay within Drill repository and owned by Drill
community, which has been confirmed with Ranger community.
The drill‑security‑spi is designed as a generic authorization contract, so
other security frameworks can implement this SPI besides Ranger in the future.
Looking forward to your suggestions.
Best regards, Huafeng Shi
---- Replied Message ----
FromMadhan Neethiraj<[email protected]>Date8/30/2026
23:[email protected]<[email protected]>[email protected]<[email protected]>SubjectRe:
Subject: [DISCUSS] Integrate Apache Drill with Apache Ranger via authorization
SPI
Hi Huafeng Shi,
Thank you for reaching out to Ranger community and the providing appropriate
details.
Here is the preferred integration approach from Ranger community:
1. Service-definition, describing the resources and access-types, is committed
in
Apache Ranger git repo, so that it is automatically registered during Ranger
admin server bootstrap.
2. Plugin implementation itself remains in Apache Drill git repo
With this approach, Apache Drill community will avoid dependency on Apache
Ranger
releases for any enhancements in authorizer implementation - for example
changes in
authorization SPI or fixes. This will also help avoid having to deal with JDK
version
differences across the projects.
Here are few references for this integration model:
- Trino: https://github.com/trinodb/trino/tree/master/plugin/trino-ranger
- Polaris: https://github.com/apache/polaris/tree/main/extensions/auth/ranger
Blog on "Integrating Applications with Apache Ranger" can be a useful reference
as you
look into integration:
https://ranger.apache.org/blogs/integrating_applications.html.
Thanks,
Madhan
On 8/30/26, 4:55 AM, "huafengshi" <[email protected]
<mailto:[email protected]>> wrote:
Hi Ranger Devs,
I'm Huafeng Shi, a Committer of Apache Drill. I’m working on authorization
integration between Apache Drill and Apache Ranger, and would like to start a
discussion with the community.
Here is my proposed design:
Apache Drill will expose a generic authorization SPI (contract) for access
control. This SPI is not yet released, I am preparing to submit the
implementation to Drill.
Based on this SPI, we can build a Ranger‑specific authorization plugin for
Drill.
Two possible options for hosting this Ranger‑Drill plugin:
Option A: Contribute the Drill Ranger plugin into Apache Ranger codebase
(target Ranger 3.0).
Option B: Maintain the plugin within Apache Drill repository, once the Drill
authorization SPI is finalized and released.
I would like to clarify several key questions with the Ranger community:
Does the Ranger community accept this overall integration approach: Drill
provides authorization SPI contract, Ranger supplies the plugin implementation
to enforce access control for Drill?
If Ranger community is willing to host this Drill plugin inside Ranger repo for
Ranger 3.0:
I will collaborate with Drill community to finalize and formalize the
authorization SPI in Drill first.
Then I will contribute the Drill‑Ranger plugin code to Ranger.
If the plugin is accepted for Ranger 3.0, are there any requirements for SPI
stability, testing scope, documentation, compatibility we need to follow?
If Ranger prefers not to host Drill plugin inside Ranger repository, is there
any guidance for maintaining it in Drill side as a third‑party Ranger plugin?
Looking forward to your feedback. Thanks a lot.
Best regards,
Huafeng Shi
---- Replied Message ----
FromCharles Givre<[email protected] <mailto:[email protected]>>Date8/29/2026
05:30To<[email protected] <mailto:[email protected]>>SubjectRe: [DISCUSS]
Package namespace and module ownership for Drill-Ranger integration (DRILL-8548)
Huafeng, Thanks for this. Have you contacted the Ranger community and asked
this question of them? Are they willing to accept your plugin? It would be
helpful to know this.
For the record, I think this integration will be very valuable to Drill once
complete.
Best,
— Charles
On Aug 27, 2026, at 09:40, huafengshi <[email protected]
<mailto:[email protected]>> wrote:
Dear Drill developers,
First of all, thank you for the detailed review and for raising the
governance‑level issue regarding the Ranger integration PR. I would like to
open a discussion on this matter with the community.
Background The PR currently adds two modules – drill-ranger-plugin and
drill-ranger-service – and all Java code within them resides under the
org.apache.ranger.* package namespace. This potentially raises two issues:
Placing Drill‑specific code under org.apache.ranger.* means that Drill releases
artifacts occupying another ASF project’s namespace. This brings questions such
as:
Who owns, versions, and applies CVE patches to these classes?
Is this acceptable from an ASF policy perspective?
This practice deviates from the established convention in the Apache Ranger
ecosystem. All similar service plugins (e.g., ranger-hive-plugin,
ranger-hbase-plugin, ranger-kafka-plugin, ranger-presto-plugin) are maintained
and released directly by the Ranger PMC, which keeps the package namespace and
release ownership unified within a single project.
Current proposals At this point, we see two high‑level approaches:
Option A: Keep the modules as an independent plugin, rename the package to a
Drill‑owned namespace, and have Drill manage the plugin entirely without being
constrained by Ranger’s release cycle.
Option B: Wait for official feedback from the Ranger community to decide
whether they are willing to accept this plugin as part of their project.
Detailed analysis
Issue 1 (namespace ownership) The solution is straightforward: keep both
modules in Drill but rename the package to
org.apache.drill.exec.security.ranger (or similar), making it clear that this
code belongs to Drill and is maintained by the Drill community.
Issue 2 (alignment with Ranger ecosystem) Moving the drill‑ranger code to
Ranger, where it would be maintained by the Ranger PMC, depends critically on
the attitude of the dev@ranger community. I would like to discuss a few
possible directions with everyone.
(1) What parts of this PR truly belong to Drill?
Sub‑option 1 – Recommended: Drill retains only the generic authorization
contract, and the Ranger‑specific implementation is moved out. Specifically,
the following should stay in the Drill repository:
AccessAuthorizer SPI – the interface contract that any authorization engine can
implement (not biased toward Ranger)
AccessTypes – access type constants
NoOpAccessAuthorizer – the default no‑op implementation, which also
demonstrates the neutrality of the SPI
AccessAuthorizerFactory – the standard ServiceLoader‑based discovery mechanism
(fail‑closed if no provider is found)
ColumnAccessChecker call sites – the integration points within the execution
framework
The parts to be moved out of Drill include: policy engine integration,
RangerAdmin client, auditing, and RangerServiceDrill (the admin‑side service
definition).
With this split, Drill core has zero org.apache.ranger dependencies and can be
released independently.
However, this sub‑option raises several practical concerns that need to be
addressed:
SPI dependency for Ranger: The Ranger plugin would depend on Drill’s
AccessAuthorizer SPI. Drill would only need a one‑time bootstrapping release:
first release a version containing the SPI, then Ranger follows up with the
plugin release. Documentation would clearly state that the actual authorization
capability is provided by the Ranger plugin at installation time. Is this
release ordering feasible?
JDK version mismatch: Ranger currently runs on JDK 8, while Drill uses JDK 17+.
Approach a (recommended): Extract a minimal, independent module drill‑auth‑spi
from java‑exec, compile it with --release 8, and ship it with Drill. This is
exactly what Presto does – presto‑spi is a small, separate artifact published
to Central, and the Ranger Presto shim depends only on that, not on the entire
Presto engine. A side benefit: future implementors for OPA or other
authorization engines would also only need to depend on this small JAR.
Approach b: On the Ranger side, use Maven profiles + toolchains to compile
against Drill’s java‑exec. This is possible but pulls in a much larger
dependency footprint for Ranger, making it less clean than approach (a).
Approach c: Wait for Ranger 3.0 (which will support newer JDK versions).
Summary and next steps I would like the community’s guidance on which direction
we should take. At the same time, if we decide that the service‑plugin part
should move to Ranger, I think we should proactively reach out to the
[email protected] <mailto:[email protected]> mailing list to
understand their willingness to host that code.
Please share your thoughts. Once we reach consensus, I will update the PR
accordingly.
Thanks,
[huafeng.shi]
