Dear Drill developers,
First of all, thank you for the detailed review and for raising the
governance‑level issue regarding the Ranger integration PR. I would like to
open a discussion on this matter with the community.
Background The PR currently adds two modules – drill-ranger-plugin and
drill-ranger-service – and all Java code within them resides under the
org.apache.ranger.* package namespace. This potentially raises two issues:
Placing Drill‑specific code under org.apache.ranger.* means that Drill releases
artifacts occupying another ASF project’s namespace. This brings questions such
as:
Who owns, versions, and applies CVE patches to these classes?
Is this acceptable from an ASF policy perspective?
This practice deviates from the established convention in the Apache Ranger
ecosystem. All similar service plugins (e.g., ranger-hive-plugin,
ranger-hbase-plugin, ranger-kafka-plugin, ranger-presto-plugin) are maintained
and released directly by the Ranger PMC, which keeps the package namespace and
release ownership unified within a single project.
Current proposals At this point, we see two high‑level approaches:
Option A: Keep the modules as an independent plugin, rename the package to a
Drill‑owned namespace, and have Drill manage the plugin entirely without being
constrained by Ranger’s release cycle.
Option B: Wait for official feedback from the Ranger community to decide
whether they are willing to accept this plugin as part of their project.
Detailed analysis
Issue 1 (namespace ownership) The solution is straightforward: keep both
modules in Drill but rename the package to
org.apache.drill.exec.security.ranger (or similar), making it clear that this
code belongs to Drill and is maintained by the Drill community.
Issue 2 (alignment with Ranger ecosystem) Moving the drill‑ranger code to
Ranger, where it would be maintained by the Ranger PMC, depends critically on
the attitude of the dev@ranger community. I would like to discuss a few
possible directions with everyone.
(1) What parts of this PR truly belong to Drill?
Sub‑option 1 – Recommended: Drill retains only the generic authorization
contract, and the Ranger‑specific implementation is moved out. Specifically,
the following should stay in the Drill repository:
AccessAuthorizer SPI – the interface contract that any authorization engine can
implement (not biased toward Ranger)
AccessTypes – access type constants
NoOpAccessAuthorizer – the default no‑op implementation, which also
demonstrates the neutrality of the SPI
AccessAuthorizerFactory – the standard ServiceLoader‑based discovery mechanism
(fail‑closed if no provider is found)
ColumnAccessChecker call sites – the integration points within the execution
framework
The parts to be moved out of Drill include: policy engine integration,
RangerAdmin client, auditing, and RangerServiceDrill (the admin‑side service
definition).
With this split, Drill core has zero org.apache.ranger dependencies and can be
released independently.
However, this sub‑option raises several practical concerns that need to be
addressed:
SPI dependency for Ranger: The Ranger plugin would depend on Drill’s
AccessAuthorizer SPI. Drill would only need a one‑time bootstrapping release:
first release a version containing the SPI, then Ranger follows up with the
plugin release. Documentation would clearly state that the actual authorization
capability is provided by the Ranger plugin at installation time. Is this
release ordering feasible?
JDK version mismatch: Ranger currently runs on JDK 8, while Drill uses JDK 17+.
Approach a (recommended): Extract a minimal, independent module drill‑auth‑spi
from java‑exec, compile it with --release 8, and ship it with Drill. This is
exactly what Presto does – presto‑spi is a small, separate artifact published
to Central, and the Ranger Presto shim depends only on that, not on the entire
Presto engine. A side benefit: future implementors for OPA or other
authorization engines would also only need to depend on this small JAR.
Approach b: On the Ranger side, use Maven profiles + toolchains to compile
against Drill’s java‑exec. This is possible but pulls in a much larger
dependency footprint for Ranger, making it less clean than approach (a).
Approach c: Wait for Ranger 3.0 (which will support newer JDK versions).
Summary and next steps I would like the community’s guidance on which direction
we should take. At the same time, if we decide that the service‑plugin part
should move to Ranger, I think we should proactively reach out to the
[email protected] mailing list to understand their willingness to host
that code.
Please share your thoughts. Once we reach consensus, I will update the PR
accordingly.
Thanks,
[huafeng.shi]
