andygrove opened a new issue, #6175:
URL: https://github.com/apache/datafusion-comet/issues/6175
`CometNativeUDF.register` loads any shared library at any path the caller
names, on the driver and
then on every executor, with the full privileges of the JVM process. There
is no configuration an
operator can set to turn the feature off or to limit which paths may be
loaded.
Options:
- `spark.comet.nativeUdf.enabled`, checked in `register` on the driver and
in the planner on the
executor, so a plan built elsewhere cannot bypass it. Whether it defaults
to on (the feature is
already opt-in by API call) or off (loading native code should need an
operator's consent) is
the main question for this issue.
- `spark.comet.nativeUdf.allowedPaths`, a list of directory prefixes. Paths
would be canonicalized
before the check so `..` and symlinks cannot escape it.
The executor-side check matters as much as the driver-side one: the library
path travels in the
`NativeScalarUdf` proto, so the executor must not trust that the driver
checked it.
Follow-up to #4459.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
