zozo123 opened a new pull request, #59232: URL: https://github.com/apache/spark/pull/59232
### What changes were proposed in this pull request? Add `bin/sbx-udf-worker`. It is an external worker command. Spark does not call it, and Spark gains no sandbox configuration. An operator puts the command in the direct worker spec. Spark appends `--id` and `--connection` as it already does for every direct worker. The command starts the real worker with `sbx exec` and binds the host socket named by `--connection`. A socket created inside the sandbox belongs to the guest kernel, so the host executor cannot connect to that file even when the path is mounted. This is a draft. It is not part of https://github.com/apache/spark/pull/59155. ### Why are the changes needed? A direct worker has to bind the Unix socket on the executor host. Running that worker inside Docker Sandboxes needs a host process that owns the host socket and forwards bytes. The existing worker command is that extension point. ### Does this PR introduce _any_ user-facing change? Yes, for an operator who chooses this command. The command is: ``` bin/sbx-udf-worker --sandbox NAME -- GUEST_COMMAND ``` Spark then appends `--id` and `--connection`. Executors that do not use the command are unchanged. Spark does not require `sbx`. ### How was this patch tested? The same forwarding program returned rows `(0, 0)` through `(5, 5)` from `PlanExternalUDFsSuite` through two guest workers. Each worker logged `Linux aarch64` and `transport=epoll`, accepted one connection, and stopped on SIGTERM. That run is not repeated here because the suite change is still in https://github.com/apache/spark/pull/59155, and Spark CI has no `sbx` login. `python3 -m py_compile bin/sbx-udf-worker` passed. The command rejects a missing `--` and a command that has no `--id` or `--connection`. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Cursor Grok 4.7 Made with [Cursor](https://cursor.com) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
