HyukjinKwon opened a new pull request, #57028:
URL: https://github.com/apache/spark/pull/57028
### What changes were proposed in this pull request?
`TestDirectWorkerDispatcher.shortTempBase` (in
`TestDirectWorkerHelpers.scala`) chooses a
base directory for the direct-worker Unix-domain socket by taking the
**first** writable
entry among `$TMPDIR`, `$TEMP`, `$TMP`, `/tmp`. This PR changes it to pick
the **shortest**
usable base, measured by its real (symlink-resolved) path.
### Why are the changes needed?
On macOS, `$TMPDIR` is a long per-user path like
`/var/folders/k8/j7r3p6cx43xdqhzy2rmp6tqr0000gn/T/` (~49 chars). Combined
with the
`createTempDirectory` random suffix and the `w-<id>.sock` leaf, the bound
socket path
exceeds the **104-byte macOS `sun_path` limit**, so
`PythonUDFWorkerSpecificationSuite`
fails with:
```
OSError: AF_UNIX path too long
at
...TestDirectWorkerDispatcher.throwWorkerExitedBeforeSocket(TestDirectWorkerHelpers.scala:207)
```
This keeps the scheduled `Build / Maven (Scala 2.13, JDK 21, MacOS-26)` lane
red. Picking the
shortest base makes macOS use `/tmp` (which resolves to the short
`/private/tmp`) instead of
the long `$TMPDIR`, keeping the socket path well within the limit. Linux is
unaffected — its
per-user temp dirs are already short — and the choice is now deterministic
(shortest wins)
rather than order-dependent.
### Does this PR introduce _any_ user-facing change?
No. Test-only.
### How was this patch tested?
Ran the affected suites on both `macos-14` and `macos-15` GitHub Actions
runners (arm64, the
same short-`$TMPDIR` layout as `macos-26`), where `$TMPDIR` was confirmed to
be the long
`/var/folders/...` path (`len=49`):
- `udf-worker-core`: `DirectWorkerDispatcherSuite` + `WorkerSessionSuite` —
**48 tests, 0 failures**
- `sql/core`: `PythonUDFWorkerSpecificationSuite` — **1 test, 0 failures**
(previously
`AF_UNIX path too long`)
Both runners green:
https://github.com/HyukjinKwon/spark/actions/runs/28765870386
Before this change the same suite fails on `macos-26`:
https://github.com/apache/spark/actions/runs/28753698265
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
