aglinxinyuan opened a new issue, #6917: URL: https://github.com/apache/texera/issues/6917
### Task Summary `fs` (PyFilesystem2) is effectively EOL: 2.4.16 (May 2022) is the last release ever published, upstream master still imports `pkg_resources` at import time (PyFilesystem/pyfilesystem2#577 and #597 have been open for years), and there is no maintained successor package. Since setuptools 82 removed `pkg_resources` entirely, `import fs` hard-crashes there — which is the only reason amber pins `setuptools==80.10.2` (#4199, #6412). PR #6880 silences the resulting deprecation warning in CI, so this issue keeps the underlying debt visible (as suggested in [this review comment](https://github.com/apache/texera/pull/6880#discussion_r3649685557)). ``` Today: ExecutorManager -> fs.open_fs("temp://") -> pkg_resources -> setuptools held < 82 After: ExecutorManager -> tempfile.mkdtemp + shutil.rmtree -> fs gone -> setuptools unpinned ``` `ExecutorManager` is the only consumer, and fs's `TempFS` is internally exactly `tempfile.mkdtemp` + `shutil.rmtree`, so the swap is behavior-preserving. Full footprint: | File | Change | | --- | --- | | `amber/src/main/python/core/architecture/managers/executor_manager.py` | replace `fs.open_fs("temp://")` with `tempfile.mkdtemp()` + `shutil.rmtree`, preserving lazy creation, `sys.path` handling, the never-materialized close guard, and the tolerated leak-on-force-kill semantics | | `amber/src/test/python/core/architecture/managers/test_executor_manager.py` | two lines read `manager.fs.getsyspath(...)`; no test mocks `fs` | | `amber/requirements.txt` | drop `fs==2.4.16` and the `setuptools==80.10.2` pin plus its comment (fs is the only distribution in the closure requiring setuptools; pytz's `pkg_resources` use is a guarded fallback) | | `amber/LICENSE-binary-python` | drop `fs==2.4.16` and `appdirs==1.4.4` (an fs-only transitive); keep `six` (python-dateutil needs it) | The migration PR should also revert the two warning-filter hunks from #6880 (worker entry point + pytest `filterwarnings`), which become dead weight once `fs` is gone. Implementation notes: - Use `tempfile.mkdtemp`, not `tempfile.TemporaryDirectory` — its GC/exit finalizer would silently clean up abandoned directories, changing the documented leak-on-force-kill behavior (see the TODO in `executor_manager.py`). - Both license bullets must land in the same PR: stale `LICENSE-binary-python` entries hard-fail `check_binary_deps.py` even in PR mode. - torch (operator-requirements) still declares `setuptools<82` transitively; harmless, and invisible to the license check. ### Task Type - [x] Refactor / Cleanup - [ ] DevOps / Deployment / CI - [ ] Testing / QA - [ ] Documentation - [ ] Performance - [ ] Other -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
