Yicong-Huang opened a new issue, #7126: URL: https://github.com/apache/texera/issues/7126
### What happened? `ExecutorManager.load_executor_definition` writes the UDF code to a new module file on its tmp filesystem and immediately calls `importlib.import_module`, without calling `importlib.invalidate_caches()` first. Python's `FileFinder` caches each directory listing keyed on the directory's mtime. When two executor updates land within the filesystem's timestamp granularity, the second module file is invisible to the import system and `import_module` raises `ModuleNotFoundError` (e.g. `No module named 'udf-v21'`). Expected: every `initialize_executor` / `update_executor` call imports the module it just wrote — the Python docs require `invalidate_caches()` after creating a module file at runtime. ### How to reproduce? Call `update_executor` twice in quick succession so both module files are written within the filesystem's mtime granularity window; the second import fails. `src/test/python/core/architecture/managers/test_executor_manager.py::TestUpdateExecutor::test_repeated_updates_keep_carrying_the_running_state` reproduces this reliably on filesystems with coarse mtime resolution (e.g. overlayfs); on ext4 it depends on timing. ### Version/Branch 1.3.0-incubating-SNAPSHOT (main) ### Commit Hash (Optional) ebfd1db5f9c8c427a406cf09318e7012436edbde ### Relevant log output ```shell FAILED src/test/python/core/architecture/managers/test_executor_manager.py::TestUpdateExecutor::test_repeated_updates_keep_carrying_the_running_state - ModuleNotFoundError: No module named 'udf-v21' ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
