rdtr opened a new issue, #13569: URL: https://github.com/apache/gluten/issues/13569
### Description ### Description The Spark test `CreateTableAsSelectSuite."CREATE TABLE USING AS SELECT based on the file without write permission"` removes the write permission from a directory and expects the CTAS to fail with `Job aborted`. Gluten's CI runs the Spark unit tests as root (the maven cache lives in /root/.m2 inside the container). Root ignores the write permission bit, so the write succeeds, no exception is thrown and the test fails. The test has been excluded in VeloxTestSettings since the suite was imported in 2023 (#908), with the comment "Gluten can not catch the spark exception in Driver side", which is not the actual reason: the test passes as a non-root user with no Gluten change. https://github.com/apache/gluten/pull/13400 replaces the exclude with a Gluten version of the test that is skipped (`assume`) when removing the write permission does not stop writes, so it runs on developer machines but is still skipped in CI. This issue tracks making the test actually run in CI. Options considered so far: - Run the Spark UT jobs as a non-root user in the CI container. This is the proper fix but touches the CI images/workflows. - Point the output path at a location that refuses writes for every user (e.g. under /proc on Linux). This changes what the test checks and is environment-dependent, so it was not done. - `chattr +i` on the directory does not work either: Docker drops CAP_LINUX_IMMUTABLE by default. Other permission-based Spark tests probably hit the same problem when they are re-enabled. ### Relates to https://github.com/apache/gluten/issues/11088 https://github.com/apache/gluten/pull/13400 ### Gluten version None -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
