rdtr opened a new issue, #13569:
URL: https://github.com/apache/gluten/issues/13569

   ### Description
   
   ### Description
   The Spark test `CreateTableAsSelectSuite."CREATE TABLE USING AS SELECT based 
on the file without write permission"` removes the write permission from a 
directory and expects the CTAS to fail with `Job aborted`.
   
   Gluten's CI runs the Spark unit tests as root (the maven cache lives in 
/root/.m2 inside the container). Root ignores the write permission bit, so the 
write succeeds, no exception is thrown and the test fails. The test has been 
excluded in VeloxTestSettings since the suite was imported in 2023 (#908), with 
the comment "Gluten can not catch the spark exception in Driver side", which is 
not the actual reason: the test passes as a non-root user with no Gluten change.
   
   https://github.com/apache/gluten/pull/13400 replaces the exclude with a 
Gluten version of the test that is skipped (`assume`) when removing the write 
permission does not stop writes, so it runs on developer machines but is still 
skipped in CI.
   
   This issue tracks making the test actually run in CI. Options considered so 
far:
   - Run the Spark UT jobs as a non-root user in the CI container. This is the 
proper fix but touches the CI images/workflows.
   - Point the output path at a location that refuses writes for every user 
(e.g. under /proc on Linux). This changes what the test checks and is 
environment-dependent, so it was not done.
   - `chattr +i` on the directory does not work either: Docker drops 
CAP_LINUX_IMMUTABLE by default.
   
   Other permission-based Spark tests probably hit the same problem when they 
are re-enabled.
   
   ### Relates to
   https://github.com/apache/gluten/issues/11088
   https://github.com/apache/gluten/pull/13400
   
   ### Gluten version
   
   None


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to