[ https://issues.apache.org/jira/browse/SPARK-36965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Attila Zsolt Piros updated SPARK-36965: --------------------------------------- Description: I was running a python test which was extremely slow and I was surprised the unit-tests.log has not been even created. Looked into the code and as I got tests can be executed in parallel and each one has its own temporary output file which is only added to the unit-tests.log when a test is finished with a failure (after acquiring a lock to avoid parallel write on unit-tests.log). To avoid such a confusion it would make sense to log out the path of the temporary output files. was: I was running a python test which was extremely slow and I was surprised the unit-tests.log has not been even created. Looked into the code I got for the parallel execution each test has its own temporary output file which are only added to the unit-tests.log when a test is finished with a failure (after acquiring a lock to avoid parallel write on unit-tests.log). To avoid such a confusion it would make sense to log out the path of the temporary output files. > Extend python test runner by logging out the temp output files > -------------------------------------------------------------- > > Key: SPARK-36965 > URL: https://issues.apache.org/jira/browse/SPARK-36965 > Project: Spark > Issue Type: Improvement > Components: Tests > Affects Versions: 3.3.0 > Reporter: Attila Zsolt Piros > Assignee: Attila Zsolt Piros > Priority: Minor > > I was running a python test which was extremely slow and I was surprised the > unit-tests.log has not been even created. Looked into the code and as I got > tests can be executed in parallel and each one has its own temporary output > file which is only added to the unit-tests.log when a test is finished with a > failure (after acquiring a lock to avoid parallel write on unit-tests.log). > To avoid such a confusion it would make sense to log out the path of the > temporary output files. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org