Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/9084#issuecomment-147602003
It looks like the test failures are occurring in the external sorter
suites. Some of these tests exercise spilling-related logic (such as ensuring
that spill files are cleaned up); in order to induce spilling, these tests
configured the legacy memory fractions to be really small. In order to fix
these tests, we can either enable legacy mode for those tests only or we can
modify the tests to set the new configurations. Ideally these sorter unit tests
wouldn't be relying on memory management behavior in order to test spill
cleanup, but they were written a long time ago. We could refactor these tests,
but I'd prefer a band-aid solution for now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
