Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12153#issuecomment-212424785
All the timeout in the Jenkins builds were due to a deadlock in
`DAGScheduler`, and can be steadily reproduced locally by running the following
test case
> BucketedReadSuite.only shuffle one side when 2 bucketed tables have
different bucket keys.
This test case creates two bucketed tables both with 8 buckets and then
joins them. Reducing 8 to 5 eliminates the deadlock. But I haven't figured out
the real reason behind the deadlock. The deadlock also disappears if I remove
FileScanRDD.preferredLocations(). Maybe that too many tasks are scheduled to
the same place and exhausted some thread-pool?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
