Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/19307#discussion_r140210894
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -396,12 +396,12 @@ class DAGScheduler(
/** Find ancestor shuffle dependencies that are not registered in
shuffleToMapStage yet */
private def getMissingAncestorShuffleDependencies(
- rdd: RDD[_]): Stack[ShuffleDependency[_, _, _]] = {
- val ancestors = new Stack[ShuffleDependency[_, _, _]]
+ rdd: RDD[_]): ArrayStack[ShuffleDependency[_, _, _]] = {
--- End diff --
Stack is deprecated in 2.12 for poor performance; ArrayStack should work
the same and be faster, in 2.11 too
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
