[
https://issues.apache.org/jira/browse/SPARK-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matei Zaharia resolved SPARK-2490.
----------------------------------
Resolution: Fixed
Fix Version/s: 1.1.0
> StackOverflowError when RDD dependencies are too long
> -----------------------------------------------------
>
> Key: SPARK-2490
> URL: https://issues.apache.org/jira/browse/SPARK-2490
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.0.0
> Reporter: Liang-Chi Hsieh
> Fix For: 1.1.0
>
>
> When performing some transformations on RDDs after many iterations, the
> dependencies of RDDs could be very long. It can easily cause
> StackOverflowError when recursively visiting these dependencies in Spark
> core. For example:
> var rdd = sc.makeRDD(Array(1))
> for (i <- 1 to 1000) {
> rdd = rdd.coalesce(1).cache()
> rdd.collect()
> }
--
This message was sent by Atlassian JIRA
(v6.2#6252)
