Andrew Or created SPARK-7644: -------------------------------- Summary: Ensure all scoped RDD operations are tested and cleaned Key: SPARK-7644 URL: https://issues.apache.org/jira/browse/SPARK-7644 Project: Spark Issue Type: Bug Components: Spark Core, SQL, Streaming Affects Versions: 1.4.0 Reporter: Andrew Or Assignee: Andrew Or Priority: Blocker
If all goes well, this will be a "Won't Fix". Before releasing we should make sure all operations wrapped in `RDDOperationScope.withScope` are actually tested and enclosed closures are actually cleaned. This is because a big change went into `ClosureCleaner` and wrapping methods in closures may change whether they are serializable. TL;DR we should run all the wrapped operations to make sure we don't run into java.lang.NotSerializableException. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org