But i was wrong - map also fails on big file and setting spark.shuffle.spill doesn't help. Map fails with the same error.
-- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/distinct-on-huge-dataset-tp3025p3039.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
