How to improve performance of JavaRDD<String>.saveAsTextFile(“hdfs://…“). This is taking over 30 minutes on a cluster of 10 nodes. Running Spark on YARN.
JavaRDD<String> has 120 million entries. Thank you, Best regards, Mahmoud
How to improve performance of JavaRDD<String>.saveAsTextFile(“hdfs://…“). This is taking over 30 minutes on a cluster of 10 nodes. Running Spark on YARN.
JavaRDD<String> has 120 million entries. Thank you, Best regards, Mahmoud