Abdeali Kothari wrote > I am using Spark 2.3.0 and trying to read a CSV file which has 500 > records. > When I try to read it, spark says that it has two stages: 10, 11 and then > they join into stage 12.
What's your CSV size per file? I think Spark optimizer may put many files into one task when reading small files. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
