Thanks. Unfortunately I dont have control over how data is inserted and the table is not partitioned.
The reason the sub directories are being created is because when Tez does an INSERT into a table from a UNION query it creates sub directories so that it can write in parallel. I've also realized that its only with Parquet ... csv works fine -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
