Re: How spark makes partition when we insert data using the Sql query, and how the permissions to the partitions is assigned.?

Let us take this for a ride. Simple code. Reads from an existing of 22miilion rows stored as ORC and saves it as a Parquet val HiveContext = new org.apache.spark.sql.hive.HiveContext(sc) HiveContext.sql("use oraclehadoop") val s = HiveContext.table("sales2") val sorted = s.sort("prod_id","cust_id

How spark makes partition when we insert data using the Sql query, and how the permissions to the partitions is assigned.?

Hey guys I am using Apache Spark 1.5.2, and I am running the Sql query using the SQLContext and when I run the insert query it saves the data in partition (as expected). I am just curious and want to know how these partitions are made and how the permissions to these partition is assigned . Can we