Hi,
I am getting issue while converting dataframe to Rdd, it reduces partitions.
In our code, Dataframe was created as :
DataFrame DF = hiveContext.sql("select * from table_instance");
When I convert my dataframe to rdd and try to get its number of partitions
as
RDD<Row> newRDD = Df.rdd();
System.out.println(newRDD.getNumPartitions());
It reduces the number of partitions to 1(1 is printed in the console).
Originally my dataframe has 102 partitions . Any reasons for this? Please
let me know if my understanding is wrong here.
