Re: Spark Dataframe: Save to hdfs is taking long time

2016-12-28 Thread Raju Bairishetti
..@gmail.com> >> *Sent:* Thursday, December 15, 2016 7:54:27 PM >> *To:* user @spark >> *Subject:* Spark Dataframe: Save to hdfs is taking long time >> >> Hi, >> >> I am using issue while saving the dataframe back to HDFS. It's taking >>

Re: Spark Dataframe: Save to hdfs is taking long time

2016-12-15 Thread KhajaAsmath Mohammed
, December 15, 2016 7:54:27 PM > *To:* user @spark > *Subject:* Spark Dataframe: Save to hdfs is taking long time > > Hi, > > I am using issue while saving the dataframe back to HDFS. It's taking long > time to run. > > val results_dataframe = sqlContext.sql("se

Re: Spark Dataframe: Save to hdfs is taking long time

2016-12-15 Thread Felix Cheung
What is the format? From: KhajaAsmath Mohammed <mdkhajaasm...@gmail.com> Sent: Thursday, December 15, 2016 7:54:27 PM To: user @spark Subject: Spark Dataframe: Save to hdfs is taking long time Hi, I am using issue while saving the dataframe back to HDFS

Spark Dataframe: Save to hdfs is taking long time

2016-12-15 Thread KhajaAsmath Mohammed
Hi, I am using issue while saving the dataframe back to HDFS. It's taking long time to run. val results_dataframe = sqlContext.sql("select gt.*,ct.* from PredictTempTable pt,ClusterTempTable ct,GamificationTempTable gt where gt.vin=pt.vin and pt.cluster=ct.cluster")