Re: Code review / sqlContext Scope

2016-10-19 Thread Ajay Chander
Can someone please shed some lights on this. I wrote the below code in Scala 2.10.5, can someone please tell me if this is the right way of doing it? import org.apache.spark.{SparkConf, SparkContext} import org.apache.spark.sql.{DataFrame, Row} import org.apache.spark.sql.functions._ import org.a

Code review / sqlContext Scope

2016-10-08 Thread Ajay Chander
Hi Everyone, Can anyone tell me if there is anything wrong with my code flow below ? Based on each element from the text file I would like to run a query against Hive table and persist results in another Hive table. I want to do this in parallel for each element in the file. I appreciate any of yo