You might want to consider Hive on Spark where you can work directly with Hive and your query execution is powered by Spark as an engine.
--Xuefu On Tue, Dec 15, 2015 at 6:04 PM, Divya Gehlot <[email protected]> wrote: > Hi, > I am new bee to Spark and I am exploring option and pros and cons which > one will work best in spark and hive context.My dataset inputs are CSV > files, using spark to process the my data and saving it in hive using > hivecontext > > 1) Process the CSV file using spark-csv package and create temptable and > store the data in hive using hive context. > 2) Process the file as normal text file in sqlcontext ,register its as > temptable in sqlcontext and store it as ORC file and read that ORC file in > hive context and store it in hive. > > Is there any other best options apart from mentioned above. > Would really appreciate the inputs. > Thanks in advance. > > Thanks, > Regards, > Divya >
