RE: Hive using Spark engine alone

2015-11-27 Thread Mich Talebzadeh
] Sent: 27 November 2015 14:03 To: Mich Talebzadeh <m...@peridale.co.uk> Cc: user <user@spark.apache.org> Subject: Re: Hive using Spark engine alone Hi, I recommend to use the latest version of Hive. You may also wait for hive on tez with tez version >= 0.8 and hive >

Re: Hive using Spark engine alone

2015-11-27 Thread Jörn Franke
Hi, I recommend to use the latest version of Hive. You may also wait for hive on tez with tez version >= 0.8 and hive > 1.2. Before that I recommend first trying other optimizations of Hive and have a look at the storage format together with storage indexes (not the regular ones), bloom

Hive using Spark engine alone

2015-11-27 Thread Mich Talebzadeh
Hi, As a matter of interest has anyone installed and configured Spark to be used as the execution engine for Hive please? This is in contrast to install and configure Spark as an application. Spark by default uses MapReduce as its execution engine which is more for batch processing.