RE: Hive using Spark engine vs native spark with hive integration.

Thank you so much Mich! Although a bit older, this is the most detailed comparison I’ve read on the subject. Thanks again. Regards, -Manu From: Mich Talebzadeh Sent: Tuesday, October 06, 2020 12:37 PM To: user Subject: Re: Hive using Spark engine vs native spark with hive integration

Re: Hive using Spark engine vs native spark with hive integration.

Hi Manu, In the past (July 2016), I made a presentation organised by then Hortonworks in London titled "Query Engines for Hive: MR, Spark, Tez with LLAP – Considerations! " The PDF presentation is here . With a caveat

Re: Hive using Spark engine vs native spark with hive integration.

hi, if you are already running hive with tez,the perf gain won't be obvious camparing with spark. I'd recommend experimenting with spark on something new until a better understanding is formed Manu Jacob 于2020年10月6日 周二23:47写道： > Hi All, > > > > Not sure if I need to ask this question on hive

Hive using Spark engine vs native spark with hive integration.

Hi All, Not sure if I need to ask this question on hive community or spark community. We have a set of hive scripts that runs on EMR (Tez engine). We would like to experiment by moving some of it onto Spark. We are planning to experiment with two options. 1. Use the current code based on