Re: Hive on Tez vs Impala

2019-04-22 Thread Gopal Vijayaraghavan
> I wish the Hive team to keep things more backward-compatible as well. Hive is > such an enormous system with a wide-spread impact so any > backward-incompatible change could cause an uproar in the community. The incompatibilities were not avoidable in a set of situations - a lot of those

Re: Hive on Tez vs Impala

2019-04-22 Thread Thai Bui
I'm using Hive 3.1 on Tez/LLAP and I must say the experience was not good but it was worth it. We built Hive from HDP's hive-release and add Tez UI back, combined that with Hue 4.3 (also built from Cloudera Hue). Now that the two companies have merged I think things are going to get better (I'm

Re: Hive on Tez vs Impala

2019-04-16 Thread Edward Capriolo
I have changes jobs 3 times since tez was introduced. It is a true waste of compute resources and time that it was never patched in. So I either have to waste my time patching it in, waste my time running a side deployment, or not installing it and waste money having queries run longer on mr/spark

Re: Hive on Tez vs Impala

2019-04-15 Thread Manoj Murumkar
If we install our own build of Hive, we'll be out of support from CDH. Tez is not supported anyway and we're not touching any CDH bits, so it's not a big issue to have our own build of Tez engine. > On Apr 15, 2019, at 9:20 PM, Gopal Vijayaraghavan wrote: > > > Hi, > >>> However, we have

Re: Hive on Tez vs Impala

2019-04-15 Thread Gopal Vijayaraghavan
Hi, >> However, we have built Tez on CDH and it runs just fine. Down that path you'll also need to deploy a slightly newer version of Hive as well, because Hive 1.1 is a bit ancient & has known bugs with the tez planner code. You effectively end up building the hortonworks/hive-release

Re: Hive on Tez vs Impala

2019-04-15 Thread Edward Capriolo
Lol. I was hoping that the merger would unblock the "saltyness". I wonder what is the official position is now because back in the day there was a puff piece produced to the effect of hive was not the way forward and impala is the bees knees. On Monday, April 15, 2019, Manoj Murumkar wrote: >

Re: Hive on Tez vs Impala

2019-04-15 Thread Manoj Murumkar
No, not yet. However, we have built Tez on CDH and it runs just fine. Following blog summarizes part of the work (bit old, we currently run Tez 0.9.1 on CDH 5.16.1). https://blog.upala.com/2017/03/04/setting-up-tez-on-cdh-cluster/ Blog says use ATS from open source hadoop, which will not work if

Re: Hive on Tez vs Impala

2019-04-15 Thread Artur Sukhenko
Thanks Sungwoo, very nice articles. On Mon, Apr 15, 2019 at 5:38 PM Sungwoo Park wrote: > I tested the performance of Impala 2.12.0+cdh5.15.2+0 in Cloudera CDH > 5.15.2 a while ago. I compared it with Hive 3.1.1 on MR3 (where MR3 is a > new execution engine for Hadoop and Kubernetes). You can

Re: Hive on Tez vs Impala

2019-04-15 Thread Edward Capriolo
Out of band question. Given: https://hortonworks.com/blog/welcome-brand-new-cloudera/ Does cdh finally ship with a tea you dont have to manually patch in? On Monday, April 15, 2019, Sungwoo Park wrote: > I tested the performance of Impala 2.12.0+cdh5.15.2+0 in Cloudera CDH > 5.15.2 a while ago.

Re: Hive on Tez vs Impala

2019-04-15 Thread Sungwoo Park
I tested the performance of Impala 2.12.0+cdh5.15.2+0 in Cloudera CDH 5.15.2 a while ago. I compared it with Hive 3.1.1 on MR3 (where MR3 is a new execution engine for Hadoop and Kubernetes). You can find the result at: https://mr3.postech.ac.kr/blog/2019/03/22/performance-evaluation-0.6/ On