Hi,
Thanks for trying out Hive Server with Tez. The second set of queries should start executing on the same sessions/containers as the first query as soon as the previous queries are complete. We will need some information to understand whats happening. For starters 1) Hive server has 10 threads in parallel right? 2) How many queues and sessions per queue have been configured for Hive server? 3) Have these queues been mapped to corresponding queues in YARN? 4) What you are seeing is the following. You see full concurrency of 10 queries in the first set of queries. Then the second set of queries has gotten stalled for longer than expected. Right? I am adding Vikram, who is the implementer of Hive server support on Tez to this thread for more insights/questions. In the meanwhile, the best progress on this can be made by opening a jira in the Apache Hive project detailing your observations. Please attach your hive-site.xml and tez-site.xml to that jira. Its not possible to attach anything to this email thread. If you can share those files in any other manner and provide a link in this thread then it will help in the investigation. Thanks Bikas *From:* Natarajan, Prabakaran 1. (NSN - IN/Bangalore) [mailto: [email protected]] *Sent:* Tuesday, May 20, 2014 1:22 AM *To:* [email protected] *Cc:* Sachwani, Yogesh (NSN - IN/Bangalore) *Subject:* RE: Tez - Hive Parallel queries Hi Rajesh We have this set in tez-site.xml and also before running the JDBC sql query we first fire a sql setting this parameter too. *Thanks and Regards* Prabakaran.N aka NP nsn, Bangalore *When "I" is replaced by "We" - even Illness becomes "Wellness"* *From:* ext Rajesh Balamohan [mailto:[email protected]<[email protected]>] *Sent:* Tuesday, May 20, 2014 12:58 PM *To:* [email protected] *Cc:* Sachwani, Yogesh (NSN - IN/Bangalore) *Subject:* Re: Tez - Hive Parallel queries Hi Prbakaran, Are you passing these parameters via CLI or via tez-site.xml? ~Rajesh.B On Tue, May 20, 2014 at 12:29 PM, Natarajan, Prabakaran 1. (NSN - IN/Bangalore) <[email protected]> wrote: Hi, We tried running tez queries on Hive server 1 with 10 threads in Parallel. First 10 queries go through fine and result comes in 16-18 seconds but when we fire the same queries again on same connection we find that connection/queries are blocked for more than 200 seconds and only after some 200-260 seconds the queries get fired. In YARN job tracker UI we see the previous query containers still not being closed and once they are closed after > 200 seconds the new queries get fired . We tried both enabling tez AM reuse( same JVM size for map and reduce ) and 100 ms value for "tez.am.container.session.delay-allocation-millis" parameter but none of these help. Any idea on how can we reduce this > 200 sec block/latency *Thanks and Regards* Prabakaran.N aka NP nsn, Bangalore *When "I" is replaced by "We" - even Illness becomes "Wellness"* -- ~Rajesh.B -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
