Need hive-site and tez-site configuration samples

2018-03-13 Thread Jyotsna G
All, Can anyone share their Hive-site.xml and tez-site.xml config settings wrt hive on TEZ? Specifically looking for hive server 2 config settings for TEZ. Thanks, Jyotsna

Re: Best way/tool to debug memory leaks in HiveServer2

2018-03-13 Thread Gopal Vijayaraghavan
> It also shows that the process is consuming more than 30GB. However, it is > not clear what is causing the process to consume more than 30GB. The Xmx only applies to the heap size, there's another factor that is usually ignored which are the network buffers and compression buffers used by

Re: Best way/tool to debug memory leaks in HiveServer2

2018-03-13 Thread Prasanth Jayachandran
One option to check for is if THP is enabled on the node running HS2. If enable try disabling THP and see if it helps # Disable THP echo never | sudo tee -a /sys/kernel/mm/transparent_hugepage/enabled echo never | sudo tee -a /sys/kernel/mm/transparent_hugepage/defrag Thanks Prasanth On Mar

Best way/tool to debug memory leaks in HiveServer2

2018-03-13 Thread rajan sthapit
Hi, I am currently trying to debug a memory leak in a HiveServer2 java process. The heap size for the jvm is set to 16GB. When I look at the heap usage it is well under 16 GB, However, if I look at the total memory consumed by the process(top command) it's more than 30 GB. Currently, I am