Re: Fail to load table via Tez

2017-07-07 Thread Loïc Chanel
Hi Rajesh, Thanks for your quick answer. Seems like we also have a problem with the logs, as even with aggreagation enabled we get the following : /app-logs/yarn/logs/application_1499426430661_0113 does not exist. Log aggregation has not completed or is not enabled. Still, I tried what you

Re: Fail to load table via Tez

2017-07-07 Thread Rajesh Balamohan
You can run *"yarn logs -applicationId application_1499426430661_0113 > application_1499426430661_**0113.log"* to get the app logs. Would suggest you to try with *"hive --hiveconf tez.grouping.max-size=134217728 --hiveconf tez.grouping.min-size=** 134217728" *for running your hive query. You may

Fail to load table via Tez

2017-07-07 Thread Loïc Chanel
Hi guys, I'm having some troubles with Tez when I try to load some data stored in small JSON files in HDFS into a Hive table. At first I got some Out of memory exceptions, so I tried increasing the amount of memory allocated to Tez, until the problem turned to a GC Overhead limit exceeded after