Hi All, I need the following details. I am new to TEZ and could you please help to understand how TEZ works and how we can control the Number of mappers and can we execute instead of increasing the AM memory (tez.am.resource.memory.mb) and TEZ container size. We are dealing with huge volume of Data of size 4.5 TB per partition. Is TEZ advisable for these much high volume of data?
In our Cluster We are running a Hive query with TEZ as execution engine and we faced an issue like the below : Container *XYZ* is running beyond physical memory limits. Current usage: 1.1 GB of 1 GB physical memory used; 2.7 GB of 4 GB virtual memory used. Killing container Current AM memory was 1 GB and TEZ container size was 1 GB. I increased the AM memory to 6 GB and TEZ container size to 2 GB, then the query ran fine. My question is , Why we need to increase the AM memory here? what is the exact role AM is doing here in tez and different from MR. Regards, Raghu M
