Re: Review Request 49014: AMBARI-17331. Determine Tez for Hive2 config 'tez.am.resource.memory.mb' based on cluster capacity.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49014/#review138924 --- Ship it! Looks good. The Xmx change isn't required since it's not set for tez either (and tez will set this to 80% of the container size in this case) - Siddharth Seth On June 21, 2016, 7:59 p.m., Swapan Shridhar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/49014/ > --- > > (Updated June 21, 2016, 7:59 p.m.) > > > Review request for Ambari, Alejandro Fernandez and Sumit Mohanty. > > > Bugs: AMBARI-17331 > https://issues.apache.org/jira/browse/AMBARI-17331 > > > Repository: ambari > > > Description > --- > > The current computations done for tez ends up with a large AM size. > For LLAP, we'd like stack-advisor to change these computations. > > - Total Cluster Memory <=4GB - Tez AM size = 256MB and then normalized based > on YARN minimum container size. > - Total Cluster Memory >4GB && <= 72GB - Tez AM size = 512MB and then > normalized based on YARN minimum container size. > - Total Cluster Memory >72GB - Tez AM size = 1536MB and then normalized based > on YARN minimum container size. > > > Fix for safeguaring 'llap' Queue creation to max. size of 100%, in case the > minimum size required caluclated size is more than 100%. > > > Diffs > - > > > ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/tez-interactive-site.xml > 3c83c5c > ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py > 76654c3 > ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py > 1bc53ea > > Diff: https://reviews.apache.org/r/49014/diff/ > > > Testing > --- > > Python UT added. > - Python UT passes. > > > Thanks, > > Swapan Shridhar > >
Re: Review Request 49014: AMBARI-17331. Determine Tez for Hive2 config 'tez.am.resource.memory.mb' based on cluster capacity.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49014/ --- (Updated June 21, 2016, 7:59 p.m.) Review request for Ambari, Alejandro Fernandez and Sumit Mohanty. Changes --- Updated Test cases. Bugs: AMBARI-17331 https://issues.apache.org/jira/browse/AMBARI-17331 Repository: ambari Description --- The current computations done for tez ends up with a large AM size. For LLAP, we'd like stack-advisor to change these computations. - Total Cluster Memory <=4GB - Tez AM size = 256MB and then normalized based on YARN minimum container size. - Total Cluster Memory >4GB && <= 72GB - Tez AM size = 512MB and then normalized based on YARN minimum container size. - Total Cluster Memory >72GB - Tez AM size = 1536MB and then normalized based on YARN minimum container size. Fix for safeguaring 'llap' Queue creation to max. size of 100%, in case the minimum size required caluclated size is more than 100%. Diffs (updated) - ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/tez-interactive-site.xml 3c83c5c ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 76654c3 ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 1bc53ea Diff: https://reviews.apache.org/r/49014/diff/ Testing --- Python UT added. - Python UT passes. Thanks, Swapan Shridhar
Re: Review Request 49014: AMBARI-17331. Determine Tez for Hive2 config 'tez.am.resource.memory.mb' based on cluster capacity.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49014/#review138915 --- Ship it! Ship It! - Sumit Mohanty On June 21, 2016, 7:10 p.m., Swapan Shridhar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/49014/ > --- > > (Updated June 21, 2016, 7:10 p.m.) > > > Review request for Ambari, Alejandro Fernandez and Sumit Mohanty. > > > Bugs: AMBARI-17331 > https://issues.apache.org/jira/browse/AMBARI-17331 > > > Repository: ambari > > > Description > --- > > The current computations done for tez ends up with a large AM size. > For LLAP, we'd like stack-advisor to change these computations. > > - Total Cluster Memory <=4GB - Tez AM size = 256MB and then normalized based > on YARN minimum container size. > - Total Cluster Memory >4GB && <= 72GB - Tez AM size = 512MB and then > normalized based on YARN minimum container size. > - Total Cluster Memory >72GB - Tez AM size = 1536MB and then normalized based > on YARN minimum container size. > > > Fix for safeguaring 'llap' Queue creation to max. size of 100%, in case the > minimum size required caluclated size is more than 100%. > > > Diffs > - > > > ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/tez-interactive-site.xml > 3c83c5c > ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py > 76654c3 > ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py > 1bc53ea > > Diff: https://reviews.apache.org/r/49014/diff/ > > > Testing > --- > > Python UT added. > - Python UT passes. > > > Thanks, > > Swapan Shridhar > >
Re: Review Request 49014: AMBARI-17331. Determine Tez for Hive2 config 'tez.am.resource.memory.mb' based on cluster capacity.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49014/ --- (Updated June 21, 2016, 7:10 p.m.) Review request for Ambari, Alejandro Fernandez and Sumit Mohanty. Changes --- - Changed code to deposit/set normalized value for 'tez.am.resource.memory.mb'. - Aded fix for safeguaring 'llap' Queue creation to max. size of 100%, in case the minimum size required caluclated size is more than 100%. Bugs: AMBARI-17331 https://issues.apache.org/jira/browse/AMBARI-17331 Repository: ambari Description (updated) --- The current computations done for tez ends up with a large AM size. For LLAP, we'd like stack-advisor to change these computations. - Total Cluster Memory <=4GB - Tez AM size = 256MB and then normalized based on YARN minimum container size. - Total Cluster Memory >4GB && <= 72GB - Tez AM size = 512MB and then normalized based on YARN minimum container size. - Total Cluster Memory >72GB - Tez AM size = 1536MB and then normalized based on YARN minimum container size. Fix for safeguaring 'llap' Queue creation to max. size of 100%, in case the minimum size required caluclated size is more than 100%. Diffs (updated) - ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/tez-interactive-site.xml 3c83c5c ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 76654c3 ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py 1bc53ea Diff: https://reviews.apache.org/r/49014/diff/ Testing --- Python UT added. - Python UT passes. Thanks, Swapan Shridhar
Re: Review Request 49014: AMBARI-17331. Determine Tez for Hive2 config 'tez.am.resource.memory.mb' based on cluster capacity.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49014/#review138833 --- Ship it! Ship It! - Sumit Mohanty On June 21, 2016, 9:25 a.m., Swapan Shridhar wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/49014/ > --- > > (Updated June 21, 2016, 9:25 a.m.) > > > Review request for Ambari, Alejandro Fernandez and Sumit Mohanty. > > > Bugs: AMBARI-17331 > https://issues.apache.org/jira/browse/AMBARI-17331 > > > Repository: ambari > > > Description > --- > > The current computations done for tez ends up with a large AM size. > For LLAP, we'd like stack-advisor to change these computations. > > - Total Cluster Memory <=4GB - Tez AM size = 256MB and then normalized based > on YARN minimum container size. > - Total Cluster Memory >4GB && <= 72GB - Tez AM size = 512MB and then > normalized based on YARN minimum container size. > - Total Cluster Memory >72GB - Tez AM size = 1536MB and then normalized based > on YARN minimum container size. > > > Diffs > - > > > ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/tez-interactive-site.xml > 3c83c5c > ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py > 76654c3 > ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py > 1bc53ea > > Diff: https://reviews.apache.org/r/49014/diff/ > > > Testing > --- > > Python UT added. > - Python UT passes. > > > Thanks, > > Swapan Shridhar > >
