[
https://issues.apache.org/jira/browse/AMBARI-20225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890969#comment-15890969
]
Swapan Shridhar commented on AMBARI-20225:
------------------------------------------
commit
trunk:
{code}
commit baefbcca0b60caac801f3addf0d015df23a5e96d
Author: Swapan Shridhar <[email protected]>
Date: Mon Feb 27 22:56:54 2017 -0800
AMBARI-20225. Take care of hive-interactive-site's
'hive.tez.container.size' config during HDP upgrade from 2.5 to 2.6 and update
default value for tez-interactive-site's config 'tez.am.resource.memory.mb'.
{code}
branch-2.5:
{code}
commit ce1ed6e1aade5333c0ee26de469e8487061ccb8b
Author: Swapan Shridhar <[email protected]>
Date: Mon Feb 27 22:56:54 2017 -0800
AMBARI-20225. Take care of hive-interactive-site's
'hive.tez.container.size' config during HDP upgrade from 2.5 to 2.6 and update
default value for tez-interactive-site's config 'tez.am.resource.memory.mb'.
{code}
> Take care of hive-interactive-site's 'hive.tez.container.size' config during
> HDP upgrade from 2.5 to 2.6 and update default value for
> tez-interactive-site's config 'tez.am.resource.memory.mb'.
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: AMBARI-20225
> URL: https://issues.apache.org/jira/browse/AMBARI-20225
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: trunk, 2.5.0
> Reporter: Swapan Shridhar
> Assignee: Swapan Shridhar
> Fix For: trunk, 2.5.0
>
> Attachments: AMBARI-20225.patch
>
>
> *Background :*
> We are dealing with 2 specific configs here.
> *(1).* 'tez.am.resource.memory.mb' for tez-interactive-site and
> *(2).* 'hive.tez.container.size' for hive-interactive-site.
> - 'tez.am.resource.memory.mb' already exists in tez-interactive-site from
> Ambari-2.4 onwards and recently its default value had been changed to
> 'SET_ON_FIRST_INVOCATION' in its hive-interacive-site.
> - 'hive.tez.container.size' was inherited and used from Hive1/hive-site in
> Ambari 2.4. Starting from Ambari 2.5, it was added as a config explicitly in
> hive-interactive-site with a default value of 'SET_ON_FIRST_INVOCATION'.
> The non-standard 'SET_ON_FIRST_INVOCATION' was introduced, so as to give a
> hint in Stack Advisor (SA) call for the 1st time on Hive Server Interactive
> ON (enable_hive_interactive config), so that config values for both can be
> calculated and set for the 1st time. Subsequesnt calls in the lifecycle, will
> just read the value set by SA or if later changed by user manually.
> *Problem :*
> 'SET_ON_FIRST_INVOCATION', being non-standard was breaking the code in
> upgrade scenario.
> *Fix:*
> Removing the use of 'SET_ON_FIRST_INVOCATION' and adding code for correct
> behavior of configs during HDP upgrade.
>
> *Possible scenarios:*
> *1. Ambari upgrade from 2.4 - > 2.5 having HDP 2.5. Implying
> 2.5/stack_advisor.py code would have been updated with new LLAP GA logic.*
> - 'tez.am.resource.memory.mb' will already be there whether Hive Server
> Interactive (HSI) is ON or OFF after Ambari upgrade.
> - If HSI if OFF before the upgrade was done, nothing to do over there.
> - If HSI on ON before AMbari upgrade, or is made ON after Ambari
> upgrade, SA logic has been updated to take of reading
> 'hive.tez.container.size' from hive-site as it wont exist in
> hive-interactive-site at this point. It will show up in hive-interative-site
> aftre the HDP upgrade to 2.6
> *2. Ambari 2.4 having HDP 2.5 -> Upgrade to Ambari 2.5 -> Upgraded to HDP 2.6*
> - 'tez.am.resource.memory.mb' will already be there
> - As part of HDP 2.5 upgrade to HDP 2.6, 'hive.tez.container.size' would
> have got added, having current value picked from
> hive-site/hive.tez.container.size. SA code carries a calculation logic for
> config 'hive.tez.container.size', which is bound to yield different value
> compared to hive-site/'hive.tez.container.size'. The config's value will get
> updated/caluclated (1). when HSI is made ON (meaning
> 'enabled_hive_interactive' config detected as **True** in
> changed-configurations).
>
> *3. Ambari 2.5 with HDP 2.5 -> Upgraded to HDP 2.6*
> - Both 'hive.tez.container.size' and 'tez.am.resource.memory.mb' (alreday
> present at HDP 2.5 in hive-interactive-site) will exist after the HDP upgrade.
> *Note* that a fresh value for 'hive.tez.container.size' and
> 'tez.am.resource.memory.mb' is caluclated at below conditions:
> (1). when HSI is made ON, meaning 'enabled_hive_interactive' config detected
> as **True** in changed-configurations, or
> (2). it's a cluster create operation, where HSI also is supposed to be
> started. (BP scenario)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
