[
https://issues.apache.org/jira/browse/AMBARI-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hurley updated AMBARI-19130:
-------------------------------------
Status: Patch Available (was: Open)
> Downgrade Can Create Multiple Mappings For Latest Configs
> ---------------------------------------------------------
>
> Key: AMBARI-19130
> URL: https://issues.apache.org/jira/browse/AMBARI-19130
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.2.0
> Reporter: Jonathan Hurley
> Assignee: Jonathan Hurley
> Priority: Critical
> Fix For: 2.5.0
>
> Attachments: AMBARI-19130.patch
>
>
> During a downgrade which crosses major stack versions (such as from HDP 2.y
> to HDP 2.x), Ambari attempts to set the latest configurations from HDP 2.x as
> "current". However, after the downgrade succeeds, the database fails with the
> following consistency check:
> {code}
> ERROR - You have config(s), in cluster c1, that is(are) selected more than
> once in clusterconfigmapping table:
> ranger-storm-plugin-properties,mapred-env,webhcat-env,hive-exec-log4j,hive-log4j,webhcat-log4j,storm-env,yarn-log4j,hcat-env
> {code}
> The problem is actually not with {{clusterconfig}} but with the state of the
> {{clusterconfigmapping}} table _before_ upgrade. Apparently, it is possible
> to have multiple mappings for the same config. We match on the config type
> and tag (such as hdfs-site / version1234566789). There should never, EVER be
> duplicate mappings for the same config and version tag.
> The current downgrade code doesn't "break" after finding its first match:
> {code}
> for(ClusterConfigMappingEntity configMappingEntity:
> configMappingEntities){
> String type = configMappingEntity.getType();
> String tag = configMappingEntity.getTag();
> for (ClusterConfigMappingEntity latest : latestConfigMappingByStack) {
> String latestType = latest.getType();
> String latestTag = latest.getTag();
> // find the latest config of a given mapping entity
> if (StringUtils.equals(type, latestType) && StringUtils.equals(tag,
> latestTag)) {
> LOG.info("{} with version tag {} is selected for stack {}", type,
> tag, stackId.toString());
> configMappingEntity.setSelected(1);
> }
> }
> }
> {code}
> I'm not sure that breaking will work here since we don't know the ordering.
> We could order the results and then break on the first match. In any event,
> it's a problem. But it's only a problem for the latest version of a config
> since that's what we're going to try to make selected.
> Here's a query from a test cluster which shows that if this cluster were to
> be downgraded, 5 configs would have multiple mappings created. This is
> because for the latest config, there are multiple mappings in
> {{clusterconfigmapping}}.
> {noformat}
> SELECT
> mapping.type_name,
> mapping.version_tag,
> COUNT(*)
> FROM clusterconfigmapping mapping
> JOIN (SELECT
> config.type_name,
> config.version_tag,
> MAX(config.version) AS latest_version
> FROM clusterconfig config
> GROUP BY config.type_name) AS latestConfig
> ON latestConfig.type_name = mapping.type_name
> AND latestConfig.version_tag = mapping.version_tag
> GROUP BY
> type_name, version_tag
> HAVING COUNT(*) > 1
> +-------------------------------+----------------------+----------+
> | type_name | version_tag | COUNT(*) |
> +-------------------------------+----------------------+----------+
> | hcat-env | version1 | 5 |
> | hive-exec-log4j | version1 | 5 |
> | hive-log4j | version1 | 5 |
> | ranger-hive-plugin-properties | version1436918769763 | 3 |
> | webhcat-env | version1 | 5 |
> +-------------------------------+----------------------+----------+
> 5 rows in set (0.00 sec)
> {noformat}
> STR:
> # Install a cluster, and instrument the {{clusterconfigmapping}} table to
> have multiple mappings for a given type (with only 1 currently selected).
> # Upgrade and then downgrade
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
