Jonathan Hurley created AMBARI-19130:
----------------------------------------
Summary: Downgrade Can Create Multiple Mappings For Latest Configs
Key: AMBARI-19130
URL: https://issues.apache.org/jira/browse/AMBARI-19130
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.2.0
Reporter: Jonathan Hurley
Assignee: Jonathan Hurley
Priority: Critical
Fix For: 2.5.0
During a downgrade which crosses major stack versions (such as from HDP 2.y to
HDP 2.x), Ambari attempts to set the latest configurations from HDP 2.x as
"current". However, after the downgrade succeeds, the database fails with the
following consistency check:
{code}
ERROR - You have config(s), in cluster c1, that is(are) selected more than once
in clusterconfigmapping table:
ranger-storm-plugin-properties,mapred-env,webhcat-env,hive-exec-log4j,hive-log4j,webhcat-log4j,storm-env,yarn-log4j,hcat-env
{code}
The problem is actually not with {{clusterconfig}} but with the state of the
{{clusterconfigmapping}} table _before_ upgrade. Apparently, it is possible to
have multiple mappings for the same config. We match on the config type and tag
(such as hdfs-site / version1234566789). There should never, EVER be duplicate
mappings for the same config and version tag.
The current downgrade code doesn't "break" after finding its first match:
{code}
for(ClusterConfigMappingEntity configMappingEntity:
configMappingEntities){
String type = configMappingEntity.getType();
String tag = configMappingEntity.getTag();
for (ClusterConfigMappingEntity latest : latestConfigMappingByStack) {
String latestType = latest.getType();
String latestTag = latest.getTag();
// find the latest config of a given mapping entity
if (StringUtils.equals(type, latestType) && StringUtils.equals(tag,
latestTag)) {
LOG.info("{} with version tag {} is selected for stack {}", type,
tag, stackId.toString());
configMappingEntity.setSelected(1);
}
}
}
{code}
I'm not sure that breaking will work here since we don't know the ordering. We
could order the results and then break on the first match. In any event, it's a
problem. But it's only a problem for the latest version of a config since
that's what we're going to try to make selected.
Here's a query from a test cluster which shows that if this cluster were to be
downgraded, 5 configs would have multiple mappings created. This is because for
the latest config, there are multiple mappings in {{clusterconfigmapping}}.
{noformat}
SELECT
mapping.type_name,
mapping.version_tag,
COUNT(*)
FROM clusterconfigmapping mapping
JOIN (SELECT
config.type_name,
config.version_tag,
MAX(config.version) AS latest_version
FROM clusterconfig config
GROUP BY config.type_name) AS latestConfig
ON latestConfig.type_name = mapping.type_name
AND latestConfig.version_tag = mapping.version_tag
GROUP BY
type_name, version_tag
HAVING COUNT(*) > 1
+-------------------------------+----------------------+----------+
| type_name | version_tag | COUNT(*) |
+-------------------------------+----------------------+----------+
| hcat-env | version1 | 5 |
| hive-exec-log4j | version1 | 5 |
| hive-log4j | version1 | 5 |
| ranger-hive-plugin-properties | version1436918769763 | 3 |
| webhcat-env | version1 | 5 |
+-------------------------------+----------------------+----------+
5 rows in set (0.00 sec)
{noformat}
STR:
# Install a cluster, and instrument the {{clusterconfigmapping}} table to have
multiple mappings for a given type (with only 1 currently selected).
# Upgrade and then downgrade
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
