Dmitry Lysnichenko created AMBARI-20875:
-------------------------------------------
Summary: Removing A Service Causes DB Verification To Produce
Warnings
Key: AMBARI-20875
URL: https://issues.apache.org/jira/browse/AMBARI-20875
Project: Ambari
Issue Type: Bug
Reporter: Dmitry Lysnichenko
Assignee: Dmitry Lysnichenko
Attachments: AMBARI-20875.patch
When removing a service, the configurations for that service are kept for
historical purposes, but their various associations in the database are removed
(specifically, the {{serviceconfigmapping}} relationships).
After removing a service, the orphaned configurations now cause a warning to be
displayed on Ambari Server startup:
{noformat}
2017-04-06 17:15:24,003 WARN - You have config(s):
ranger-storm-policymgr-ssl-version1467149286586,atlas-env-version1471883877194,falcon-env-version1467044148480,storm-site-version1467149286586,storm-site-version1474944944095,ranger-storm-plugin-properties-version1467149286586,hana_hadoop-env-version1476989318735,hana_hadoop-env-version1468951412523,hanaes-site-version1475773173499,hanaes-site-version1477639131416,atlas-env-version1471880496396,falcon-startup.properties-version1474944962583,ranger-storm-security-version1467149286586,falcon-env-version1474944962517,application-properties-version1471883877194,hanaes-site-version1468951412523,application-properties-version1471992143777,application-properties-version1471880496396,hana_hadoop-env-version1475790068354,hana_hadoop-env-version1477639131416,falcon-runtime.properties-version1467044148480,atlas-env-version1471992143777,hana_hadoop-env-version1475773173499,storm-env-version1467149286586,hanaes-site-version1475790068354,hanaes-site-version1476902714170,atlas-env-version1471883827584,hana_hadoop-env-version1477695406433,hanaes-site-version1476989583427,falcon-log4j-version1,falcon-env-version1474944962457,hanaes-site-version1468959251565,falcon-client.properties-version1,atlas-env-version1471993347065,falcon-startup.properties-version1467044148480,storm-cluster-log4j-version1467149286586,hanaes-site-version1472285532383,hana_hadoop-env-version1477695089738,hana_hadoop-env-version1468959251565,hana_hadoop-env-version1476989821279,atlas-log4j-version1,storm-site-version1467612840864,storm-worker-log4j-version1467149286586,ranger-storm-audit-version1467149286586,application-properties-version1471993347065,application-properties-version1471883827584,hana_hadoop-env-version1477695579450
that is(are) not mapped (in serviceconfigmapping table) to any service!
{noformat}
These orphaned configurations have entries in both {{clusterconfig}} and
{{clusterconfigmapping}} but are otherwise not references anywhere. They don't
hurt anything, but do trigger this warning since we can't determine if they
_should_ have mappings in {{serviceconfigmapping}}.
A few options:
- When removing a service, remove configurations as well, leaving no orphans.
Some would argue that this is a bad move since re-adding the service later
would allow you to see the old configurations. I do not believe this is true
since the old configurations are never associated with the new service's
{{serviceconfig}} or {{serviceconfigmapping}}.
- Make the warning smarter somehow to ignore these since it's expected they are
orphaned.
-- Somehow determine the service which should own the config and see if it
exists in the cluster?
-- Add a new column to {{clusterconfig}} to mark it as deleted?
To clean these warnings, we had to:
{code}
CREATE TEMPORARY TABLE IF NOT EXISTS orphaned_configs AS
(SELECT
cc.config_id,
cc.type_name,
cc.version_tag
FROM clusterconfig cc, clusterconfigmapping ccm
WHERE cc.config_id NOT IN (SELECT
scm.config_id
FROM serviceconfigmapping scm)
AND cc.type_name != 'cluster-env'
AND cc.type_name = ccm.type_name
AND cc.version_tag = ccm.version_tag);
DELETE FROM clusterconfigmapping
WHERE EXISTS
(SELECT 1 FROM orphaned_configs
WHERE clusterconfigmapping.type_name = orphaned_configs.type_name AND
clusterconfigmapping.version_tag = orphaned_configs.version_tag);
DELETE FROM clusterconfig WHERE clusterconfig.config_id IN (SELECT config_id
FROM orphaned_configs);
SELECT * FROM orphaned_configs;
DROP TABLE orphaned_configs;
{code}
I've considered advanced heuristics based on service metainfo with config
dependencies, and service config mappings. But this approach may be unreliable
when given inaccurate service metainfo between stack upgrades. Also, this
approach is likely to delay server start on clusters with thousands of configs.
So the solution Add a new column to clusterconfig to mark it as deleted. So
warning will not be generated for such cluster configs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
