[
https://issues.apache.org/jira/browse/RANGER-4956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Nadar updated RANGER-4956:
--------------------------------
Description:
When ranger.admin.supports.tags.dedup is set to true, i.e tag dedup feature is
enabled, we have found that the ranger plugin is not getting initialised using
the below rangerbaseplugin constructor.
public RangerBasePlugin(RangerPluginConfig pluginConfig, ServicePolicies
policies, ServiceTags tags, RangerRoles roles, RangerUserStore userStore,
ServiceGdsInfo gdsInfo)
Here we have tags data as below which is provided in the above constructor
{code:java}
{
"op": "add_or_update",
"serviceName": "hive",
"tagVersion": 7,
"tagDefinitions": {
"2": {
"id": 2,
"isEnabled": true,
"name": "TAG2",
"source": "Internal"
}
},
"tags": {
"10": {
"id": 10,
"isEnabled": true,
"type": "TAG2"
}
},
"serviceResources": [
{
"id": 8,
"isEnabled": true,
"resourceElements": {
"database": {
"values": [
"db1"
],
"isExcludes": false,
"isRecursive": false
}
},
"resourceSignature":
"7ea12bf936f94ba73833373104ca818c249dfea758ed3366b675e86f42f01983"
}
],
"resourceToTagIds": {
"8": [
10
]
},
"isDelta": false,
"tagsChangeExtent": "ALL",
"isTagsDeduped": true,
"id": 0
}{code}
and therefore we set the tags in the tag enricher using the below
tagEnricher.setServiceTags(tags);
Now in the below method
protected void setServiceTags(final ServiceTags serviceTags, final boolean
rebuildOnlyIndex)
we have this piece of code
{code:java}
if (!serviceTags.getIsDelta()) {
if (serviceTags.getIsTagsDeduped()) {
final int countOfDuplicateTags =
serviceTags.dedupTags();
LOG.info("Number of duplicate tags removed from the received serviceTags:[" +
countOfDuplicateTags + "]. Number of tags in the de-duplicated serviceTags :["
+ serviceTags.getTags().size() + "]."); }
processServiceTags(serviceTags);
}{code}
Here if serviceTags.getIsTagsDeduped() is true, we are deduping the tags again.
And when this is being triggered, an infinite for loop is triggered
inserviceTags serviceTags.dedupTags() method
Below is the code which goes in infinite loop
{code:java}
for (Long replacerTagId = replacedIds.get(tagId); replacerTagId != null;
replacerTagId = replacedIds.get(tagId)) {
tagId = replacerTagId;
} {code}
{10, 1} was already available in cachedTags.
Here replacedIds has \{10=10}
tagId is 10
and for this as data, the the for loop goes into infinite.
{*}Note{*}: This behaviour is seen in the below case:
1. We initialise the plugin with the above available tag data and
Rangerbaseplugin constructor. This works fine
2. Now we are trying to initialise a new Rangerbaseplugin with the same tags
and this is when the plugin is not initialised and the for loop goes into an
infinite state.
Suggestion:
{code:java}
if (!serviceTags.getIsDelta()) {
if (serviceTags.getIsTagsDeduped()) { final int countOfDuplicateTags =
serviceTags.dedupTags(); LOG.info("Number of duplicate tags removed from the
received serviceTags:[" + countOfDuplicateTags + "]. Number of tags in the
de-duplicated serviceTags :[" + serviceTags.getTags().size() + "]."); }
processServiceTags(serviceTags); } {code}
Should the above piece of code in RangerTagEnricher be modified to
{code:java}
if (!serviceTags.getIsDelta()) {
processServiceTags(serviceTags);
} {code}
Since tags are already deduped in ranger admin, I feel there is no need to
dedup it in the plugin end.
was:
When ranger.admin.supports.tags.dedup is set to true, i.e tag dedup feature is
enabled, we have found that the ranger plugin is not getting initialised using
the below rangerbaseplugin constructor.
public RangerBasePlugin(RangerPluginConfig pluginConfig, ServicePolicies
policies, ServiceTags tags, RangerRoles roles, RangerUserStore userStore,
ServiceGdsInfo gdsInfo)
Here we have tags data as below which is provided in the above constructor
{code:java}
{
"op": "add_or_update",
"serviceName": "hive",
"tagVersion": 7,
"tagDefinitions": {
"2": {
"id": 2,
"isEnabled": true,
"name": "TAG2",
"source": "Internal"
}
},
"tags": {
"10": {
"id": 10,
"isEnabled": true,
"type": "TAG2"
}
},
"serviceResources": [
{
"id": 8,
"isEnabled": true,
"resourceElements": {
"database": {
"values": [
"db1"
],
"isExcludes": false,
"isRecursive": false
}
},
"resourceSignature":
"7ea12bf936f94ba73833373104ca818c249dfea758ed3366b675e86f42f01983"
}
],
"resourceToTagIds": {
"8": [
10
]
},
"isDelta": false,
"tagsChangeExtent": "ALL",
"isTagsDeduped": true,
"id": 0
}{code}
and therefore we set the tags in the tag enricher using the below
tagEnricher.setServiceTags(tags);
Now in the below method
protected void setServiceTags(final ServiceTags serviceTags, final boolean
rebuildOnlyIndex)
we have this piece of code
{code:java}
if (!serviceTags.getIsDelta()) {
if (serviceTags.getIsTagsDeduped()) {
final int countOfDuplicateTags =
serviceTags.dedupTags();
LOG.info("Number of duplicate tags removed from the received serviceTags:[" +
countOfDuplicateTags + "]. Number of tags in the de-duplicated serviceTags :["
+ serviceTags.getTags().size() + "]."); }
processServiceTags(serviceTags);
}{code}
Here if serviceTags.getIsTagsDeduped() is true, we are deduping the tags again.
And when this is being triggered, an infinite for loop is triggered
inserviceTags serviceTags.dedupTags() method
Below is the code which goes in infinite loop
{code:java}
for (Long replacerTagId = replacedIds.get(tagId); replacerTagId != null;
replacerTagId = replacedIds.get(tagId)) {
tagId = replacerTagId;
} {code}
Here replacedIds has \{10=10}
tagId is 10
and for this as data, the the for loop goes into infinite.
Suggestion:
{code:java}
if (!serviceTags.getIsDelta()) {
if (serviceTags.getIsTagsDeduped()) { final int countOfDuplicateTags =
serviceTags.dedupTags(); LOG.info("Number of duplicate tags removed from the
received serviceTags:[" + countOfDuplicateTags + "]. Number of tags in the
de-duplicated serviceTags :[" + serviceTags.getTags().size() + "]."); }
processServiceTags(serviceTags); } {code}
Should the above piece of code in RangerTagEnricher be modified to
{code:java}
if (!serviceTags.getIsDelta()) {
processServiceTags(serviceTags);
} {code}
Since tags are already deduped in ranger admin, I feel there is no need to
dedup it in the plugin end.
> RangerBasePlugin is not getting initialised when tag dedup feature is enabled
> -----------------------------------------------------------------------------
>
> Key: RANGER-4956
> URL: https://issues.apache.org/jira/browse/RANGER-4956
> Project: Ranger
> Issue Type: Bug
> Components: plugins
> Reporter: Anand Nadar
> Priority: Major
>
> When ranger.admin.supports.tags.dedup is set to true, i.e tag dedup feature
> is enabled, we have found that the ranger plugin is not getting initialised
> using the below rangerbaseplugin constructor.
> public RangerBasePlugin(RangerPluginConfig pluginConfig, ServicePolicies
> policies, ServiceTags tags, RangerRoles roles, RangerUserStore userStore,
> ServiceGdsInfo gdsInfo)
> Here we have tags data as below which is provided in the above constructor
> {code:java}
> {
> "op": "add_or_update",
> "serviceName": "hive",
> "tagVersion": 7,
> "tagDefinitions": {
> "2": {
> "id": 2,
> "isEnabled": true,
> "name": "TAG2",
> "source": "Internal"
> }
> },
> "tags": {
> "10": {
> "id": 10,
> "isEnabled": true,
> "type": "TAG2"
> }
> },
> "serviceResources": [
> {
> "id": 8,
> "isEnabled": true,
> "resourceElements": {
> "database": {
> "values": [
> "db1"
> ],
> "isExcludes": false,
> "isRecursive": false
> }
> },
> "resourceSignature":
> "7ea12bf936f94ba73833373104ca818c249dfea758ed3366b675e86f42f01983"
> }
> ],
> "resourceToTagIds": {
> "8": [
> 10
> ]
> },
> "isDelta": false,
> "tagsChangeExtent": "ALL",
> "isTagsDeduped": true,
> "id": 0
> }{code}
> and therefore we set the tags in the tag enricher using the below
> tagEnricher.setServiceTags(tags);
> Now in the below method
> protected void setServiceTags(final ServiceTags serviceTags, final boolean
> rebuildOnlyIndex)
> we have this piece of code
> {code:java}
> if (!serviceTags.getIsDelta()) {
> if (serviceTags.getIsTagsDeduped()) {
> final int countOfDuplicateTags =
> serviceTags.dedupTags();
> LOG.info("Number of duplicate tags removed from the received serviceTags:[" +
> countOfDuplicateTags + "]. Number of tags in the de-duplicated serviceTags
> :[" + serviceTags.getTags().size() + "].");
> } processServiceTags(serviceTags);
> }{code}
> Here if serviceTags.getIsTagsDeduped() is true, we are deduping the tags
> again.
> And when this is being triggered, an infinite for loop is triggered
> inserviceTags serviceTags.dedupTags() method
> Below is the code which goes in infinite loop
> {code:java}
> for (Long replacerTagId = replacedIds.get(tagId); replacerTagId != null;
> replacerTagId = replacedIds.get(tagId)) {
> tagId = replacerTagId;
> } {code}
> {10, 1} was already available in cachedTags.
> Here replacedIds has \{10=10}
> tagId is 10
> and for this as data, the the for loop goes into infinite.
> {*}Note{*}: This behaviour is seen in the below case:
> 1. We initialise the plugin with the above available tag data and
> Rangerbaseplugin constructor. This works fine
> 2. Now we are trying to initialise a new Rangerbaseplugin with the same tags
> and this is when the plugin is not initialised and the for loop goes into an
> infinite state.
> Suggestion:
> {code:java}
> if (!serviceTags.getIsDelta()) {
> if (serviceTags.getIsTagsDeduped()) { final int countOfDuplicateTags =
> serviceTags.dedupTags(); LOG.info("Number of duplicate tags removed from the
> received serviceTags:[" + countOfDuplicateTags + "]. Number of tags in the
> de-duplicated serviceTags :[" + serviceTags.getTags().size() + "]."); }
> processServiceTags(serviceTags); } {code}
> Should the above piece of code in RangerTagEnricher be modified to
> {code:java}
> if (!serviceTags.getIsDelta()) {
> processServiceTags(serviceTags);
> } {code}
> Since tags are already deduped in ranger admin, I feel there is no need to
> dedup it in the plugin end.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
