[
https://issues.apache.org/jira/browse/RANGER-4609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhay Kulkarni updated RANGER-4609:
-----------------------------------
Description:
As a part of test infrastructure, a file-based tag retriever class is created.
When the TagEnricher is configured with this class, a plugin gets the tag from
the configured file.
Please refer to the Service-definition for tag for tagRetrieverClassName in
enricherOptions.
If the value of this option is set to
org.apache.ranger.plugin.contextenricher.RangerFileBasedTagRetriever
then, the tags will be fetched for a file named by the value of the enricher
option
"serviceTagsFileName".
{code:java}
"contextEnrichers":[
{
"itemId": 1,
"name" : "TagEnricher",
"enricher" : "org.apache.ranger.plugin.contextenricher.RangerTagEnricher",
"enricherOptions" : {
"tagRetrieverClassName":
"org.apache.ranger.plugin.contextenricher.RangerAdminTagRetriever",
"tagRefresherPollingInterval": 60000
}
}]{code}
This enhancement supports specifying another enricher option "tagFileCount".
When this is set to a value greater than 0, each successive call made to fetch
tags will provide tags from the file name obtained by adding a suffix [0..
tagFileCount-1].json to the file named by the option "serviceTagsFileName".
When tag-delta files are set up with appropriate suffixes, this simulates the
fetching of successive tag-deltas for processing.
An example may as follows:
{code:java}
"contextEnrichers": [
{
"itemId": 1,
"name": "TagEnricher",
"enricher":
"org.apache.ranger.plugin.contextenricher.RangerTagEnricher",
"enricherOptions" :
{"tagRetrieverClassName":"org.apache.ranger.plugin.contextenricher.RangerFileBasedTagRetriever",
"tagRefresherPollingInterval":1000,
"serviceTagsFileName":"testdata/test_hive_tag.json", "deDupTags": "false",
"tagFileCount": "2"}
}
] {code}
where testdata directory contains the following files:
test_hive_tag.json :
{code:java}
{
"op": "add_or_update",
"serviceName": "test_hive",
"tagVersion": 5,
"tagDefinitions": {
"1": {
"name": "PII",
"source": "Atlas",
"attributeDefs": [],
"id": 1,
"isEnabled": true,
"version": 1
}
},
"tags": {
"1": {
"type": "PII",
"owner": 0,
"attributes": {},
"options": {},
"validityPeriods": [],
"id": 1,
"isEnabled": true
}
},
"serviceResources": [
{
"serviceName": "test_hive",
"resourceElements": {
"database": {
"values": [
"default"
],
"isExcludes": false,
"isRecursive": false
},
"table": {
"values": [
"sample_07"
],
"isExcludes": false,
"isRecursive": false
}
},
"id": 1,
"isEnabled": true,
"version": 2
}
],
"resourceToTagIds": {
"1": [
1
]
},
"isDelta": false,
"tagsChangeExtent": "ALL",
"cachedTags": {}
} {code}
file test_hive_tag.json_0.json
{code:java}
{
"op": "add_or_update",
"serviceName": "test_hive",
"tagVersion": 4,
"tagDefinitions": {
"2": {
"id": 2,
"isEnabled": true,
"version": 1,
"name": "TEST_TAG",
"source": "Atlas",
"attributeDefs": []
}
},
"tags": {
"1": {
"id": 1,
"isEnabled": true,
"type": "PII",
"owner": 0,
"attributes": {},
"options": {},
"validityPeriods": []
},
"2": {
"id": 2,
"isEnabled": true,
"type": "TEST_TAG",
"owner": 0,
"attributes": {},
"options": {},
"validityPeriods": []
}
},
"serviceResources": [
{
"id": 1,
"isEnabled": true,
"version": 4,
"serviceName": "test_hive",
"resourceElements": {
"database": {
"values": [
"default"
],
"isExcludes": false,
"isRecursive": false
},
"table": {
"values": [
"sample_07"
],
"isExcludes": false,
"isRecursive": false
}
}
}
],
"resourceToTagIds": {
"1": [
1,
2
]
},
"isDelta": true,
"tagsChangeExtent": "SERVICE_RESOURCE"
}
{code}
file test_hive_tag.json_0.json
{code:java}
{
"op": "add_or_update",
"serviceName": "test_hive",
"tagVersion": 5,
"tagDefinitions": {},
"tags": {
"1": {
"id": 1,
"isEnabled": true,
"type": "PII",
"owner": 0,
"attributes": {},
"options": {},
"validityPeriods": []
},
"2": {
"id": 2,
"isEnabled": true,
"owner": 0,
"attributes": {},
"options": {},
"validityPeriods": []
}
},
"serviceResources": [
{
"id": 1,
"isEnabled": true,
"version": 6,
"serviceName": "test_hive",
"resourceElements": {
"database": {
"values": [
"default"
],
"isExcludes": false,
"isRecursive": false
},
"table": {
"values": [
"sample_07"
],
"isExcludes": false,
"isRecursive": false
}
}
}
],
"resourceToTagIds": {
"1": [
1
]
},
"isDelta": true,
"tagsChangeExtent": "SERVICE_RESOURCE"
}
{code}
was:
As a part of test infrastructure, a file-based tag retriever class is created.
When the TagEnricher is configured with this class, a plugin gets the tag from
the configured file.
Please refer to the Service-definition for tag for tagRetrieverClassName in
enricherOptions.
If the value of this option is set to
org.apache.ranger.plugin.contextenricher.RangerFileBasedTagRetriever
then, the tags will be fetched for a file named by the value of the enricher
option
"serviceTagsFileName".
{code:java}
"contextEnrichers":[
{
"itemId": 1,
"name" : "TagEnricher",
"enricher" : "org.apache.ranger.plugin.contextenricher.RangerTagEnricher",
"enricherOptions" : {
"tagRetrieverClassName":
"org.apache.ranger.plugin.contextenricher.RangerAdminTagRetriever",
"tagRefresherPollingInterval": 60000
}
}]{code}
This enhancement supports specifying another enricher option "tagFileCount".
When this is set to a value greater than 0, each successive call made to fetch
tags will provide tags from the file name obtained by adding a suffix [0..
tagFileCount-1].json to the file named by the option "serviceTagsFileName".
When tag-delta files are set up with appropriate suffixes, this simulates the
fetching of successive tag-deltas for processing.
> Support in File-based Tag Retriever to provide tag-deltas
> ---------------------------------------------------------
>
> Key: RANGER-4609
> URL: https://issues.apache.org/jira/browse/RANGER-4609
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
> Reporter: Abhay Kulkarni
> Assignee: Abhay Kulkarni
> Priority: Major
>
> As a part of test infrastructure, a file-based tag retriever class is
> created. When the TagEnricher is configured with this class, a plugin gets
> the tag from the configured file.
> Please refer to the Service-definition for tag for tagRetrieverClassName in
> enricherOptions.
> If the value of this option is set to
> org.apache.ranger.plugin.contextenricher.RangerFileBasedTagRetriever
> then, the tags will be fetched for a file named by the value of the enricher
> option
> "serviceTagsFileName".
> {code:java}
> "contextEnrichers":[
> {
> "itemId": 1,
> "name" : "TagEnricher",
> "enricher" : "org.apache.ranger.plugin.contextenricher.RangerTagEnricher",
> "enricherOptions" : {
> "tagRetrieverClassName":
> "org.apache.ranger.plugin.contextenricher.RangerAdminTagRetriever",
> "tagRefresherPollingInterval": 60000
> }
> }]{code}
> This enhancement supports specifying another enricher option "tagFileCount".
> When this is set to a value greater than 0, each successive call made to
> fetch tags will provide tags from the file name obtained by adding a suffix
> [0.. tagFileCount-1].json to the file named by the option
> "serviceTagsFileName". When tag-delta files are set up with appropriate
> suffixes, this simulates the fetching of successive tag-deltas for processing.
>
> An example may as follows:
> {code:java}
> "contextEnrichers": [
> {
> "itemId": 1,
> "name": "TagEnricher",
> "enricher":
> "org.apache.ranger.plugin.contextenricher.RangerTagEnricher",
> "enricherOptions" :
> {"tagRetrieverClassName":"org.apache.ranger.plugin.contextenricher.RangerFileBasedTagRetriever",
> "tagRefresherPollingInterval":1000,
> "serviceTagsFileName":"testdata/test_hive_tag.json", "deDupTags": "false",
> "tagFileCount": "2"}
> }
> ] {code}
> where testdata directory contains the following files:
> test_hive_tag.json :
> {code:java}
> {
> "op": "add_or_update",
> "serviceName": "test_hive",
> "tagVersion": 5,
> "tagDefinitions": {
> "1": {
> "name": "PII",
> "source": "Atlas",
> "attributeDefs": [],
> "id": 1,
> "isEnabled": true,
> "version": 1
> }
> },
> "tags": {
> "1": {
> "type": "PII",
> "owner": 0,
> "attributes": {},
> "options": {},
> "validityPeriods": [],
> "id": 1,
> "isEnabled": true
> }
> },
> "serviceResources": [
> {
> "serviceName": "test_hive",
> "resourceElements": {
> "database": {
> "values": [
> "default"
> ],
> "isExcludes": false,
> "isRecursive": false
> },
> "table": {
> "values": [
> "sample_07"
> ],
> "isExcludes": false,
> "isRecursive": false
> }
> },
> "id": 1,
> "isEnabled": true,
> "version": 2
> }
> ],
> "resourceToTagIds": {
> "1": [
> 1
> ]
> },
> "isDelta": false,
> "tagsChangeExtent": "ALL",
> "cachedTags": {}
> } {code}
> file test_hive_tag.json_0.json
> {code:java}
> {
> "op": "add_or_update",
> "serviceName": "test_hive",
> "tagVersion": 4,
> "tagDefinitions": {
> "2": {
> "id": 2,
> "isEnabled": true,
> "version": 1,
> "name": "TEST_TAG",
> "source": "Atlas",
> "attributeDefs": []
> }
> },
> "tags": {
> "1": {
> "id": 1,
> "isEnabled": true,
> "type": "PII",
> "owner": 0,
> "attributes": {},
> "options": {},
> "validityPeriods": []
> },
> "2": {
> "id": 2,
> "isEnabled": true,
> "type": "TEST_TAG",
> "owner": 0,
> "attributes": {},
> "options": {},
> "validityPeriods": []
> }
> },
> "serviceResources": [
> {
> "id": 1,
> "isEnabled": true,
> "version": 4,
> "serviceName": "test_hive",
> "resourceElements": {
> "database": {
> "values": [
> "default"
> ],
> "isExcludes": false,
> "isRecursive": false
> },
> "table": {
> "values": [
> "sample_07"
> ],
> "isExcludes": false,
> "isRecursive": false
> }
> }
> }
> ],
> "resourceToTagIds": {
> "1": [
> 1,
> 2
> ]
> },
> "isDelta": true,
> "tagsChangeExtent": "SERVICE_RESOURCE"
> }
> {code}
> file test_hive_tag.json_0.json
> {code:java}
> {
> "op": "add_or_update",
> "serviceName": "test_hive",
> "tagVersion": 5,
> "tagDefinitions": {},
> "tags": {
> "1": {
> "id": 1,
> "isEnabled": true,
> "type": "PII",
> "owner": 0,
> "attributes": {},
> "options": {},
> "validityPeriods": []
> },
> "2": {
> "id": 2,
> "isEnabled": true,
> "owner": 0,
> "attributes": {},
> "options": {},
> "validityPeriods": []
> }
> },
> "serviceResources": [
> {
> "id": 1,
> "isEnabled": true,
> "version": 6,
> "serviceName": "test_hive",
> "resourceElements": {
> "database": {
> "values": [
> "default"
> ],
> "isExcludes": false,
> "isRecursive": false
> },
> "table": {
> "values": [
> "sample_07"
> ],
> "isExcludes": false,
> "isRecursive": false
> }
> }
> }
> ],
> "resourceToTagIds": {
> "1": [
> 1
> ]
> },
> "isDelta": true,
> "tagsChangeExtent": "SERVICE_RESOURCE"
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
