[
https://issues.apache.org/jira/browse/ATLAS-4460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Solbi Choi updated ATLAS-4460:
------------------------------
Description:
Problems
When one of the partitionKeys in hive table deleted, atlas search API still
gets all partitionKeys including deleted one. Adding `"excludeDeletedEntities":
True` in request json doesn't work in this situation.
Reproduce
* Create hive table with partition key and sync it using hive-import.
* Delete the hive table and re-create hive table with same name but without
partition key this time.
* Then you can the partitionKey deleted in the Atlas web view.
!스크린샷 2021-10-22 오후 5.07.47.png!
* But when trying search API to get the hive table entity using
{code:java}
request = { "typeName": "hive_table",
"attributes": [ "db", "name", "partitionKeys" ],
"excludeDeletedEntities": True,
"limit": limit,
"offset": offset
}
{code}
* You get the deleted partitionKey also.
**
{code:java}
'partitionKeys': [{'guid': '****', 'typeName': 'hive_column',
'uniqueAttributes': {'qualifiedName': 'foo.test_partition_drop.ds@primary'}},
{'guid': '****', 'typeName': 'hive_column', 'uniqueAttributes':
{'qualifiedName': 'foo.test_partition_drop.ts@primary'}}]{code}
was:
Problems
When one of the partitionKeys in hive table deleted, atlas search API still
gets all partitionKeys including deleted one. Adding `"excludeDeletedEntities":
True` in request json doesn't work in this situation.
Reproduce
* Create hive table with partition key and sync it using hive-import.
* Delete the hive table and re-create hive table with same name but without
partition key this time.
* Then you can the partitionKey deleted in the Atlas web view.
!스크린샷 2021-10-22 오후 5.07.47.png!
* But when trying search API to get the hive table entity using
** request = {
"typeName": "hive_table",
"attributes": [
"db",
"name",
"partitionKeys"
],
"entityFilters":
Unknown macro: \{ "criterion"}
,
"excludeDeletedEntities": True,
"limit": limit,
"offset": offset
}
* You get the deleted partitionKey also.
** 'partitionKeys': [\\{'guid': '****', 'typeName': 'hive_column',
'uniqueAttributes': {'qualifiedName': 'foo.test_partition_drop.ds@primary'}},
\\{'guid': '****', 'typeName': 'hive_column', 'uniqueAttributes':
{'qualifiedName': 'foo.test_partition_drop.ts@primary'}}]
> Search API gets deleted partitionKeys of Hive table
> ---------------------------------------------------
>
> Key: ATLAS-4460
> URL: https://issues.apache.org/jira/browse/ATLAS-4460
> Project: Atlas
> Issue Type: Bug
> Components: hive-integration
> Affects Versions: 2.2.0
> Reporter: Solbi Choi
> Priority: Major
> Attachments: 스크린샷 2021-10-22 오후 5.07.47.png
>
>
> Problems
> When one of the partitionKeys in hive table deleted, atlas search API still
> gets all partitionKeys including deleted one. Adding
> `"excludeDeletedEntities": True` in request json doesn't work in this
> situation.
>
> Reproduce
> * Create hive table with partition key and sync it using hive-import.
> * Delete the hive table and re-create hive table with same name but without
> partition key this time.
> * Then you can the partitionKey deleted in the Atlas web view.
> !스크린샷 2021-10-22 오후 5.07.47.png!
> * But when trying search API to get the hive table entity using
> {code:java}
> request = { "typeName": "hive_table",
> "attributes": [ "db", "name", "partitionKeys" ],
> "excludeDeletedEntities": True,
> "limit": limit,
> "offset": offset
> }
> {code}
> * You get the deleted partitionKey also.
> **
> {code:java}
> 'partitionKeys': [{'guid': '****', 'typeName': 'hive_column',
> 'uniqueAttributes': {'qualifiedName': 'foo.test_partition_drop.ds@primary'}},
> {'guid': '****', 'typeName': 'hive_column', 'uniqueAttributes':
> {'qualifiedName': 'foo.test_partition_drop.ts@primary'}}]{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
