Dharshana M Krishnamoorthy created ATLAS-4480:
-------------------------------------------------
Summary: [API] Observing slowness in rest api response
Key: ATLAS-4480
URL: https://issues.apache.org/jira/browse/ATLAS-4480
Project: Atlas
Issue Type: Bug
Components: atlas-core
Reporter: Dharshana M Krishnamoorthy
Attachments: Screenshot 2021-11-15 at 3.51.10 PM.png
Observing slowness in the api response
Eg: Scenario
# Create a entity
# Soft delete it
# Purge the entity
# Verify purge audit
Here we fire a admin api call immediately after the entity is purged
But the admin api response is empty []
If we query after some time, we can see the audit is present for the purge call
!Screenshot 2021-11-15 at 3.51.10 PM.png|width=377,height=180!
Logs showing the above steps
{code:java}
2021-11-12 10:34:17,702|INFO|MainThread|atlasv2.py:659 -
get_entity_def()|https://quasar-gmxrxz-2.quasar-gmxrxz.root.hwx.site:31443/api/atlas/v2/entity/guid/e15aeb8e-cfb2-4b9d-96f1-b9bce94586e4
2021-11-12 10:34:17,703|INFO|MainThread|atlas.py:1135 -
http_get_request()|https://quasar-gmxrxz-2.quasar-gmxrxz.root.hwx.site:31443/api/atlas/v2/entity/guid/e15aeb8e-cfb2-4b9d-96f1-b9bce94586e4
2021-11-12 10:34:17,703|INFO|MainThread|atlas.py:1202 -
http_request()|https://quasar-gmxrxz-2.quasar-gmxrxz.root.hwx.site:31443/api/atlas/v2/entity/guid/e15aeb8e-cfb2-4b9d-96f1-b9bce94586e4
2021-11-12 10:34:17,704|INFO|MainThread|atlas.py:1209 - http_request()|HTTP
Method: GET, Body: None
2021-11-12 10:34:17,705|INFO|MainThread|atlas.py:1238 - http_request()|Making
HTTP requests via Kerberos auth
2021-11-12 10:34:17,787|INFO|MainThread|atlas.py:1257 - http_request()|HTTP
response code: 200
2021-11-12 10:34:17,790|INFO|MainThread|atlas.py:807 - set_base_url()|Base url:
https://quasar-gmxrxz-2.quasar-gmxrxz.root.hwx.site:31443
2021-11-12 10:34:17,791|INFO|MainThread|atlas.py:1202 -
http_request()|https://quasar-gmxrxz-2.quasar-gmxrxz.root.hwx.site:31443/api/atlas/admin/purge
2021-11-12 10:34:17,791|INFO|MainThread|atlas.py:1209 - http_request()|HTTP
Method: PUT, Body: ["e15aeb8e-cfb2-4b9d-96f1-b9bce94586e4"]
2021-11-12 10:34:17,792|INFO|MainThread|atlas.py:1238 - http_request()|Making
HTTP requests via Kerberos auth
2021-11-12 10:34:18,184|INFO|MainThread|atlas.py:1257 - http_request()|HTTP
response code: 200
2021-11-12 10:34:18,186|INFO|MainThread|utils.py:18 -
purge_entity_and_verify_status()|{u'mutatedEntities': {u'PURGE': [{u'status':
u'DELETED', u'isIncomplete': False, u'guid':
u'9c9d8fc0-3f81-4d24-987f-60172e776d84', u'classifications': [], u'labels': [],
u'typeName': u'hive_column', u'meaningNames': [], u'displayText': u'id',
u'meanings': [], u'attributes': {u'owner': u'hrt_qa', u'qualifiedName':
u'default.table_lxjgr.id@cm', u'name': u'id'}, u'classificationNames': []},
{u'status': u'DELETED', u'isIncomplete': False, u'guid':
u'6c7b8038-3c46-4d25-9ef3-c4bdb7ddb5a3', u'classifications': [], u'labels': [],
u'typeName': u'hive_column', u'meaningNames': [], u'displayText': u'name',
u'meanings': [], u'attributes': {u'owner': u'hrt_qa', u'qualifiedName':
u'default.table_lxjgr.name@cm', u'name': u'name'}, u'classificationNames': []},
{u'status': u'DELETED', u'isIncomplete': False, u'guid':
u'e15aeb8e-cfb2-4b9d-96f1-b9bce94586e4', u'classifications': [], u'labels': [],
u'typeName': u'hive_table', u'meaningNames': [], u'displayText':
u'table_lxjgr', u'meanings': [], u'attributes': {u'owner': u'hrt_qa',
u'qualifiedName': u'default.table_lxjgr@cm', u'createTime': 1636713248000,
u'name': u'table_lxjgr'}, u'classificationNames': []}, {u'status': u'DELETED',
u'isIncomplete': False, u'guid': u'6132f1b4-6efb-45af-8ec2-70f197aa16c2',
u'classifications': [], u'labels': [], u'typeName': u'hive_storagedesc',
u'meaningNames': [], u'displayText': u'default.table_lxjgr@cm_storage',
u'meanings': [], u'attributes': {u'qualifiedName':
u'default.table_lxjgr@cm_storage'}, u'classificationNames': []}, {u'status':
u'DELETED', u'isIncomplete': False, u'guid':
u'0354d287-c598-4325-b0ec-d48dfaf6fd9a', u'classifications': [], u'labels': [],
u'typeName': u'hive_table_ddl', u'meaningNames': [], u'displayText':
u'default.table_lxjgr@cm:1636713248787', u'meanings': [], u'attributes':
{u'qualifiedName': u'default.table_lxjgr@cm:1636713248787'},
u'classificationNames': []}]}}
2021-11-12 10:34:18,186|INFO|MainThread|utils.py:72 -
purge_async_deleted_entities()|{u'mutatedEntities': {u'PURGE': [{u'status':
u'DELETED', u'isIncomplete': False, u'guid':
u'9c9d8fc0-3f81-4d24-987f-60172e776d84', u'classifications': [], u'labels': [],
u'typeName': u'hive_column', u'meaningNames': [], u'displayText': u'id',
u'meanings': [], u'attributes': {u'owner': u'hrt_qa', u'qualifiedName':
u'default.table_lxjgr.id@cm', u'name': u'id'}, u'classificationNames': []},
{u'status': u'DELETED', u'isIncomplete': False, u'guid':
u'6c7b8038-3c46-4d25-9ef3-c4bdb7ddb5a3', u'classifications': [], u'labels': [],
u'typeName': u'hive_column', u'meaningNames': [], u'displayText': u'name',
u'meanings': [], u'attributes': {u'owner': u'hrt_qa', u'qualifiedName':
u'default.table_lxjgr.name@cm', u'name': u'name'}, u'classificationNames': []},
{u'status': u'DELETED', u'isIncomplete': False, u'guid':
u'e15aeb8e-cfb2-4b9d-96f1-b9bce94586e4', u'classifications': [], u'labels': [],
u'typeName': u'hive_table', u'meaningNames': [], u'displayText':
u'table_lxjgr', u'meanings': [], u'attributes': {u'owner': u'hrt_qa',
u'qualifiedName': u'default.table_lxjgr@cm', u'createTime': 1636713248000,
u'name': u'table_lxjgr'}, u'classificationNames': []}, {u'status': u'DELETED',
u'isIncomplete': False, u'guid': u'6132f1b4-6efb-45af-8ec2-70f197aa16c2',
u'classifications': [], u'labels': [], u'typeName': u'hive_storagedesc',
u'meaningNames': [], u'displayText': u'default.table_lxjgr@cm_storage',
u'meanings': [], u'attributes': {u'qualifiedName':
u'default.table_lxjgr@cm_storage'}, u'classificationNames': []}, {u'status':
u'DELETED', u'isIncomplete': False, u'guid':
u'0354d287-c598-4325-b0ec-d48dfaf6fd9a', u'classifications': [], u'labels': [],
u'typeName': u'hive_table_ddl', u'meaningNames': [], u'displayText':
u'default.table_lxjgr@cm:1636713248787', u'meanings': [], u'attributes':
{u'qualifiedName': u'default.table_lxjgr@cm:1636713248787'},
u'classificationNames': []}]}}
2021-11-12 10:34:18,187|INFO|MainThread|atlas.py:807 - set_base_url()|Base url:
https://quasar-gmxrxz-2.quasar-gmxrxz.root.hwx.site:31443
2021-11-12 10:34:18,190|INFO|MainThread|atlas.py:1202 -
http_request()|https://quasar-gmxrxz-2.quasar-gmxrxz.root.hwx.site:31443/api/atlas/admin/audits
2021-11-12 10:34:18,190|INFO|MainThread|atlas.py:1209 - http_request()|HTTP
Method: POST, Body: {"sortOrder": "DESCENDING", "limit": 1, "sortBy":
"startTime", "auditFilters": {"criterion": [{"operator": "like",
"attributeName": "userName", "attributeValue": "hrt_qa"}, {"operator": "like",
"attributeName": "operation", "attributeValue": "PURGE"}], "condition": "AND"},
"offset": 0}
2021-11-12 10:34:18,191|INFO|MainThread|atlas.py:1238 - http_request()|Making
HTTP requests via Kerberos auth
2021-11-12 10:34:18,259|INFO|MainThread|atlas.py:1257 - http_request()|HTTP
response code: 200
2021-11-12 10:34:18,260|INFO|MainThread|utils.py:26 - verify_purge_audit()|[]
{code}
Please Note the *[]* in the last line. The response is empty
This is due to the slowness and the data is present if we query after 2minutes
cc: [~sharmadhas]
This example is just one of such occurrence, this is seen in many scenarios
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
