Sidharth Kumar Mishra created ATLAS-4508:
--------------------------------------------
Summary: Though a particular tag is blocked propagation , tag is
still propagated
Key: ATLAS-4508
URL: https://issues.apache.org/jira/browse/ATLAS-4508
Project: Atlas
Issue Type: Bug
Components: atlas-core
Reporter: Sidharth Kumar Mishra
Assignee: Sidharth Kumar Mishra
If we do the operation like Add classification and remove concurrently or Add
classification and block the same classification concurrently then most likely
due to deferred action we are hitting issue at Atlas and the Add/Remove/Block
is not working as expected. Example of Add and block happening concurrently
where the block is not working as expected:
{code:java}
2021-12-12 10:59:36,145 INFO -
[etp1330247343-177:hrt_qa:POST/api/atlas/v2/entity/bulk/classification] ~
Request from authenticated user: hrt_qa,
URL=/api/atlas/v2/entity/bulk/classification
(AtlasAuthenticationFilter$KerberosFilterChainWrapper:800)
2021-12-12 10:59:36,170 INFO - [etp1330247343-177 -
15c26538-375e-4352-b27e-34caf1d4a6f7:hrt_qa:POST/api/atlas/v2/entity/bulk/classification]
~ updateClassificationText: tag_zxpzn: tag_emufo_0 tag_qxnez tag_crqpn_0
tag_xldqo tag_xjdig tag_ksgxh tag_dpzcq tag_vszhd tag_zxpzn tag_crqpn_1
tag_crqpn_2 tag_hzjqk tag_tafen date Sun Dec 12 10:55:20 UTC 2021
hive_principal_type USER boolean false string nJzsc byte -21 double
1.2633179028830385E38 arr_string dYfsL mKlVp iCOwp ssfum iAaGF kcJWp dWxpm
VFsqP hYbVs short 24586 arr_int 476972899 729046663 1927464331 499332336
233258034 -670732418 -1304702956 205121029 float 1.54E38 int 1233345212 long
7716583360736583680 tag_ywwov date Thu Jan 01 00:00:00 UTC 1970 boolean false
byte 0 double 0.0 short 0 float 0.0 int 0 long 0 (EntityGraphMapper:2769)
2021-12-12 10:59:36,325 INFO - [etp1330247343-177 -
15c26538-375e-4352-b27e-34caf1d4a6f7:hrt_qa:POST/api/atlas/v2/entity/bulk/classification]
~
{"type":"CLASSIFICATION_PROPAGATION_ADD","guid":"32961799-d7e5-4aa1-a7b1-9172920f430b","createdBy":"hrt_qa","createdTime":1639306776164,"updatedTime":1639306776164,"parameters":{"relationshipGuid":null,"entityGuid":"b20848ce-4e8d-41d9-abdb-d0a846166e9e","classificationVertexId":"40972512"},"attemptCount":0,"status":"PENDING"}
(TaskExecutor$TaskLogger:170)
2021-12-12 10:59:36,368 INFO - [etp1330247343-570:] ~ Logged into Atlas as =
hrt_qa (AtlasAuthenticationFilter$KerberosFilterChainWrapper:787)
.....
....
2021-12-12 10:59:37,068 INFO - [etp1330247343-570 -
1b14bb3f-face-40f3-b4a4-cbac281ed685:hrt_qa:PUT/api/atlas/v2/relationship] ~
{"type":"CLASSIFICATION_PROPAGATION_RELATIONSHIP_UPDATE","guid":"9b36fa74-46e8-4bd4-9abc-2865354d0d46","createdBy":"hrt_qa","createdTime":1639306776679,"updatedTime":1639306776679,"parameters":{"relationshipEdgeId":"osjyc-okbog-4e1h-3v7k","relationshipObject":"{\"typeName\":\"dataset_process_inputs\",\"guid\":\"a2f03e60-d677-473b-ada5-dd77b21dc149\",\"provenanceType\":0,\"end1\":{\"guid\":\"3e9b2e57-1e32-4692-ac7a-a617ea0c4648\",\"typeName\":\"hive_process\",\"uniqueAttributes\":{\"qualifiedName\":\"default.table_xluwf_2@cm:1639305904000\"}},\"end2\":{\"guid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"typeName\":\"hive_table\",\"uniqueAttributes\":{\"qualifiedName\":\"default.table_xluwf_1@cm\"}},\"label\":\"__Process.inputs\",\"propagateTags\":\"TWO_TO_ONE\",\"status\":\"ACTIVE\",\"createdBy\":\"hrt_qa\",\"updatedBy\":\"hrt_qa\",\"createTime\":1639305904397,\"updateTime\":1639305904397,\"version\":0,\"propagatedClassifications\":[{\"typeName\":\"tag_qxnez\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_crqpn_1\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_ksgxh\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_tafen\",\"attributes\":{\"short\":24586,\"string\":\"nJzsc\",\"boolean\":false,\"double\":1.2633179028830385E+38,\"float\":1.54E+38,\"arr_string\":[\"dYfsL\",\"mKlVp\",\"iCOwp\",\"ssfum\",\"iAaGF\",\"kcJWp\",\"dWxpm\",\"VFsqP\",\"hYbVs\"],\"long\":7716583360736583680,\"arr_int\":[476972899,729046663,1927464331,499332336,233258034,-670732418,-1304702956,205121029],\"int\":1233345212,\"hive_principal_type\":\"USER\",\"date\":1639306520089,\"byte\":-21},\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_crqpn_0\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_ywwov\",\"attributes\":{\"short\":0,\"string\":null,\"boolean\":false,\"double\":0.0,\"float\":0.0,\"arr_string\":null,\"long\":0,\"arr_int\":null,\"int\":0,\"hive_principal_type\":null,\"date\":0,\"byte\":0},\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[{\"startTime\":\"2076/09/20
19:27:00\",\"endTime\":\"2076/09/21
01:10:00\",\"timeZone\":\"Pacific/Pohnpei\"},{\"startTime\":\"2053/09/20
14:14:00\",\"endTime\":\"2053/09/20
18:04:00\",\"timeZone\":\"Etc/GMT-12\"},{\"startTime\":\"1952/09/20
19:44:00\",\"endTime\":\"1952/09/20
23:31:00\",\"timeZone\":\"Asia/Tokyo\"}],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_vszhd\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_emufo_0\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_xjdig\",\"attributes\":{\"short\":null,\"string\":null,\"boolean\":null,\"double\":null,\"float\":null,\"arr_string\":null,\"long\":null,\"arr_int\":null,\"int\":null,\"hive_principal_type\":null,\"date\":null,\"byte\":null},\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_xldqo\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_crqpn_2\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_zxpzn\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false}],\"blockedPropagatedClassifications\":[{\"typeName\":\"tag_zxpzn\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false}]}"},"attemptCount":0,"status":"PENDING"}
(TaskExecutor$TaskLogger:170)
2021-12-12 10:59:37,075 INFO - [atlas-task-0-etp1330247343-570 -
6739bb3f-d2be-489f-90c0-1820a8f92a75:] ~ GraphTransaction intercept for
org.apache.atlas.repository.store.graph.v2.EntityGraphMapper.updateTagPropagations
(GraphTransactionAdvisor$1:41)
2021-12-12 10:59:37,108 INFO - [etp1330247343-180:] ~ Logged into Atlas as =
hrt_qa (AtlasAuthenticationFilter$KerberosFilterChainWrapper:787)
2021-12-12 10:59:37,110 INFO -
[etp1330247343-180:hrt_qa:GET/api/atlas/v2/relationship/guid/a2f03e60-d677-473b-ada5-dd77b21dc149]
~ Request from authenticated user: hrt_qa,
URL=/api/atlas/v2/relationship/guid/a2f03e60-d677-473b-ada5-dd77b21dc149
(AtlasAuthenticationFilter$KerberosFilterChainWrapper:800)
2021-12-12 10:59:37,158 INFO - [etp1330247343-177:] ~ Logged into Atlas as =
hrt_qa (AtlasAuthenticationFilter$KerberosFilterChainWrapper:787)
2021-12-12 10:59:37,158 INFO -
[etp1330247343-177:hrt_qa:GET/api/atlas/admin/tasks] ~ Request from
authenticated user: hrt_qa, URL=/api/atlas/admin/tasks
(AtlasAuthenticationFilter$KerberosFilterChainWrapper:800)
2021-12-12 10:59:37,205 INFO - [etp1330247343-570:] ~ Logged into Atlas as =
hrt_qa (AtlasAuthenticationFilter$KerberosFilterChainWrapper:787)
2021-12-12 10:59:37,205 INFO -
[etp1330247343-570:hrt_qa:GET/api/atlas/admin/tasks] ~ Request from
authenticated user: hrt_qa, URL=/api/atlas/admin/tasks
(AtlasAuthenticationFilter$KerberosFilterChainWrapper:800)
2021-12-12 10:59:38,181 INFO - [atlas-task-0-etp1330247343-570 -
6739bb3f-d2be-489f-90c0-1820a8f92a75:] ~
{"type":"CLASSIFICATION_PROPAGATION_RELATIONSHIP_UPDATE","guid":"9b36fa74-46e8-4bd4-9abc-2865354d0d46","createdBy":"hrt_qa","createdTime":1639306776679,"updatedTime":1639306776679,"endTime":1639306778043,"parameters":{"relationshipEdgeId":"osjyc-okbog-4e1h-3v7k","relationshipObject":"{\"typeName\":\"dataset_process_inputs\",\"guid\":\"a2f03e60-d677-473b-ada5-dd77b21dc149\",\"provenanceType\":0,\"end1\":{\"guid\":\"3e9b2e57-1e32-4692-ac7a-a617ea0c4648\",\"typeName\":\"hive_process\",\"uniqueAttributes\":{\"qualifiedName\":\"default.table_xluwf_2@cm:1639305904000\"}},\"end2\":{\"guid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"typeName\":\"hive_table\",\"uniqueAttributes\":{\"qualifiedName\":\"default.table_xluwf_1@cm\"}},\"label\":\"__Process.inputs\",\"propagateTags\":\"TWO_TO_ONE\",\"status\":\"ACTIVE\",\"createdBy\":\"hrt_qa\",\"updatedBy\":\"hrt_qa\",\"createTime\":1639305904397,\"updateTime\":1639305904397,\"version\":0,\"propagatedClassifications\":[{\"typeName\":\"tag_qxnez\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_crqpn_1\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_ksgxh\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_tafen\",\"attributes\":{\"short\":24586,\"string\":\"nJzsc\",\"boolean\":false,\"double\":1.2633179028830385E+38,\"float\":1.54E+38,\"arr_string\":[\"dYfsL\",\"mKlVp\",\"iCOwp\",\"ssfum\",\"iAaGF\",\"kcJWp\",\"dWxpm\",\"VFsqP\",\"hYbVs\"],\"long\":7716583360736583680,\"arr_int\":[476972899,729046663,1927464331,499332336,233258034,-670732418,-1304702956,205121029],\"int\":1233345212,\"hive_principal_type\":\"USER\",\"date\":1639306520089,\"byte\":-21},\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_crqpn_0\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_ywwov\",\"attributes\":{\"short\":0,\"string\":null,\"boolean\":false,\"double\":0.0,\"float\":0.0,\"arr_string\":null,\"long\":0,\"arr_int\":null,\"int\":0,\"hive_principal_type\":null,\"date\":0,\"byte\":0},\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[{\"startTime\":\"2076/09/20
19:27:00\",\"endTime\":\"2076/09/21
01:10:00\",\"timeZone\":\"Pacific/Pohnpei\"},{\"startTime\":\"2053/09/20
14:14:00\",\"endTime\":\"2053/09/20
18:04:00\",\"timeZone\":\"Etc/GMT-12\"},{\"startTime\":\"1952/09/20
19:44:00\",\"endTime\":\"1952/09/20
23:31:00\",\"timeZone\":\"Asia/Tokyo\"}],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_vszhd\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_emufo_0\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_xjdig\",\"attributes\":{\"short\":null,\"string\":null,\"boolean\":null,\"double\":null,\"float\":null,\"arr_string\":null,\"long\":null,\"arr_int\":null,\"int\":null,\"hive_principal_type\":null,\"date\":null,\"byte\":null},\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_xldqo\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_crqpn_2\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"tag_zxpzn\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false}],\"blockedPropagatedClassifications\":[{\"typeName\":\"tag_zxpzn\",\"entityGuid\":\"b20848ce-4e8d-41d9-abdb-d0a846166e9e\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"removePropagationsOnEntityDelete\":false}]}"},"attemptCount":0,"status":"COMPLETE"}
(TaskExecutor$TaskLogger:170)
....
..
2021-12-12 10:59:40,520 INFO - [atlas-task-0-etp1330247343-177 -
c4b8b5c4-4b2a-483c-b056-89e661b54534:] ~ updateClassificationText: tag_zxpzn:
tag_emufo_0 tag_crqpn_0 tag_xldqo tag_xjdig tag_ksgxh tag_emufo_1 tag_vszhd
tag_crqpn_1 tag_crqpn_2 tag_tafen date Sun Dec 12 10:55:20 UTC 2021
hive_principal_type USER boolean false string nJzsc byte -21 double
1.2633179028830385E38 arr_string dYfsL mKlVp iCOwp ssfum iAaGF kcJWp dWxpm
VFsqP hYbVs short 24586 arr_int 476972899 729046663 1927464331 499332336
233258034 -670732418 -1304702956 205121029 float 1.54E38 int 1233345212 long
7716583360736583680 tag_ywwov date Thu Jan 01 00:00:00 UTC 1970 boolean false
byte 0 double 0.0 short 0 float 0.0 int 0 long 0 (EntityGraphMapper:2769)
2021-12-12 10:59:40,520 INFO - [atlas-task-0-etp1330247343-177 -
c4b8b5c4-4b2a-483c-b056-89e661b54534:] ~ updateClassificationText: tag_zxpzn:
tag_emufo_0 tag_crqpn_0 tag_xldqo tag_xjdig tag_ksgxh tag_emufo_1 tag_vszhd
tag_crqpn_1 tag_crqpn_2 tag_tafen date Sun Dec 12 10:55:20 UTC 2021
hive_principal_type USER boolean false string nJzsc byte -21 double
1.2633179028830385E38 arr_string dYfsL mKlVp iCOwp ssfum iAaGF kcJWp dWxpm
VFsqP hYbVs short 24586 arr_int 476972899 729046663 1927464331 499332336
233258034 -670732418 -1304702956 205121029 float 1.54E38 int 1233345212 long
7716583360736583680 tag_ywwov date Thu Jan 01 00:00:00 UTC 1970 boolean false
byte 0 double 0.0 short 0 float 0.0 int 0 long 0 (EntityGraphMapper:2769)
2021-12-12 10:59:40,520 INFO - [atlas-task-0-etp1330247343-177 -
c4b8b5c4-4b2a-483c-b056-89e661b54534:] ~ updateClassificationText: tag_zxpzn:
tag_emufo_0 tag_crqpn_0 tag_xldqo tag_xjdig tag_ksgxh tag_emufo_1 tag_vszhd
tag_crqpn_1 tag_crqpn_2 tag_tafen date Sun Dec 12 10:55:20 UTC 2021
hive_principal_type USER boolean false string nJzsc byte -21 double
1.2633179028830385E38 arr_string dYfsL mKlVp iCOwp ssfum iAaGF kcJWp dWxpm
VFsqP hYbVs short 24586 arr_int 476972899 729046663 1927464331 499332336
233258034 -670732418 -1304702956 205121029 float 1.54E38 int 1233345212 long
7716583360736583680 tag_ywwov date Thu Jan 01 00:00:00 UTC 1970 boolean false
byte 0 double 0.0 short 0 float 0.0 int 0 long 0 (EntityGraphMapper:2769)
2021-12-12 10:59:40,521 INFO - [atlas-task-0-etp1330247343-177 -
c4b8b5c4-4b2a-483c-b056-89e661b54534:] ~ updateClassificationText: tag_zxpzn:
tag_emufo_0 tag_crqpn_0 tag_xldqo tag_xjdig tag_ksgxh tag_emufo_1 tag_vszhd
tag_crqpn_1 tag_crqpn_2 tag_tafen date Sun Dec 12 10:55:20 UTC 2021
hive_principal_type USER boolean false string nJzsc byte -21 double
1.2633179028830385E38 arr_string dYfsL mKlVp iCOwp ssfum iAaGF kcJWp dWxpm
VFsqP hYbVs short 24586 arr_int 476972899 729046663 1927464331 499332336
233258034 -670732418 -1304702956 205121029 float 1.54E38 int 1233345212 long
7716583360736583680 tag_ywwov date Thu Jan 01 00:00:00 UTC 1970 boolean false
byte 0 double 0.0 short 0 float 0.0 int 0 long 0 (EntityGraphMapper:2769)
....
...
2021-12-12 10:59:40,639 INFO - [atlas-task-0-etp1330247343-177 -
c4b8b5c4-4b2a-483c-b056-89e661b54534:] ~ updateClassificationText: tag_zxpzn:
tag_emufo_0 tag_crqpn_0 tag_xldqo tag_xjdig tag_ksgxh tag_emufo_1 tag_vszhd
tag_crqpn_1 tag_crqpn_2 tag_tafen date Sun Dec 12 10:55:20 UTC 2021
hive_principal_type USER boolean false string nJzsc byte -21 double
1.2633179028830385E38 arr_string dYfsL mKlVp iCOwp ssfum iAaGF kcJWp dWxpm
VFsqP hYbVs short 24586 arr_int 476972899 729046663 1927464331 499332336
233258034 -670732418 -1304702956 205121029 float 1.54E38 int 1233345212 long
7716583360736583680 tag_ywwov date Thu Jan 01 00:00:00 UTC 1970 boolean false
byte 0 double 0.0 short 0 float 0.0 int 0 long 0 (EntityGraphMapper:2769)
2021-12-12 10:59:41,944 INFO - [atlas-task-0-etp1330247343-177 -
c4b8b5c4-4b2a-483c-b056-89e661b54534:] ~
{"type":"CLASSIFICATION_PROPAGATION_ADD","guid":"32961799-d7e5-4aa1-a7b1-9172920f430b","createdBy":"hrt_qa","createdTime":1639306776164,"updatedTime":1639306776164,"endTime":1639306781813,"parameters":{"relationshipGuid":null,"entityGuid":"b20848ce-4e8d-41d9-abdb-d0a846166e9e","classificationVertexId":"40972512"},"attemptCount":0,"status":"COMPLETE"}
(TaskExecutor$TaskLogger:170){code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
