Dharshana M Krishnamoorthy created ATLAS-4153:
-------------------------------------------------
Summary: [Atlas: Spooling] The order of the entities created in
atlas is not same as the order created in hive
Key: ATLAS-4153
URL: https://issues.apache.org/jira/browse/ATLAS-4153
Project: Atlas
Issue Type: Bug
Components: atlas-core
Reporter: Dharshana M Krishnamoorthy
*Steps to re-produce:*
* Stop kafka broker
* Create a few (20) tables save the prefix (abc_table_1, abc_table_2, ...
abc_table_n)
* Make sure the data is spooled
* Start kafka and create a few more tables (xyz_table_1, xyz_table_2, ...
xyz_table_n)
* Wait for 5 mins for the tables to reflect in atlas
* Fire a basic search with abc_* prefix and verify the tables are created
* Collect the createdTime of all the table and verify the order
Here tables were created with prefix btwxb_table.
Guid: a55ca87c-1122-4ed4-9812-5047cc8a7968 of *btwxb_table_6* has "createTime":
*1612959902106*,
{code:java}
{
"referredEntities": {...},
"entity": {
"typeName": "hive_table",
"attributes": {
"owner": "hrt_qa",
"temporary": false,
"lastAccessTime": 1612959527000,
"aliases": null,
"replicatedTo": null,
"userDescription": null,
"replicatedFrom": null,
"qualifiedName": "default.btwxb_table_6@cm",
"displayName": null,
"columns": [
{
"guid": "3cf7b33b-7fb7-4de4-a967-7f89410b3a00",
"typeName": "hive_column"
},
{
"guid": "7c5b72ae-e100-467a-b3b2-caade0b7a7a1",
"typeName": "hive_column"
}
],
"description": null,
"viewExpandedText": null,
"tableType": "MANAGED_TABLE",
"sd": {
"guid": "7a1e2b04-f786-4f42-a3c4-31a0ad872256",
"typeName": "hive_storagedesc"
},
"createTime": 1612959527000,
"name": "btwxb_table_6",
"comment": null,
"partitionKeys": [],
"parameters": {...},
"retention": 0,
"viewOriginalText": null,
"db": {
"guid": "a8a3227f-ee7a-49c2-8f9e-98534a3146eb",
"typeName": "hive_db"
}
},
"guid": "a55ca87c-1122-4ed4-9812-5047cc8a7968",
"isIncomplete": false,
"status": "ACTIVE",
"createdBy": "hive",
"updatedBy": "hrt_qa",
"createTime": 1612959902106,
"updateTime": 1612959911756,
"version": 0,
"relationshipAttributes": {...},
"labels": []
}
} {code}
Guid: 70738a7e-d29f-4234-82e7-adbdc4feaed6 of *btwxb_table_1* has "createTime":
*1612959903892*,
{code:java}
{
"referredEntities": {...},
"entity": {
"typeName": "hive_table",
"attributes": {
"owner": "hrt_qa",
"temporary": false,
"lastAccessTime": 1612959523000,
"aliases": null,
"replicatedTo": null,
"userDescription": null,
"replicatedFrom": null,
"qualifiedName": "default.btwxb_table_1@cm",
"displayName": null,
"columns": [
{
"guid": "c283ee8c-d292-406a-a062-d81b12614636",
"typeName": "hive_column"
},
{
"guid": "cbb19940-e513-4295-a145-5b2d6691ea13",
"typeName": "hive_column"
}
],
"description": null,
"viewExpandedText": null,
"tableType": "MANAGED_TABLE",
"sd": {
"guid": "69d7f6fc-8194-412f-a723-5b142ade1c31",
"typeName": "hive_storagedesc"
},
"createTime": 1612959523000,
"name": "btwxb_table_1",
"comment": null,
"partitionKeys": [],
"parameters": {...},
"retention": 0,
"viewOriginalText": null,
"db": {
"guid": "a8a3227f-ee7a-49c2-8f9e-98534a3146eb",
"typeName": "hive_db"
}
},
"guid": "70738a7e-d29f-4234-82e7-adbdc4feaed6",
"isIncomplete": false,
"status": "ACTIVE",
"createdBy": "hive",
"updatedBy": "hrt_qa",
"createTime": 1612959903892,
"updateTime": 1612959919910,
"version": 0,
"relationshipAttributes": {...},
"labels": []
}
} {code}
*btwxb_table_1* has "createTime": *1612959903892*, and
*btwxb_table_6* has "createTime": *1612959902106*, which is smaller than
*btwxb_table_1's* createTime.
Which means it is created before table_6 is created before table_1 in atlas
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
