[ https://issues.apache.org/jira/browse/ATLAS-4153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342221#comment-17342221 ]
ASF subversion and git services commented on ATLAS-4153: -------------------------------------------------------- Commit e6f78f28c8167c0099c1493fee51aac39abeebea in atlas's branch refs/heads/master from Radhika Kundam [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=e6f78f2 ] ATLAS-4153: [Atlas: Spooling] The order of the entities created in atlas is not same as the order created in hive Signed-off-by: Sarath Subramanian <sar...@apache.org> > [Atlas: Spooling] The order of the entities created in atlas is not same as > the order created in hive > ----------------------------------------------------------------------------------------------------- > > Key: ATLAS-4153 > URL: https://issues.apache.org/jira/browse/ATLAS-4153 > Project: Atlas > Issue Type: Bug > Components: atlas-core > Reporter: Dharshana M Krishnamoorthy > Assignee: Ashutosh Mestry > Priority: Major > Attachments: > ATLAS-4153-Spooling-Messages-out-of-order-during-res.patch > > > *Steps to re-produce:* > * Stop kafka broker > * Create a few (20) tables save the prefix (abc_table_1, abc_table_2, ... > abc_table_n) > * Make sure the data is spooled > * Start kafka and create a few more tables (xyz_table_1, xyz_table_2, ... > xyz_table_n) > * Wait for 5 mins for the tables to reflect in atlas > * Fire a basic search with abc_* prefix and verify the tables are created > * Collect the createdTime of all the table and verify the order > Here tables were created with prefix btwxb_table. > Guid: a55ca87c-1122-4ed4-9812-5047cc8a7968 of *btwxb_table_6* has > "createTime": *1612959902106*, > {code:java} > { > "referredEntities": {...}, > "entity": { > "typeName": "hive_table", > "attributes": { > "owner": "hrt_qa", > "temporary": false, > "lastAccessTime": 1612959527000, > "aliases": null, > "replicatedTo": null, > "userDescription": null, > "replicatedFrom": null, > "qualifiedName": "default.btwxb_table_6@cm", > "displayName": null, > "columns": [ > { > "guid": "3cf7b33b-7fb7-4de4-a967-7f89410b3a00", > "typeName": "hive_column" > }, > { > "guid": "7c5b72ae-e100-467a-b3b2-caade0b7a7a1", > "typeName": "hive_column" > } > ], > "description": null, > "viewExpandedText": null, > "tableType": "MANAGED_TABLE", > "sd": { > "guid": "7a1e2b04-f786-4f42-a3c4-31a0ad872256", > "typeName": "hive_storagedesc" > }, > "createTime": 1612959527000, > "name": "btwxb_table_6", > "comment": null, > "partitionKeys": [], > "parameters": {...}, > "retention": 0, > "viewOriginalText": null, > "db": { > "guid": "a8a3227f-ee7a-49c2-8f9e-98534a3146eb", > "typeName": "hive_db" > } > }, > "guid": "a55ca87c-1122-4ed4-9812-5047cc8a7968", > "isIncomplete": false, > "status": "ACTIVE", > "createdBy": "hive", > "updatedBy": "hrt_qa", > "createTime": 1612959902106, > "updateTime": 1612959911756, > "version": 0, > "relationshipAttributes": {...}, > "labels": [] > } > } {code} > Guid: 70738a7e-d29f-4234-82e7-adbdc4feaed6 of *btwxb_table_1* has > "createTime": *1612959903892*, > {code:java} > { > "referredEntities": {...}, > "entity": { > "typeName": "hive_table", > "attributes": { > "owner": "hrt_qa", > "temporary": false, > "lastAccessTime": 1612959523000, > "aliases": null, > "replicatedTo": null, > "userDescription": null, > "replicatedFrom": null, > "qualifiedName": "default.btwxb_table_1@cm", > "displayName": null, > "columns": [ > { > "guid": "c283ee8c-d292-406a-a062-d81b12614636", > "typeName": "hive_column" > }, > { > "guid": "cbb19940-e513-4295-a145-5b2d6691ea13", > "typeName": "hive_column" > } > ], > "description": null, > "viewExpandedText": null, > "tableType": "MANAGED_TABLE", > "sd": { > "guid": "69d7f6fc-8194-412f-a723-5b142ade1c31", > "typeName": "hive_storagedesc" > }, > "createTime": 1612959523000, > "name": "btwxb_table_1", > "comment": null, > "partitionKeys": [], > "parameters": {...}, > "retention": 0, > "viewOriginalText": null, > "db": { > "guid": "a8a3227f-ee7a-49c2-8f9e-98534a3146eb", > "typeName": "hive_db" > } > }, > "guid": "70738a7e-d29f-4234-82e7-adbdc4feaed6", > "isIncomplete": false, > "status": "ACTIVE", > "createdBy": "hive", > "updatedBy": "hrt_qa", > "createTime": 1612959903892, > "updateTime": 1612959919910, > "version": 0, > "relationshipAttributes": {...}, > "labels": [] > } > } {code} > Attributes's create time: > *btwxb_table_1 : 1612959523000* > *btwxb_table_6 : 1612959527000*** > In hive *btwxb_table_1* is created before *btwxb_table_6* > Entity's create time:** > *btwxb_table_1* has "createTime": *1612959903892*, and > *btwxb_table_6* has "createTime": *1612959902106*, which is smaller than > *btwxb_table_1's* createTime. > *Which means table_6 is created before table_1 in atlas* > -- This message was sent by Atlassian Jira (v8.3.4#803005)