[jira] [Assigned] (ATLAS-4153) [Atlas: Spooling] The order of the entities created in atlas is not same as the order created in hive

2021-05-10 Thread Radhika Kundam (Jira) 


 [ 
https://issues.apache.org/jira/browse/ATLAS-4153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Radhika Kundam reassigned ATLAS-4153:
-

  Assignee: Radhika Kundam  (was: Ashutosh Mestry)
Resolution: Fixed

> [Atlas: Spooling] The order of the entities created in atlas is not same as 
> the order created in hive
> -
>
> Key: ATLAS-4153
> URL: https://issues.apache.org/jira/browse/ATLAS-4153
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Radhika Kundam
>Priority: Major
> Attachments: 
> ATLAS-4153-Spooling-Messages-out-of-order-during-res.patch
>
>
> *Steps to re-produce:*
>  * Stop kafka broker
>  * Create a few (20) tables save the prefix (abc_table_1, abc_table_2, ... 
> abc_table_n)
>  * Make sure the data is spooled
>  * Start kafka and create a few more tables (xyz_table_1, xyz_table_2, ... 
> xyz_table_n)
>  * Wait for 5 mins for the tables to reflect in atlas
>  * Fire a basic search with abc_* prefix and verify the tables are created
>  * Collect the createdTime of all the table and verify the order
> Here tables were created with prefix btwxb_table.
> Guid: a55ca87c-1122-4ed4-9812-5047cc8a7968 of *btwxb_table_6* has 
> "createTime": *1612959902106*,
> {code:java}
> {
> "referredEntities": {...},
> "entity": {
> "typeName": "hive_table",
> "attributes": {
> "owner": "hrt_qa",
> "temporary": false,
> "lastAccessTime": 1612959527000,
> "aliases": null,
> "replicatedTo": null,
> "userDescription": null,
> "replicatedFrom": null,
> "qualifiedName": "default.btwxb_table_6@cm",
> "displayName": null,
> "columns": [
> {
> "guid": "3cf7b33b-7fb7-4de4-a967-7f89410b3a00",
> "typeName": "hive_column"
> },
> {
> "guid": "7c5b72ae-e100-467a-b3b2-caade0b7a7a1",
> "typeName": "hive_column"
> }
> ],
> "description": null,
> "viewExpandedText": null,
> "tableType": "MANAGED_TABLE",
> "sd": {
> "guid": "7a1e2b04-f786-4f42-a3c4-31a0ad872256",
> "typeName": "hive_storagedesc"
> },
> "createTime": 1612959527000,
> "name": "btwxb_table_6",
> "comment": null,
> "partitionKeys": [],
> "parameters": {...},
> "retention": 0,
> "viewOriginalText": null,
> "db": {
> "guid": "a8a3227f-ee7a-49c2-8f9e-98534a3146eb",
> "typeName": "hive_db"
> }
> },
> "guid": "a55ca87c-1122-4ed4-9812-5047cc8a7968",
> "isIncomplete": false,
> "status": "ACTIVE",
> "createdBy": "hive",
> "updatedBy": "hrt_qa",
> "createTime": 1612959902106,
> "updateTime": 1612959911756,
> "version": 0,
> "relationshipAttributes": {...},
> "labels": []
> }
> } {code}
> Guid: 70738a7e-d29f-4234-82e7-adbdc4feaed6 of *btwxb_table_1* has 
> "createTime": *1612959903892*, 
> {code:java}
> {
> "referredEntities": {...},
> "entity": {
> "typeName": "hive_table",
> "attributes": {
> "owner": "hrt_qa",
> "temporary": false,
> "lastAccessTime": 1612959523000,
> "aliases": null,
> "replicatedTo": null,
> "userDescription": null,
> "replicatedFrom": null,
> "qualifiedName": "default.btwxb_table_1@cm",
> "displayName": null,
> "columns": [
> {
> "guid": "c283ee8c-d292-406a-a062-d81b12614636",
> "typeName": "hive_column"
> },
> {
> "guid": "cbb19940-e513-4295-a145-5b2d6691ea13",
> "typeName": "hive_column"
> }
> ],
> "description": null,
> "viewExpandedText": null,
> "tableType": "MANAGED_TABLE",
> "sd": {
> "guid": "69d7f6fc-8194-412f-a723-5b142ade1c31",
> "typeName": "hive_storagedesc"
> },
> "createTime": 1612959523000,
> "name": "btwxb_table_1",
> "comment": null,
> "partitionKeys": [],
> "parameters": {...},
> "retention": 0,
> "viewOriginalText": null,
> "db": {
> "guid": "a8a3227f-ee7a-49c2-8f9e-98534a3146eb",
>

[jira] [Assigned] (ATLAS-4153) [Atlas: Spooling] The order of the entities created in atlas is not same as the order created in hive

2021-05-10 Thread Ashutosh Mestry (Jira) 


 [ 
https://issues.apache.org/jira/browse/ATLAS-4153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry reassigned ATLAS-4153:
--

Assignee: Ashutosh Mestry

> [Atlas: Spooling] The order of the entities created in atlas is not same as 
> the order created in hive
> -
>
> Key: ATLAS-4153
> URL: https://issues.apache.org/jira/browse/ATLAS-4153
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Ashutosh Mestry
>Priority: Major
>
> *Steps to re-produce:*
>  * Stop kafka broker
>  * Create a few (20) tables save the prefix (abc_table_1, abc_table_2, ... 
> abc_table_n)
>  * Make sure the data is spooled
>  * Start kafka and create a few more tables (xyz_table_1, xyz_table_2, ... 
> xyz_table_n)
>  * Wait for 5 mins for the tables to reflect in atlas
>  * Fire a basic search with abc_* prefix and verify the tables are created
>  * Collect the createdTime of all the table and verify the order
> Here tables were created with prefix btwxb_table.
> Guid: a55ca87c-1122-4ed4-9812-5047cc8a7968 of *btwxb_table_6* has 
> "createTime": *1612959902106*,
> {code:java}
> {
> "referredEntities": {...},
> "entity": {
> "typeName": "hive_table",
> "attributes": {
> "owner": "hrt_qa",
> "temporary": false,
> "lastAccessTime": 1612959527000,
> "aliases": null,
> "replicatedTo": null,
> "userDescription": null,
> "replicatedFrom": null,
> "qualifiedName": "default.btwxb_table_6@cm",
> "displayName": null,
> "columns": [
> {
> "guid": "3cf7b33b-7fb7-4de4-a967-7f89410b3a00",
> "typeName": "hive_column"
> },
> {
> "guid": "7c5b72ae-e100-467a-b3b2-caade0b7a7a1",
> "typeName": "hive_column"
> }
> ],
> "description": null,
> "viewExpandedText": null,
> "tableType": "MANAGED_TABLE",
> "sd": {
> "guid": "7a1e2b04-f786-4f42-a3c4-31a0ad872256",
> "typeName": "hive_storagedesc"
> },
> "createTime": 1612959527000,
> "name": "btwxb_table_6",
> "comment": null,
> "partitionKeys": [],
> "parameters": {...},
> "retention": 0,
> "viewOriginalText": null,
> "db": {
> "guid": "a8a3227f-ee7a-49c2-8f9e-98534a3146eb",
> "typeName": "hive_db"
> }
> },
> "guid": "a55ca87c-1122-4ed4-9812-5047cc8a7968",
> "isIncomplete": false,
> "status": "ACTIVE",
> "createdBy": "hive",
> "updatedBy": "hrt_qa",
> "createTime": 1612959902106,
> "updateTime": 1612959911756,
> "version": 0,
> "relationshipAttributes": {...},
> "labels": []
> }
> } {code}
> Guid: 70738a7e-d29f-4234-82e7-adbdc4feaed6 of *btwxb_table_1* has 
> "createTime": *1612959903892*, 
> {code:java}
> {
> "referredEntities": {...},
> "entity": {
> "typeName": "hive_table",
> "attributes": {
> "owner": "hrt_qa",
> "temporary": false,
> "lastAccessTime": 1612959523000,
> "aliases": null,
> "replicatedTo": null,
> "userDescription": null,
> "replicatedFrom": null,
> "qualifiedName": "default.btwxb_table_1@cm",
> "displayName": null,
> "columns": [
> {
> "guid": "c283ee8c-d292-406a-a062-d81b12614636",
> "typeName": "hive_column"
> },
> {
> "guid": "cbb19940-e513-4295-a145-5b2d6691ea13",
> "typeName": "hive_column"
> }
> ],
> "description": null,
> "viewExpandedText": null,
> "tableType": "MANAGED_TABLE",
> "sd": {
> "guid": "69d7f6fc-8194-412f-a723-5b142ade1c31",
> "typeName": "hive_storagedesc"
> },
> "createTime": 1612959523000,
> "name": "btwxb_table_1",
> "comment": null,
> "partitionKeys": [],
> "parameters": {...},
> "retention": 0,
> "viewOriginalText": null,
> "db": {
> "guid": "a8a3227f-ee7a-49c2-8f9e-98534a3146eb",
> "typeName": "hive_db"
> }
> },
> "guid": "70738a7e-d29f-4234-82e7-adbdc4feaed6",
>