[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Description: 
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
 This enhances functionality provided by ATLAS-3324.

*Background*:
 Requirement for import API is that if request contains "replicatedFrom" 
option, in this case import API should find & delete tables which are not 
imported but are part of database entity.
 E.g. Consider t1...t5 under db1 replicated from source to destination using 
table level incremental export from source.
 Now suppose subsequent import request at destination contains only t1...t3, in 
such case t4 & t5 must be deleted at destination.

*Solution*:
 * Check if import request contains "replicatedFrom" option.
 * Extract exportRequest from imported zip.
 * Identify table entities which are imported using exportRequest.
 * Identify database entity using imported table entities.
 * Identify table entities which are part of database entity obtained in 
previous step but are not imported in current import request.
 * Delete tables.

  was:
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
 This enhances functionality provided by ATLAS-3324.

 

*Background*:
        Requirement for import API is that if request contains "replicatedFrom" 
option, in this case import API should find & delete tables which are not 
imported but are part of database entity.


        E.g. Consider tables t1...t5 under database db1, replicated from source 
to destination using table level incremental export.
 Now suppose subsequent import request at destination contains only t1...t3, in 
such case t4 & t5 must be deleted at destination.

 

*Solution approach*:
 * Check if import request contains "replicatedFrom" option.
 * Extract exportRequest from imported zip.
 * Identify table entities which are imported using exportRequest.
 * Identify database entity using imported table entities.
 * Identify table entities which are part of database entity obtained in 
previous step but are not imported in current import request.
 * Delete tables.


> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v11.patch, 
> Import-API-delete-non-exported-hive_table-for-table-v2-branch-0.8.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are present in the hive_db but are 
> not the part of the incremental export request.
>  This enhances functionality provided by ATLAS-3324.
> *Background*:
>  Requirement for import API is that if request contains "replicatedFrom" 
> option, in this case import API should find & delete tables which are not 
> imported but are part of database entity.
>  E.g. Consider t1...t5 under db1 replicated from source to destination using 
> table level incremental export from source.
>  Now suppose subsequent import request at destination contains only t1...t3, 
> in such case t4 & t5 must be deleted at destination.
> *Solution*:
>  * Check if import request contains "replicatedFrom" option.
>  * Extract exportRequest from imported zip.
>  * Identify table entities which are imported using exportRequest.
>  * Identify database entity using imported table entities.
>  * Identify table entities which are part of database entity obtained in 
> previous step but are not imported in current import request.
>  * Delete tables.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Description: 
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
 This enhances functionality provided by ATLAS-3324.

 

*Background*:
        Requirement for import API is that if request contains "replicatedFrom" 
option, in this case import API should find & delete tables which are not 
imported but are part of database entity.


        E.g. Consider tables t1...t5 under database db1, replicated from source 
to destination using table level incremental export.
 Now suppose subsequent import request at destination contains only t1...t3, in 
such case t4 & t5 must be deleted at destination.

 

*Solution approach*:
 * Check if import request contains "replicatedFrom" option.
 * Extract exportRequest from imported zip.
 * Identify table entities which are imported using exportRequest.
 * Identify database entity using imported table entities.
 * Identify table entities which are part of database entity obtained in 
previous step but are not imported in current import request.
 * Delete tables.

  was:
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
 This enhances functionality provided by ATLAS-3324.

*Background*:
 Requirement for import API is that if request contains "replicatedFrom" 
option, in this case import API should find & delete tables which are not 
imported but are part of database entity.
 E.g. Consider t1...t5 under db1 replicated from source to destination using 
table level incremental export from source.
 Now suppose subsequent import request at destination contains only t1...t3, in 
such case t4 & t5 must be deleted at destination.

*Solution approach*:
 * Check if import request contains "replicatedFrom" option.
 * Extract exportRequest from imported zip.
 * Identify table entities which are imported using exportRequest.
 * Identify database entity using imported table entities.
 * Identify table entities which are part of database entity obtained in 
previous step but are not imported in current import request.
 * Delete tables.


> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v11.patch, 
> Import-API-delete-non-exported-hive_table-for-table-v2-branch-0.8.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are present in the hive_db but are 
> not the part of the incremental export request.
>  This enhances functionality provided by ATLAS-3324.
>  
> *Background*:
>         Requirement for import API is that if request contains 
> "replicatedFrom" option, in this case import API should find & delete tables 
> which are not imported but are part of database entity.
>         E.g. Consider tables t1...t5 under database db1, replicated from 
> source to destination using table level incremental export.
>  Now suppose subsequent import request at destination contains only t1...t3, 
> in such case t4 & t5 must be deleted at destination.
>  
> *Solution approach*:
>  * Check if import request contains "replicatedFrom" option.
>  * Extract exportRequest from imported zip.
>  * Identify table entities which are imported using exportRequest.
>  * Identify database entity using imported table entities.
>  * Identify table entities which are part of database entity obtained in 
> previous step but are not imported in current import request.
>  * Delete tables.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Description: 
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
 This enhances functionality provided by ATLAS-3324.

*Background*:
 Requirement for import API is that if request contains "replicatedFrom" 
option, in this case import API should find & delete tables which are not 
imported but are part of database entity.
 E.g. Consider t1...t5 under db1 replicated from source to destination using 
table level incremental export from source.
 Now suppose subsequent import request at destination contains only t1...t3, in 
such case t4 & t5 must be deleted at destination.

*Solution approach*:
 * Check if import request contains "replicatedFrom" option.
 * Extract exportRequest from imported zip.
 * Identify table entities which are imported using exportRequest.
 * Identify database entity using imported table entities.
 * Identify table entities which are part of database entity obtained in 
previous step but are not imported in current import request.
 * Delete tables.

  was:
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
 This enhances functionality provided by ATLAS-3324.

*Background*:
 Requirement for import API is that if request contains "replicatedFrom" 
option, in this case import API should find & delete tables which are not 
imported but are part of database entity.
 E.g. Consider t1...t5 under db1 replicated from source to destination using 
table level incremental export from source.
 Now suppose subsequent import request at destination contains only t1...t3, in 
such case t4 & t5 must be deleted at destination.

*Solution approach*:
 * Check if import request contains "replicatedFrom" option.
 * Extract exportRequest from imported zip.
 * Identify table entities which are imported using exportRequest.
 * Identify database entity using imported table entities.
 * Identify table entities which are part of database entity got from previous 
step but are not imported in current import request.
 * Delete tables.


> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v11.patch, 
> Import-API-delete-non-exported-hive_table-for-table-v2-branch-0.8.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are present in the hive_db but are 
> not the part of the incremental export request.
>  This enhances functionality provided by ATLAS-3324.
> *Background*:
>  Requirement for import API is that if request contains "replicatedFrom" 
> option, in this case import API should find & delete tables which are not 
> imported but are part of database entity.
>  E.g. Consider t1...t5 under db1 replicated from source to destination using 
> table level incremental export from source.
>  Now suppose subsequent import request at destination contains only t1...t3, 
> in such case t4 & t5 must be deleted at destination.
> *Solution approach*:
>  * Check if import request contains "replicatedFrom" option.
>  * Extract exportRequest from imported zip.
>  * Identify table entities which are imported using exportRequest.
>  * Identify database entity using imported table entities.
>  * Identify table entities which are part of database entity obtained in 
> previous step but are not imported in current import request.
>  * Delete tables.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Description: 
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
 This enhances functionality provided by ATLAS-3324.

*Background*:
 Requirement for import API is that if request contains "replicatedFrom" 
option, in this case import API should find & delete tables which are not 
imported but are part of database entity.
 E.g. Consider t1...t5 under db1 replicated from source to destination using 
table level incremental export from source.
 Now suppose subsequent import request at destination contains only t1...t3, in 
such case t4 & t5 must be deleted at destination.

*Solution approach*:
 * Check if import request contains "replicatedFrom" option.
 * Extract exportRequest from imported zip.
 * Identify table entities which are imported using exportRequest.
 * Identify database entity using imported table entities.
 * Identify table entities which are part of database entity got from previous 
step but are not imported in current import request.
 * Delete tables.

  was:
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
This enhances functionality provided by 
[ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].

*Background*:
Requirement for import API is that if request contains "replicatedFrom" 
option, in this case import API should find & delete tables which are not 
imported but are part of database entity.
E.g. Consider t1...t5 under db1 replicated from source to destination 
using table level incremental export from source.
Now suppose subsequent import request at destination contains only 
t1...t3, in such case t4 & t5 must be deleted at destination.

*Solution approach*:
Check if import request contains "replicatedFrom" option.
Extract exportRequest from imported zip.
Identify table entities which are imported using exportRequest.
Identify database entity using imported table entities.
Identify table entities which are part of database entity got from 
previous step but are not imported in current import request.
Delete tables.


> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v11.patch, 
> Import-API-delete-non-exported-hive_table-for-table-v2-branch-0.8.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are present in the hive_db but are 
> not the part of the incremental export request.
>  This enhances functionality provided by ATLAS-3324.
> *Background*:
>  Requirement for import API is that if request contains "replicatedFrom" 
> option, in this case import API should find & delete tables which are not 
> imported but are part of database entity.
>  E.g. Consider t1...t5 under db1 replicated from source to destination using 
> table level incremental export from source.
>  Now suppose subsequent import request at destination contains only t1...t3, 
> in such case t4 & t5 must be deleted at destination.
> *Solution approach*:
>  * Check if import request contains "replicatedFrom" option.
>  * Extract exportRequest from imported zip.
>  * Identify table entities which are imported using exportRequest.
>  * Identify database entity using imported table entities.
>  * Identify table entities which are part of database entity got from 
> previous step but are not imported in current import request.
>  * Delete tables.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Description: 
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
This enhances functionality provided by 
[ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].

*Background*:
Requirement for import API is that if request contains "replicatedFrom" 
option, in this case import API should find & delete tables which are not 
imported but are part of database entity.
E.g. Consider t1...t5 under db1 replicated from source to destination 
using table level incremental export from source.
Now suppose subsequent import request at destination contains only 
t1...t3, in such case t4 & t5 must be deleted at destination.

*Solution approach*:
Check if import request contains "replicatedFrom" option.
Extract exportRequest from imported zip.
Identify table entities which are imported using exportRequest.
Identify database entity using imported table entities.
Identify table entities which are part of database entity got from 
previous step but are not imported in current import request.
Delete tables.

  was:
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
This enhances functionality provided by 
[ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].

*Background*:
Requirement for import API is that if request contains "replicatedFrom" 
option, in this case import API should find & delete tables which are not 
imported but are part of database entity.
E.g. Consider t1...t5 under db1 replicated from source to destination 
using table level incremental export from source.
Now suppose subsequent import request at destination contains only 
t1...t3, in such case t4 & t5 must be deleted at destination.

*Solution approach*:
Check if import request contains "replicatedFrom" option.
Extract exportRequest from imported zip.
Identify table entities which are imported using exportRequest.
Identify database entity using imported table entities.
Identify table entities which are part of database entity got from 
previous step but are not imported.
Delete tables.


> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v11.patch, 
> Import-API-delete-non-exported-hive_table-for-table-v2-branch-0.8.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are present in the hive_db but are 
> not the part of the incremental export request.
> This enhances functionality provided by 
> [ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].
> *Background*:
>   Requirement for import API is that if request contains "replicatedFrom" 
> option, in this case import API should find & delete tables which are not 
> imported but are part of database entity.
>   E.g. Consider t1...t5 under db1 replicated from source to destination 
> using table level incremental export from source.
>   Now suppose subsequent import request at destination contains only 
> t1...t3, in such case t4 & t5 must be deleted at destination.
> *Solution approach*:
>   Check if import request contains "replicatedFrom" option.
>   Extract exportRequest from imported zip.
>   Identify table entities which are imported using exportRequest.
>   Identify database entity using imported table entities.
>   Identify table entities which are part of database entity got from 
> previous step but are not imported in current import request.
>   Delete tables.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Description: 
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
This enhances functionality provided by 
[ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].

*Background*:
Requirement for import API is that if request contains "replicatedFrom" 
option, in this case import API should find & delete tables which are not 
imported but are part of database entity.
E.g. Consider t1...t5 under db1 replicated from source to destination 
using table level incremental export from source.
Now suppose subsequent import request at destination contains only 
t1...t3, in such case t4 & t5 must be deleted at destination.

*Solution approach*:
Check if import request contains "replicatedFrom" option.
Extract exportRequest from imported zip.
Identify table entities which are imported using exportRequest.
Identify database entity using imported table entities.
Identify table entities which are part of database entity got from 
previous step but are not imported.
Delete tables.

  was:
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
This enhances functionality provided by 
[ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].

Background:
Requirement for import API is that if request contains "replicatedFrom" 
option, in this case import API should find & delete tables which are not 
imported but are part of database entity.
E.g. Consider t1...t5 under db1 replicated from source to destination 
using table level incremental export from source.
Now suppose subsequent import request at destination contains only 
t1...t3, in such case t4 & t5 must be deleted at destination.

Solution approach:
Check if import request contains "replicatedFrom" option.
Extract exportRequest from imported zip.
Identify table entities which are imported using exportRequest.
Identify database entity using imported table entities.
Identify table entities which are part of database entity got from 
previous step but are not imported.
Delete tables.


> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v11.patch, 
> Import-API-delete-non-exported-hive_table-for-table-v2-branch-0.8.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are present in the hive_db but are 
> not the part of the incremental export request.
> This enhances functionality provided by 
> [ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].
> *Background*:
>   Requirement for import API is that if request contains "replicatedFrom" 
> option, in this case import API should find & delete tables which are not 
> imported but are part of database entity.
>   E.g. Consider t1...t5 under db1 replicated from source to destination 
> using table level incremental export from source.
>   Now suppose subsequent import request at destination contains only 
> t1...t3, in such case t4 & t5 must be deleted at destination.
> *Solution approach*:
>   Check if import request contains "replicatedFrom" option.
>   Extract exportRequest from imported zip.
>   Identify table entities which are imported using exportRequest.
>   Identify database entity using imported table entities.
>   Identify table entities which are part of database entity got from 
> previous step but are not imported.
>   Delete tables.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Description: 
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
This enhances functionality provided by 
[ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].

Background:
Requirement for import API is that if request contains "replicatedFrom" 
option, in this case import API should find & delete tables which are not 
imported but are part of database entity.
E.g. Consider t1...t5 under db1 replicated from source to destination 
using table level incremental export from source.
Now suppose subsequent import request at destination contains only 
t1...t3, in such case t4 & t5 must be deleted at destination.

Solution approach:
Check if import request contains "replicatedFrom" option.
Extract exportRequest from imported zip.
Identify table entities which are imported using exportRequest.
Identify database entity using imported table entities.
Identify table entities which are part of database entity got from 
previous step but are not imported.
Delete tables.

  was:
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
This enhances functionality provided by 
[ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].

Detailed Background:
Requirement for import API is that if request contains "replicatedFrom" 
option, in this case import API should find & delete tables which are not 
imported but are part of database entity.
E.g. Consider t1...t5 under db1 replicated from source to destination 
using table level incremental export from source.
Now suppose subsequent import request at destination contains only 
t1...t3, in such case t4 & t5 must be deleted at destination.

Solution approach:
Check if import request contains "replicatedFrom" option.
Extract exportRequest from imported zip.
Identify table entities which are imported using exportRequest.
Identify database entity using imported table entities.
Identify table entities which are part of database entity got from 
previous step but are not imported.
Delete tables.


> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v11.patch, 
> Import-API-delete-non-exported-hive_table-for-table-v2-branch-0.8.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are present in the hive_db but are 
> not the part of the incremental export request.
> This enhances functionality provided by 
> [ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].
> Background:
>   Requirement for import API is that if request contains "replicatedFrom" 
> option, in this case import API should find & delete tables which are not 
> imported but are part of database entity.
>   E.g. Consider t1...t5 under db1 replicated from source to destination 
> using table level incremental export from source.
>   Now suppose subsequent import request at destination contains only 
> t1...t3, in such case t4 & t5 must be deleted at destination.
> Solution approach:
>   Check if import request contains "replicatedFrom" option.
>   Extract exportRequest from imported zip.
>   Identify table entities which are imported using exportRequest.
>   Identify database entity using imported table entities.
>   Identify table entities which are part of database entity got from 
> previous step but are not imported.
>   Delete tables.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Description: 
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
This enhances functionality provided by 
[ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].

Detailed Background:
Requirement for import API is that if request contains "replicatedFrom" 
option, in this case import API should find & delete tables which are not 
imported but are part of database entity.
E.g. Consider t1...t5 under db1 replicated from source to destination 
using table level incremental export from source.
Now suppose subsequent import request at destination contains only 
t1...t3, in such case t4 & t5 must be deleted at destination.

Solution approach:
Check if import request contains "replicatedFrom" option.
Extract exportRequest from imported zip.
Identify table entities which are imported using exportRequest.
Identify database entity using imported table entities.
Identify table entities which are part of database entity got from 
previous step but are not imported.
Delete tables.

  was:
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
This enhances functionality provided by 
[ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].


> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v11.patch, 
> Import-API-delete-non-exported-hive_table-for-table-v2-branch-0.8.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are present in the hive_db but are 
> not the part of the incremental export request.
> This enhances functionality provided by 
> [ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].
> Detailed Background:
>   Requirement for import API is that if request contains "replicatedFrom" 
> option, in this case import API should find & delete tables which are not 
> imported but are part of database entity.
>   E.g. Consider t1...t5 under db1 replicated from source to destination 
> using table level incremental export from source.
>   Now suppose subsequent import request at destination contains only 
> t1...t3, in such case t4 & t5 must be deleted at destination.
> Solution approach:
>   Check if import request contains "replicatedFrom" option.
>   Extract exportRequest from imported zip.
>   Identify table entities which are imported using exportRequest.
>   Identify database entity using imported table entities.
>   Identify table entities which are part of database entity got from 
> previous step but are not imported.
>   Delete tables.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-20 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Attachment: 
Import-API-delete-non-exported-hive_table-for-table-v2-branch-0.8.patch

> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v11.patch, 
> Import-API-delete-non-exported-hive_table-for-table-v2-branch-0.8.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are present in the hive_db but are 
> not the part of the incremental export request.
> This enhances functionality provided by 
> [ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-20 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Attachment: (was: 
Import-API-delete-non-exported-hive_table-for-table-v11.patch)

> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v11.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are present in the hive_db but are 
> not the part of the incremental export request.
> This enhances functionality provided by 
> [ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-19 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Attachment: Import-API-delete-non-exported-hive_table-for-table-v11.patch

> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v11.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are present in the hive_db but are 
> not the part of the incremental export request.
> This enhances functionality provided by 
> [ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-19 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Attachment: (was: 
Import-API-delete-non-exported-hive_table-for-table-v10.patch)

> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v11.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are present in the hive_db but are 
> not the part of the incremental export request.
> This enhances functionality provided by 
> [ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-19 Thread Nixon Rodrigues (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nixon Rodrigues updated ATLAS-3416:
---
Fix Version/s: trunk

> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v10.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are present in the hive_db but are 
> not the part of the incremental export request.
> This enhances functionality provided by 
> [ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-18 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Description: 
This will delete hive_table entities which are present in the hive_db but are 
not the part of the incremental export request.
This enhances functionality provided by 
[ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].

  was:
This will delete hive_table entities which are not part of incremental export 
request.
This enhances functionality provided by 
[ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].


> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
> Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v10.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are present in the hive_db but are 
> not the part of the incremental export request.
> This enhances functionality provided by 
> [ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-18 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Attachment: repl_exp_1.zip
repl_exp_2.zip

> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
> Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v10.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are not part of incremental export 
> request.
> This enhances functionality provided by 
> [ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-18 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Attachment: Import-API-delete-non-exported-hive_table-for-table-v10.patch

> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
> Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v10.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are not part of incremental export 
> request.
> This enhances functionality provided by 
> [ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-18 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Description: 

This will delete hive_table entities which are not part of incremental export.
This enhances functionality provided by 
[ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].

  was:Backgroung :ATLAS-3324


> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
>
> This will delete hive_table entities which are not part of incremental export.
> This enhances functionality provided by 
> [ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-18 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Description: 
This will delete hive_table entities which are not part of incremental export 
request.
This enhances functionality provided by 
[ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].

  was:

This will delete hive_table entities which are not part of incremental export.
This enhances functionality provided by 
[ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].


> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
>
> This will delete hive_table entities which are not part of incremental export 
> request.
> This enhances functionality provided by 
> [ATLAS-3324|https://issues.apache.org/jira/browse/ATLAS-3324].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-18 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Description: Backgroung :ATLAS-3324  (was: ATLAS-3324)

> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
>
> Backgroung :ATLAS-3324



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-18 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Description: ATLAS-3324

> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
>
> ATLAS-3324



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3416) Import API: delete non-exported hive_table entities for table level replication

2019-09-18 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3416:

Summary: Import API: delete non-exported hive_table entities for table 
level replication  (was: Nikhil Bonte 11:08 AM Import API: delete non-exported 
hive_table entities for table level replication)

> Import API: delete non-exported hive_table entities for table level 
> replication
> ---
>
> Key: ATLAS-3416
> URL: https://issues.apache.org/jira/browse/ATLAS-3416
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)