[ 
https://issues.apache.org/jira/browse/ATLAS-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968769#comment-16968769
 ] 

ASF subversion and git services commented on ATLAS-3416:
--------------------------------------------------------

Commit 1c078edc1fa0a6befff232b6f75fc0b4ae330ce4 in atlas's branch 
refs/heads/branch-2.0 from nikhilbonte
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=1c078ed ]

ATLAS-3416 Import API: delete non-exported hive_table for table level 
replication

Signed-off-by: nixonrodrigues <[email protected]>


> Import API: delete non-exported hive_table entities for table level 
> replication
> -------------------------------------------------------------------------------
>
>                 Key: ATLAS-3416
>                 URL: https://issues.apache.org/jira/browse/ATLAS-3416
>             Project: Atlas
>          Issue Type: New Feature
>            Reporter: Nikhil Bonte
>            Assignee: Nikhil Bonte
>            Priority: Major
>             Fix For: trunk
>
>         Attachments: 
> Import-API-delete-non-exported-hive_table-for-table-v11.patch, 
> Import-API-delete-non-exported-hive_table-for-table-v2-branch-0.8.patch, 
> repl_exp_1.zip, repl_exp_2.zip
>
>
> This will delete hive_table entities which are present in the hive_db but are 
> not the part of the incremental export request.
>  This enhances functionality provided by ATLAS-3324.
> *Background*:
>  Requirement for import API is that if request contains "replicatedFrom" 
> option, in this case import API should find & delete tables which are not 
> imported but are part of database entity.
>  E.g. Consider t1...t5 under db1 replicated from source to destination using 
> table level incremental export from source.
>  Now suppose subsequent import request at destination contains only t1...t3, 
> in such case t4 & t5 must be deleted at destination.
> *Solution*:
>  * Check if import request contains "replicatedFrom" option.
>  * Extract exportRequest from imported zip.
>  * Identify table entities which are imported using exportRequest.
>  * Identify database entity using imported table entities.
>  * Identify table entities which are part of database entity obtained in 
> previous step but are not imported in current import request.
>  * Delete tables.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to