[jira] [Updated] (ATLAS-4591) [Hive import v2] While importing with hive import V2, seeing exceptions in the log for successful imports

[Dharshana M Krishnamoorthy (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dharshana M Krishnamoorthy updated ATLAS-4591:
--
Summary: [Hive import v2] While importing with hive import V2, seeing 
exceptions in the log for successful imports  (was: [Hive import v2] While 
importing with hive import V2, seeing exceptions in the log)

> [Hive import v2] While importing with hive import V2, seeing exceptions in 
> the log for successful imports
> -
>
> Key: ATLAS-4591
> URL: https://issues.apache.org/jira/browse/ATLAS-4591
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Reporter: Dharshana M Krishnamoorthy
>Priority: Major
>
> Seeing this error in import log
> {code:java}
> 2022-04-27 07:34:02,760 INFO  - [main:] ~ method=GET 
> path=api/atlas/v2/entity/uniqueAttribute/type/hive_db/header 
> contentType=application/json; charset=UTF-8 accept=application/json 
> status=404 (AtlasBaseClient:407)
> 2022-04-27 07:34:02,762 WARN  - [main:] ~ Failed to get DB guid from Atlas 
> with qualified name db_ecwvo@cm (HiveMetaStoreBridgeV2:740)
> org.apache.atlas.AtlasServiceException: Metadata service API 
> org.apache.atlas.AtlasBaseClient$API@69de5bed failed with status 404 (Not 
> Found) Response Body 
> ({"errorCode":"ATLAS-404-00-009","errorMessage":"Instance hive_db with unique 
> attribute {qualifiedName=db_ecwvo@cm} does not exist"})
>         at 
> org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:447)
>         at 
> org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:366)
>         at 
> org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:352)
>         at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:256)
>         at 
> org.apache.atlas.AtlasClientV2.getEntityHeaderByAttribute(AtlasClientV2.java:413)
>         at 
> org.apache.atlas.hive.bridge.HiveMetaStoreBridgeV2.getDBGuidFromAtlas(HiveMetaStoreBridgeV2.java:738)
>         at 
> org.apache.atlas.hive.bridge.HiveMetaStoreBridgeV2.getGuid(HiveMetaStoreBridgeV2.java:1019)
>         at 
> org.apache.atlas.hive.bridge.HiveMetaStoreBridgeV2.toDbEntity(HiveMetaStoreBridgeV2.java:718)
>         at 
> org.apache.atlas.hive.bridge.HiveMetaStoreBridgeV2.toDbEntity(HiveMetaStoreBridgeV2.java:705)
>         at 
> org.apache.atlas.hive.bridge.HiveMetaStoreBridgeV2.writeDatabase(HiveMetaStoreBridgeV2.java:645)
>         at 
> org.apache.atlas.hive.bridge.HiveMetaStoreBridgeV2.importHiveDatabases(HiveMetaStoreBridgeV2.java:314)
>         at 
> org.apache.atlas.hive.bridge.HiveMetaStoreBridgeV2.exportDataToZipAndRunAtlasImport(HiveMetaStoreBridgeV2.java:190)
>         at 
> org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:180)
> 2022-04-27 07:34:02,813 INFO  - [main:] ~ Importing Hive Tables 
> (HiveMetaStoreBridgeV2:405) {code}
> Steps followed:
>  # Disabled Hive hook
>  # Created a database
>  # Called the import v2 api by passing -o to import-hive.sh 
>  # The above step will create the zip and also import the data 
> Though the import is successful, seeing the above mentioned error in the logs.
> This is not seen when -o is not passed to the import script (Not seeing issue 
> in old import script)



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Created] (ATLAS-4598) [Hive import v2] deleteNonExisting is not honoured when used with v2 api

[Dharshana M Krishnamoorthy (Jira)]

Dharshana M Krishnamoorthy created ATLAS-4598:
-

 Summary: [Hive import v2] deleteNonExisting is not honoured when 
used with v2 api
 Key: ATLAS-4598
 URL: https://issues.apache.org/jira/browse/ATLAS-4598
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Reporter: Dharshana M Krishnamoorthy


Scenario:

Perform import using v2 api by using *deleteNonExisting*

Steps:
 # Create a db 
 # Create 2 tables in that db
 # Perform an import
 # Verify the tables are reflected in atlas
 # Drop the table
 # Perform an import with *deleteNonExisting*
 # command: *export JAVA_HOME=/usr/java/default; 
/opt/cloudera/parcels/CDH/lib/atlas/hook-bin/import-hive.sh -d db_hive_db_jwics 
-deleteNonExisting -o /tmp/db_wetbq.zip*

Problem:

Here deleteNonExisting is not honoured when used in the V2 api and the dropped 
table's status is still active in Atlas



--
This message was sent by Atlassian Jira
(v8.20.7#820007)