[
https://issues.apache.org/jira/browse/ATLAS-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nixon Rodrigues updated ATLAS-3551:
-----------------------------------
Description:
*Description*
Importing Hbase entities fails if an Hbase column family was previously deleted.
*Steps to reproduce:*
kinit as user hbase
create a table:
{noformat}
$ echo "create 'default:hbase_table_csaba2'
,{NAME=>'col_fam_csaba2_1'},{NAME=>'col_fam_csaba2_2'}" |
/usr/hdp/current/hbase-client/bin/hbase shell -n
Created table default:hbase_table_csaba2
Took 1.2510 seconds
Hbase::Table - hbase_table_csaba2
{noformat}
import the table:
{noformat}
$ /usr/hdp/current/atlas-server/hook-bin/import-hbase.sh -t hbase_table_csaba2
-Datlas.conf=/etc/hbase/conf
HBase Data Model imported successfully!!!
{noformat}
delete one of the column families:
{noformat}
$ echo "alter 'default:hbase_table_csaba2', NAME => 'col_fam_csaba2_1',
METHOD=>'delete'" | /usr/hdp/current/hbase-client/bin/hbase shell -n
Updating all regions with the new schema...
1/1 regions updated.
Done.
Took 2.5881 seconds
{noformat}
import the table again:
{noformat}
$ /usr/hdp/current/atlas-server/hook-bin/import-hbase.sh -t hbase_table_csaba2
-Datlas.conf=/etc/hbase/conf
ImportHBaseEntities failed. Please check the log file for the detailed error
message
Failed to import HBase Data Model!!!
Log file content: (/grid/0/log/import-hbase.log):
2019-10-03 12:41:28,391 INFO - [main:] ~ Table already present in Atlas.
Updating it..: default:hbase_table_csaba2@cl1 (HBaseBridge:403)
2019-10-03 12:41:28,416 INFO - [main:] ~ method=POST path=api/atlas/v2/entity/
contentType=application/json; charset=UTF-8 accept=application/json status=404
(AtlasBaseClient:387)
2019-10-03 12:41:28,418 ERROR - [main:] ~ ImportHBaseEntities failed
(HBaseBridge:196)
org.apache.atlas.AtlasServiceException: Metadata service API
org.apache.atlas.AtlasClientV2$API_V2@2a389173 failed with status 404 (Not
Found) Response Body (
{"errorCode":"ATLAS-404-00-007","errorMessage":"Invalid instance
creation/updation parameters passed : hbase_column_family.table: mandatory
attribute value missing in type hbase_column_family"}
)
at
org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:427)
at
org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:353)
at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:229)
at org.apache.atlas.AtlasClientV2.updateEntity(AtlasClientV2.java:324)
at
org.apache.atlas.hbase.bridge.HBaseBridge.updateEntityInAtlas(HBaseBridge.java:615)
at
org.apache.atlas.hbase.bridge.HBaseBridge.createOrUpdateTable(HBaseBridge.java:409)
at org.apache.atlas.hbase.bridge.HBaseBridge.importTable(HBaseBridge.java:282)
at
org.apache.atlas.hbase.bridge.HBaseBridge.importHBaseEntities(HBaseBridge.java:235)
at org.apache.atlas.hbase.bridge.HBaseBridge.main(HBaseBridge.java:186)
This causes the atlas split 2 failure: test_import_hbase_without_args
{noformat}
was:
Importing Hbase entities fails if an Hbase column family was previously deleted.
Steps to reproduce:
kinit as user hbase
create a table:
{{$ echo "create 'default:hbase_table_csaba2'
,\{NAME=>'col_fam_csaba2_1'},\{NAME=>'col_fam_csaba2_2'}" |
/usr/hdp/current/hbase-client/bin/hbase shell -n
Created table default:hbase_table_csaba2
Took 1.2510 seconds
Hbase::Table - hbase_table_csaba2}}
import the table:
{{$ /usr/hdp/current/atlas-server/hook-bin/import-hbase.sh -t
hbase_table_csaba2 -Datlas.conf=/etc/hbase/conf
HBase Data Model imported successfully!!!}}
delete one of the column families:
{{$ echo "alter 'default:hbase_table_csaba2', NAME => 'col_fam_csaba2_1',
METHOD=>'delete'" | /usr/hdp/current/hbase-client/bin/hbase shell -n
Updating all regions with the new schema...
1/1 regions updated.
Done.
Took 2.5881 seconds}}
import the table again:
{{$ /usr/hdp/current/atlas-server/hook-bin/import-hbase.sh -t
hbase_table_csaba2 -Datlas.conf=/etc/hbase/conf
ImportHBaseEntities failed. Please check the log file for the detailed error
message
Failed to import HBase Data Model!!!}}
Log file content: (/grid/0/log/import-hbase.log):
{{2019-10-03 12:41:28,391 INFO - [main:] ~ Table already present in Atlas.
Updating it..: default:hbase_table_csaba2@cl1 (HBaseBridge:403)
2019-10-03 12:41:28,416 INFO - [main:] ~ method=POST path=api/atlas/v2/entity/
contentType=application/json; charset=UTF-8 accept=application/json status=404
(AtlasBaseClient:387)
2019-10-03 12:41:28,418 ERROR - [main:] ~ ImportHBaseEntities failed
(HBaseBridge:196)
org.apache.atlas.AtlasServiceException: Metadata service API
org.apache.atlas.AtlasClientV2$API_V2@2a389173 failed with status 404 (Not
Found) Response Body (\{"errorCode":"ATLAS-404-00-007","errorMessage":"Invalid
instance creation/updation parameters passed : hbase_column_family.table:
mandatory attribute value missing in type hbase_column_family"})
at
org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:427)
at
org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:353)
at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:229)
at org.apache.atlas.AtlasClientV2.updateEntity(AtlasClientV2.java:324)
at
org.apache.atlas.hbase.bridge.HBaseBridge.updateEntityInAtlas(HBaseBridge.java:615)
at
org.apache.atlas.hbase.bridge.HBaseBridge.createOrUpdateTable(HBaseBridge.java:409)
at
org.apache.atlas.hbase.bridge.HBaseBridge.importTable(HBaseBridge.java:282)
at
org.apache.atlas.hbase.bridge.HBaseBridge.importHBaseEntities(HBaseBridge.java:235)
at
org.apache.atlas.hbase.bridge.HBaseBridge.main(HBaseBridge.java:186)}}
# This causes the atlas split 2 failure: test_import_hbase_without_args
> Atlas unable to import entities if an HBase column family is deleted
> --------------------------------------------------------------------
>
> Key: ATLAS-3551
> URL: https://issues.apache.org/jira/browse/ATLAS-3551
> Project: Atlas
> Issue Type: Bug
> Affects Versions: 2.0.0
> Reporter: Nixon Rodrigues
> Priority: Minor
>
> *Description*
> Importing Hbase entities fails if an Hbase column family was previously
> deleted.
> *Steps to reproduce:*
> kinit as user hbase
> create a table:
>
> {noformat}
> $ echo "create 'default:hbase_table_csaba2'
> ,{NAME=>'col_fam_csaba2_1'},{NAME=>'col_fam_csaba2_2'}" |
> /usr/hdp/current/hbase-client/bin/hbase shell -n
> Created table default:hbase_table_csaba2
> Took 1.2510 seconds
> Hbase::Table - hbase_table_csaba2
> {noformat}
> import the table:
> {noformat}
> $ /usr/hdp/current/atlas-server/hook-bin/import-hbase.sh -t
> hbase_table_csaba2 -Datlas.conf=/etc/hbase/conf
> HBase Data Model imported successfully!!!
>
> {noformat}
> delete one of the column families:
> {noformat}
> $ echo "alter 'default:hbase_table_csaba2', NAME => 'col_fam_csaba2_1',
> METHOD=>'delete'" | /usr/hdp/current/hbase-client/bin/hbase shell -n
> Updating all regions with the new schema...
> 1/1 regions updated.
> Done.
> Took 2.5881 seconds
>
> {noformat}
> import the table again:
> {noformat}
> $ /usr/hdp/current/atlas-server/hook-bin/import-hbase.sh -t
> hbase_table_csaba2 -Datlas.conf=/etc/hbase/conf
> ImportHBaseEntities failed. Please check the log file for the detailed error
> message
> Failed to import HBase Data Model!!!
> Log file content: (/grid/0/log/import-hbase.log):
> 2019-10-03 12:41:28,391 INFO - [main:] ~ Table already present in Atlas.
> Updating it..: default:hbase_table_csaba2@cl1 (HBaseBridge:403)
> 2019-10-03 12:41:28,416 INFO - [main:] ~ method=POST
> path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8
> accept=application/json status=404 (AtlasBaseClient:387)
> 2019-10-03 12:41:28,418 ERROR - [main:] ~ ImportHBaseEntities failed
> (HBaseBridge:196)
> org.apache.atlas.AtlasServiceException: Metadata service API
> org.apache.atlas.AtlasClientV2$API_V2@2a389173 failed with status 404 (Not
> Found) Response Body (
> {"errorCode":"ATLAS-404-00-007","errorMessage":"Invalid instance
> creation/updation parameters passed : hbase_column_family.table: mandatory
> attribute value missing in type hbase_column_family"}
> )
> at
> org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:427)
> at
> org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:353)
> at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:229)
> at org.apache.atlas.AtlasClientV2.updateEntity(AtlasClientV2.java:324)
> at
> org.apache.atlas.hbase.bridge.HBaseBridge.updateEntityInAtlas(HBaseBridge.java:615)
> at
> org.apache.atlas.hbase.bridge.HBaseBridge.createOrUpdateTable(HBaseBridge.java:409)
> at
> org.apache.atlas.hbase.bridge.HBaseBridge.importTable(HBaseBridge.java:282)
> at
> org.apache.atlas.hbase.bridge.HBaseBridge.importHBaseEntities(HBaseBridge.java:235)
> at org.apache.atlas.hbase.bridge.HBaseBridge.main(HBaseBridge.java:186)
> This causes the atlas split 2 failure: test_import_hbase_without_args
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
