[
https://issues.apache.org/jira/browse/ATLAS-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16914003#comment-16914003
]
Kai Chen commented on ATLAS-3130:
---------------------------------
We came across similar issue when using sqoop hook to import tables from mysql
to hive. we got the exception below.
{code:java}
org.apache.atlas.exception.AtlasBaseException: invalid relationshipDef:
process_dataset_outputs: end type 1: DataSet, end type 2: sqoop_process
at
org.apache.atlas.repository.store.graph.v2.AtlasRelationshipStoreV2.validateRelationship(AtlasRelationshipStoreV2.java:653)
{code}
> Atlas stops reporting CTAS tables once an invalid relationshipDef is created
> -----------------------------------------------------------------------------
>
> Key: ATLAS-3130
> URL: https://issues.apache.org/jira/browse/ATLAS-3130
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
> Reporter: Umesh Padashetty
> Assignee: Merryle Wang
> Priority: Minor
>
> Created an invalid relationshipDef with the following body
> {code:java}
> {
> "relationshipDefs": [{
> "propagateTags": "ONE_TO_TWO",
> "description": "default relationshipDef description with name:
> sample_dcef_KcFcR3xIN",
> "relationshipCategory": "ASSOCIATION",
> "typeVersion": "1.0",
> "attributeDefs": [],
> "endDef2": {
> "name": "inputs",
> "isLegacyAttribute": false,
> "isContainer": false,
> "cardinality": "SINGLE",
> "type": "Process",
> "description": "default relationshipEndDef description with name: inputs"
> },
> "endDef1": {
> "name": "hive_table_to_process",
> "isLegacyAttribute": false,
> "isContainer": false,
> "cardinality": "SET",
> "type": "hive_table",
> "description": "default relationshipEndDef description with name:
> hive_table_to_process"
> },
> "guid": "-302256315524233986",
> "name": "sample_dcef_KcFcR3xIN"
> }]
> }{code}
> After this, all the CTAS queries start failing with the following error
> {code:java}
> 2019-04-09 17:47:38,992 ERROR - [NotificationHookConsumer thread-0:] ~ graph
> rollback due to exception (GraphTransactionInterceptor:154)
> org.apache.atlas.exception.AtlasBaseException: invalid relationshipDef:
> sample_dcef_KcFcR3xIN: end type 1: Process, end type 2:
> hive_column_lineage{code}
> The tables are created in the hive, but they are not reported in Atlas.
> Creation of such an invalid relationshipDef should ideally be blocked.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
