[ https://issues.apache.org/jira/browse/ATLAS-3640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046551#comment-17046551 ]
Vladislav Glinskiy commented on ATLAS-3640: ------------------------------------------- cc [~sarath] > Update 'spark_ml_model_ml_directory' and 'spark_ml_pipeline_ml_directory' > relationship definitions > -------------------------------------------------------------------------------------------------- > > Key: ATLAS-3640 > URL: https://issues.apache.org/jira/browse/ATLAS-3640 > Project: Atlas > Issue Type: Task > Reporter: Vladislav Glinskiy > Priority: Major > Fix For: 2.1.0, 3.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Update 'spark_ml_model_ml_directory' and 'spark_ml_pipeline_ml_directory' > relationship definitions to use 'DataSet' type instead of it's child type > 'spark_ml_directory'. This is required in order to integrate Spark Atlas > Connector's ML event processor. > Previously, Spark Atlas Connector used the 'spark_ml_directory' model for ML > model directory but this is changed in the scope of > [https://github.com/hortonworks-spark/spark-atlas-connector/issues/61], > [https://github.com/hortonworks-spark/spark-atlas-connector/pull/62] so ML > model directory is 'DataSet' entity(i.e. 'hdfs_path'). > Thus, relationship definitions must be updated, otherwise, an attempt to > create relation leads to: > {code:java} > org.apache.atlas.exception.AtlasBaseException: invalid relationshipDef: > spark_ml_model_ml_directory: end type 1: spark_ml_directory, end type 2: > spark_ml_model > {code} > since 'COMPOSITION' requires 'spark_ml_directory' to be set. -- This message was sent by Atlassian Jira (v8.3.4#803005)