[GitHub] [atlas] vladhlinsky commented on issue #89: ATLAS-3646 Create new 'spark_ml_model_dataset', 'spark_ml_pipeline_dataset' relationship defs
vladhlinsky commented on issue #89: ATLAS-3646 Create new 'spark_ml_model_dataset','spark_ml_pipeline_dataset' relationship defs URL: https://github.com/apache/atlas/pull/89#issuecomment-599603296 @sarathsubramanian I guess in this case the proper way will be changing `"relationshipCategory"` value to `"ASSOCIATION"` and removing `isContainer` attributes. PR is updated with these changes. Retested manually using the steps above. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [atlas] vladhlinsky commented on issue #89: ATLAS-3646 Create new 'spark_ml_model_dataset', 'spark_ml_pipeline_dataset' relationship defs
vladhlinsky commented on issue #89: ATLAS-3646 Create new 'spark_ml_model_dataset','spark_ml_pipeline_dataset' relationship defs URL: https://github.com/apache/atlas/pull/89#issuecomment-599593065 @sarathsubramanian sorry, i just realized that i tested proposed by you changes with previously existing relationship definitions and thus I missed that changing "isContainer" => false leads to: ``` org.apache.atlas.exception.AtlasBaseException: AGGREGATION relationshipDef spark_ml_model_dataset creation attempted without an end specifying isContainer at org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(AtlasRelationshipType.java:309) at org.apache.atlas.type.AtlasRelationshipType.resolveReferences(AtlasRelationshipType.java:110) at org.apache.atlas.type.AtlasTypeRegistry$AtlasTransientTypeRegistry.resolveReferences(AtlasTypeRegistry.java:426) at org.apache.atlas.type.AtlasTypeRegistry$AtlasTransientTypeRegistry.addTypes(AtlasTypeRegistry.java:500) at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.tryTypeCreation(AtlasTypeDefGraphStore.java:1136) at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.createUpdateTypesDef(AtlasTypeDefGraphStore.java:410) at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$$FastClassBySpringCGLIB$$5226c80b.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ``` changes reverted back to "isContainer" => true. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [atlas] vladhlinsky commented on issue #89: ATLAS-3646 Create new 'spark_ml_model_dataset', 'spark_ml_pipeline_dataset' relationship defs
vladhlinsky commented on issue #89: ATLAS-3646 Create new 'spark_ml_model_dataset','spark_ml_pipeline_dataset' relationship defs URL: https://github.com/apache/atlas/pull/89#issuecomment-593355434 cc @HeartSaVioR @sarathsubramanian This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [atlas] vladhlinsky commented on issue #89: ATLAS-3646 Create new 'spark_ml_model_dataset', 'spark_ml_pipeline_dataset' relationship defs
vladhlinsky commented on issue #89: ATLAS-3646 Create new 'spark_ml_model_dataset','spark_ml_pipeline_dataset' relationship defs URL: https://github.com/apache/atlas/pull/89#issuecomment-593355095 Attaching screenshots. - Create Spark ML model and ML directory entities: ``` create_ml_directory mldir create_ml_model model_with_mldir directory spark_ml_directory mldir ``` - Verify that `model_with_mldir` is accessible via Atlas UI and has `mldir` as `directory`. ![Screenshot from 2020-02-29 13-51-35](https://user-images.githubusercontent.com/61428392/75672015-7d8e3200-5c88-11ea-9ea0-8a559702c80a.png) ![Screenshot from 2020-02-29 13-51-50](https://user-images.githubusercontent.com/61428392/75672021-8121b900-5c88-11ea-9c88-a0b7000283a7.png) - Create Spark ML model and `hdfs_path` entities: ``` create_hdfs_path path create_ml_model model_with_path dataset hdfs_path path ``` - Verify that `model_with_path` is accessible via Atlas UI and has `path` as `dataset`. ![Screenshot from 2020-02-29 13-52-45](https://user-images.githubusercontent.com/61428392/75672058-9860a680-5c88-11ea-96f4-dc82db6c2d56.png) ![Screenshot from 2020-02-29 13-53-16](https://user-images.githubusercontent.com/61428392/75672066-9b5b9700-5c88-11ea-9611-3f73ba3299bc.png) - Create Spark ML pipeline and ML directory entities: ``` create_ml_directory mldir2 create_ml_pipeline pipeline_with_mldir directory spark_ml_directory mldir2 ``` - Verify that `pipeline_with_mldir` is accessible via Atlas UI and has `mldir2` as `directory`. ![Screenshot from 2020-02-29 13-55-13](https://user-images.githubusercontent.com/61428392/75672084-a44c6880-5c88-11ea-9791-eb6203982574.png) ![Screenshot from 2020-02-29 13-55-27](https://user-images.githubusercontent.com/61428392/75672096-a7dfef80-5c88-11ea-9395-837e0b397a8c.png) - Create Spark ML pipeline and `hdfs_path` entities: ``` create_hdfs_path path2 create_ml_pipeline pipeline_with_path dataset hdfs_path path2 ``` - Verify that `pipeline_with_path` is accessible via Atlas UI and has `path2` as `dataset`. ![Screenshot from 2020-02-29 13-56-24](https://user-images.githubusercontent.com/61428392/75672122-b9c19280-5c88-11ea-8e23-74d5c0ea01d7.png) ![Screenshot from 2020-02-29 13-56-41](https://user-images.githubusercontent.com/61428392/75672127-bd551980-5c88-11ea-9330-4570d0972528.png) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services