> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/1111-ml_model.json
> > Lines 10 (patched)
> > <https://reviews.apache.org/r/71619/diff/1/?file=2169131#file2169131line10>
> >
> > Why make project a subtype of dataset?
Atlas server has a lot of code that build lineage based on Process and DataSet
types
1) Process only handle types derives from `DataSet`
{
"name": "Process",
"superTypes": [
"Asset"
],
"serviceType": "atlas_core",
"typeVersion": "1.1",
"attributeDefs": [
{
"name": "inputs",
"typeName": "array<DataSet>",
"cardinality": "SET",
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
{
"name": "outputs",
"typeName": "array<DataSet>",
"cardinality": "SET",
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
]
}
2) Lineage is only built between Process type and DataSet type.
https://github.com/apache/atlas/blob/master/repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java#L117
3) If we don't want to re-implement the lineage in Atlas, it is better to
deriver action type from Process type, and data type from DataSet.
> On Oct. 16, 2019, 4:28 p.m., Anand Patil wrote:
> > addons/models/1000-Hadoop/1111-ml_model.json
> > Lines 51 (patched)
> > <https://reviews.apache.org/r/71619/diff/1/?file=2169131#file2169131line51>
> >
> > businessUseCase, modelFramework, modelAlgorithms, githubRepoURL,
> > notebookURL and resourceURL feel prematurely opinionated to me. Some
> > projects will not use all these fields, and for some projects fields other
> > than these will probably be more important.
> >
> > Would it be possible in Atlas to move these to a single "metadata"
> > attribute whose value is key-value pairs? We can then move to stronger
> > typing as common patterns emerge.
replace those attributes by
{
"name": "metadata",
"description": "Contains key-value pairs that provide project
metadata",
"typeName": "map<string,string>",
"cardinality": "SINGLE",
"isIndexable": false,
"isOptional": true,
"isUnique": false
}
- Na
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71619/#review218231
-----------------------------------------------------------
On Oct. 16, 2019, 12:30 a.m., Na Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71619/
> -----------------------------------------------------------
>
> (Updated Oct. 16, 2019, 12:30 a.m.)
>
>
> Review request for atlas, Austin Nobis, Ashutosh Mestry, Karthik Manamcheri,
> Sridhar K, Madhan Neethiraj, and Sarath Subramanian.
>
>
> Bugs: atlas-3464
> https://issues.apache.org/jira/browse/atlas-3464
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Define entities used for Machine Learning Governance
>
>
> Diffs
> -----
>
> addons/models/1000-Hadoop/1111-ml_model.json PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/71619/diff/1/
>
>
> Testing
> -------
>
> verified it is valid json file
>
>
> Thanks,
>
> Na Li
>
>
