[jira] [Created] (ATLAS-1891) Introduce prechecks for relationshipDef update
David Radley created ATLAS-1891: --- Summary: Introduce prechecks for relationshipDef update Key: ATLAS-1891 URL: https://issues.apache.org/jira/browse/ATLAS-1891 Project: Atlas Issue Type: Sub-task Reporter: David Radley Assignee: David Radley -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1887) Coverity Scan defects in AtlasRelationshipType
[
https://issues.apache.org/jira/browse/ATLAS-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Radley updated ATLAS-1887:
Attachment: rb60213.patch
Amended the logic in AtlasRelationshipDef so that it does not address
relationship end content and then later check whether the relationship end was
null. I introduced a junit as well.
> Coverity Scan defects in AtlasRelationshipType
> --
>
> Key: ATLAS-1887
> URL: https://issues.apache.org/jira/browse/ATLAS-1887
> Project: Atlas
> Issue Type: Bug
>Reporter: David Radley
>Assignee: David Radley
> Attachments: rb60213.patch
>
>
> 3 new defect(s) introduced to Apache Atlas found with Coverity Scan.
> 3 defect(s), reported by Coverity Scan earlier, were marked fixed in the
> recent build analyzed by Coverity Scan.
>
> New defect(s) Reported-by: Coverity Scan
> Showing 3 of 3 defect(s)
>
>
> ** CID 164610:(REVERSE_INULL)
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 140
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 147
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
>
>
>
>
> *** CID 164610:(REVERSE_INULL)
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 140
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> 134 // composition containers should not be multiple
> cardinality
> 135 if (endDef1 != null &&
> 136 endDef1.getCardinality() ==
> AtlasAttributeDef.Cardinality.SET &&
> 137 endDef1.getIsContainer()) {
> 138 throw new
> AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
> name);
> 139 }
> >>> CID 164610:(REVERSE_INULL)
> >>> Null-checking "endDef2" suggests that it may be null, but it has
> already been dereferenced on all paths leading to the check.
> 140 if (endDef2 != null && endDef2 != null &&
> 141 endDef2.getCardinality() ==
> AtlasAttributeDef.Cardinality.SET &&
> 142 endDef2.getIsContainer()) {
> 143 throw new
> AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
> name);
> 144 }
> 145 }
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 147
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> 141 endDef2.getCardinality() ==
> AtlasAttributeDef.Cardinality.SET &&
> 142 endDef2.getIsContainer()) {
> 143 throw new
> AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
> name);
> 144 }
> 145 }
> 146 if ((endDef1 != null && endDef1.getCardinality() ==
> AtlasAttributeDef.Cardinality.LIST) ||
> >>> CID 164610:(REVERSE_INULL)
> >>> Null-checking "endDef2" suggests that it may be null, but it has
> already been dereferenced on all paths leading to the check.
> 147 (endDef2 != null && endDef2.getCardinality() ==
> AtlasAttributeDef.Cardinality.LIST)) {
> 148 throw new
> AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_LIST_ON_END, name);
> 149 }
> 150 }
>
> ** CID 164609:(REVERSE_INULL)
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 135
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 146
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
>
>
>
>
> *** CID 164609:(REVERSE_INULL)
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 135
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelat
[jira] [Commented] (ATLAS-1887) Coverity Scan defects in AtlasRelationshipType
[
https://issues.apache.org/jira/browse/ATLAS-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054897#comment-16054897
]
David Radley commented on ATLAS-1887:
-
https://reviews.apache.org/r/60213
> Coverity Scan defects in AtlasRelationshipType
> --
>
> Key: ATLAS-1887
> URL: https://issues.apache.org/jira/browse/ATLAS-1887
> Project: Atlas
> Issue Type: Bug
>Reporter: David Radley
>Assignee: David Radley
>
> 3 new defect(s) introduced to Apache Atlas found with Coverity Scan.
> 3 defect(s), reported by Coverity Scan earlier, were marked fixed in the
> recent build analyzed by Coverity Scan.
>
> New defect(s) Reported-by: Coverity Scan
> Showing 3 of 3 defect(s)
>
>
> ** CID 164610:(REVERSE_INULL)
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 140
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 147
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
>
>
>
>
> *** CID 164610:(REVERSE_INULL)
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 140
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> 134 // composition containers should not be multiple
> cardinality
> 135 if (endDef1 != null &&
> 136 endDef1.getCardinality() ==
> AtlasAttributeDef.Cardinality.SET &&
> 137 endDef1.getIsContainer()) {
> 138 throw new
> AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
> name);
> 139 }
> >>> CID 164610:(REVERSE_INULL)
> >>> Null-checking "endDef2" suggests that it may be null, but it has
> already been dereferenced on all paths leading to the check.
> 140 if (endDef2 != null && endDef2 != null &&
> 141 endDef2.getCardinality() ==
> AtlasAttributeDef.Cardinality.SET &&
> 142 endDef2.getIsContainer()) {
> 143 throw new
> AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
> name);
> 144 }
> 145 }
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 147
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> 141 endDef2.getCardinality() ==
> AtlasAttributeDef.Cardinality.SET &&
> 142 endDef2.getIsContainer()) {
> 143 throw new
> AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
> name);
> 144 }
> 145 }
> 146 if ((endDef1 != null && endDef1.getCardinality() ==
> AtlasAttributeDef.Cardinality.LIST) ||
> >>> CID 164610:(REVERSE_INULL)
> >>> Null-checking "endDef2" suggests that it may be null, but it has
> already been dereferenced on all paths leading to the check.
> 147 (endDef2 != null && endDef2.getCardinality() ==
> AtlasAttributeDef.Cardinality.LIST)) {
> 148 throw new
> AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_LIST_ON_END, name);
> 149 }
> 150 }
>
> ** CID 164609:(REVERSE_INULL)
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 135
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 146
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
>
>
>
>
> *** CID 164609:(REVERSE_INULL)
> /intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 135
> in
> org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
> 129 // AGGREGATION needs one end to be the container.
> 130 throw new
> AtlasBaseException(AtlasError
[jira] [Created] (ATLAS-1887) Coverity Scan defects in AtlasRelationshipType
David Radley created ATLAS-1887:
---
Summary: Coverity Scan defects in AtlasRelationshipType
Key: ATLAS-1887
URL: https://issues.apache.org/jira/browse/ATLAS-1887
Project: Atlas
Issue Type: Bug
Reporter: David Radley
Assignee: David Radley
3 new defect(s) introduced to Apache Atlas found with Coverity Scan.
3 defect(s), reported by Coverity Scan earlier, were marked fixed in the
recent build analyzed by Coverity Scan.
New defect(s) Reported-by: Coverity Scan
Showing 3 of 3 defect(s)
** CID 164610:(REVERSE_INULL)
/intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 140
in
org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
/intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 147
in
org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
*** CID 164610:(REVERSE_INULL)
/intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 140
in
org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
134 // composition containers should not be multiple
cardinality
135 if (endDef1 != null &&
136 endDef1.getCardinality() ==
AtlasAttributeDef.Cardinality.SET &&
137 endDef1.getIsContainer()) {
138 throw new
AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
name);
139 }
>>> CID 164610:(REVERSE_INULL)
>>> Null-checking "endDef2" suggests that it may be null, but it has
already been dereferenced on all paths leading to the check.
140 if (endDef2 != null && endDef2 != null &&
141 endDef2.getCardinality() ==
AtlasAttributeDef.Cardinality.SET &&
142 endDef2.getIsContainer()) {
143 throw new
AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
name);
144 }
145 }
/intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 147
in
org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
141 endDef2.getCardinality() ==
AtlasAttributeDef.Cardinality.SET &&
142 endDef2.getIsContainer()) {
143 throw new
AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_COMPOSITION_SET_CONTAINER,
name);
144 }
145 }
146 if ((endDef1 != null && endDef1.getCardinality() ==
AtlasAttributeDef.Cardinality.LIST) ||
>>> CID 164610:(REVERSE_INULL)
>>> Null-checking "endDef2" suggests that it may be null, but it has
already been dereferenced on all paths leading to the check.
147 (endDef2 != null && endDef2.getCardinality() ==
AtlasAttributeDef.Cardinality.LIST)) {
148 throw new
AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_LIST_ON_END, name);
149 }
150 }
** CID 164609:(REVERSE_INULL)
/intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 135
in
org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
/intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 146
in
org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
*** CID 164609:(REVERSE_INULL)
/intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java: 135
in
org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(org.apache.atlas.model.typedef.AtlasRelationshipDef)()
129 // AGGREGATION needs one end to be the container.
130 throw new
AtlasBaseException(AtlasErrorCode.RELATIONSHIPDEF_AGGREGATION_NO_CONTAINER,
name);
131 }
132 }
133 if (relationshipCategory ==
RelationshipCategory.COMPOSITION) {
134 // composition containers should not be multiple
cardinality
>>> CID 164609:(REVERSE_INULL)
>>> Null-checking "endDef1" suggests that it may be null, but it has
already been dereferenced on all
[jira] [Commented] (ATLAS-1873) Rename AtlasRelationshipEndPoint to Rename AtlasRelationshipEnd
[ https://issues.apache.org/jira/browse/ATLAS-1873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049421#comment-16049421 ] David Radley commented on ATLAS-1873: - https://reviews.apache.org/r/60059/ > Rename AtlasRelationshipEndPoint to Rename AtlasRelationshipEnd > --- > > Key: ATLAS-1873 > URL: https://issues.apache.org/jira/browse/ATLAS-1873 > Project: Atlas > Issue Type: Sub-task >Reporter: David Radley >Assignee: David Radley > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ATLAS-1873) Rename AtlasRelationshipEndPoint to Rename AtlasRelationshipEnd
[ https://issues.apache.org/jira/browse/ATLAS-1873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley resolved ATLAS-1873. - Resolution: Fixed > Rename AtlasRelationshipEndPoint to Rename AtlasRelationshipEnd > --- > > Key: ATLAS-1873 > URL: https://issues.apache.org/jira/browse/ATLAS-1873 > Project: Atlas > Issue Type: Sub-task >Reporter: David Radley >Assignee: David Radley > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1873) Rename AtlasRelationshipEndPoint to Rename AtlasRelationshipEnd
David Radley created ATLAS-1873: --- Summary: Rename AtlasRelationshipEndPoint to Rename AtlasRelationshipEnd Key: ATLAS-1873 URL: https://issues.apache.org/jira/browse/ATLAS-1873 Project: Atlas Issue Type: Sub-task Reporter: David Radley Assignee: David Radley -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1854) Complete relationshipDef implementation
[ https://issues.apache.org/jira/browse/ATLAS-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1854: Description: Ensure: update works and also delete then create works (was: Ensure get by guid works and ensure update works. ) > Complete relationshipDef implementation > --- > > Key: ATLAS-1854 > URL: https://issues.apache.org/jira/browse/ATLAS-1854 > Project: Atlas > Issue Type: Sub-task > Components: atlas-core >Reporter: David Radley >Assignee: David Radley > > Ensure: update works and also delete then create works -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1867) org.apache.atlas.AtlasClientV2.deleteAtlasTypeDefs gives an exception
David Radley created ATLAS-1867: --- Summary: org.apache.atlas.AtlasClientV2.deleteAtlasTypeDefs gives an exception Key: ATLAS-1867 URL: https://issues.apache.org/jira/browse/ATLAS-1867 Project: Atlas Issue Type: Bug Reporter: David Radley The org.apache.atlas.AtlasClientV2 is the Java API for Atlas. As such the helper methods in it are the API. calling deleteAtlasTypeDefs gives the exception Exception in thread "main" com.sun.jersey.api.client.ClientHandlerException: java.net.ProtocolException: HTTP method DELETE doesn't support output at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155) at com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:105) at com.sun.jersey.api.client.Client.handle(Client.java:652) at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:634) at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:298) at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:285) at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:427) at org.apache.atlas.AtlasClientV2.deleteAtlasTypeDefs(AtlasClientV2.java:258) at org.apache.atlas.examples.DeleteTypesFromJsonFileUtil.runTypeDelete(DeleteTypesFromJsonFileUtil.java:73) at org.apache.atlas.examples.DeleteTypesFromJsonFileUtil.main(DeleteTypesFromJsonFileUtil.java:58) Caused by: java.net.ProtocolException: HTTP method DELETE doesn't support output at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1083) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler$1$1.getOutputStream(URLConnectionClientHandler.java:238) at com.sun.jersey.api.client.CommittingOutputStream.commitStream(CommittingOutputStream.java:117) at com.sun.jersey.api.client.CommittingOutputStream.write(CommittingOutputStream.java:89) at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291) at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295) at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141) at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229) at java.io.BufferedWriter.flush(BufferedWriter.java:254) at com.sun.jersey.core.util.ReaderWriter.writeToAsString(ReaderWriter.java:191) at com.sun.jersey.core.provider.AbstractMessageReaderWriterProvider.writeToAsString(AbstractMessageReaderWriterProvider.java:128) at com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:88) at com.sun.jersey.core.impl.provider.entity.StringProvider.writeTo(StringProvider.java:58) at com.sun.jersey.api.client.RequestWriter.writeRequestEntity(RequestWriter.java:300) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:217) at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1859) Add validation checking around relationshipDefs
David Radley created ATLAS-1859: --- Summary: Add validation checking around relationshipDefs Key: ATLAS-1859 URL: https://issues.apache.org/jira/browse/ATLAS-1859 Project: Atlas Issue Type: Sub-task Reporter: David Radley Assignee: Sarath Subramanian When a relationshipDef creation request is attempted, Atlas needs to check: - does the endpoint name in the relationship already exist in another active relationshipdef that applies to that type - does the endpoint name in the relationship already exist in another attributedef that applies to that type The above needs to account for any attributeDefs and relationshipdefs that apply due to type inheritance. There is also the case that for supplied models like Hive, there may need to be some mechanism to allow the old forms of composition relationships - that Gremlin queries might relay on. The old forms of relationship instances are implemented by 2 edges - the new form is one edge. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1690: Attachment: Atlas Relationships proposal v1.10.pdf Added a sections on RelationshipDef edges and validation. > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas_RelationDef_Json_Structure_v1.pdf, Atlas > Relationships proposal v1.0.pdf, Atlas Relationships proposal v1.10.pdf, > Atlas Relationships proposal v1.1.pdf, Atlas Relationships proposal v1.2.pdf, > Atlas Relationships proposal v1.3.pdf, Atlas Relationships proposal v1.4.pdf, > Atlas Relationships proposal v1.5.pdf, Atlas Relationships proposal v1.6.pdf, > Atlas Relationships proposal v1.7.pdf, Atlas Relationships proposal v1.8.pdf, > Atlas Relationships proposal v1.9.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (ATLAS-1854) Complete relationshipDef implementation
[ https://issues.apache.org/jira/browse/ATLAS-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley reassigned ATLAS-1854: --- Assignee: David Radley > Complete relationshipDef implementation > --- > > Key: ATLAS-1854 > URL: https://issues.apache.org/jira/browse/ATLAS-1854 > Project: Atlas > Issue Type: Sub-task > Components: atlas-core >Reporter: David Radley >Assignee: David Radley > > Ensure get by guid works and ensure update works. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1854) Complete relationshipDef implementation
David Radley created ATLAS-1854: --- Summary: Complete relationshipDef implementation Key: ATLAS-1854 URL: https://issues.apache.org/jira/browse/ATLAS-1854 Project: Atlas Issue Type: Sub-task Components: atlas-core Reporter: David Radley Ensure get by guid works and ensure update works. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1853) Change relationshipDef to be implmented as an edge in the type graph
David Radley created ATLAS-1853: --- Summary: Change relationshipDef to be implmented as an edge in the type graph Key: ATLAS-1853 URL: https://issues.apache.org/jira/browse/ATLAS-1853 Project: Atlas Issue Type: Sub-task Reporter: David Radley Assignee: David Radley -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1852) Create RelationshipDef
David Radley created ATLAS-1852: --- Summary: Create RelationshipDef Key: ATLAS-1852 URL: https://issues.apache.org/jira/browse/ATLAS-1852 Project: Atlas Issue Type: Sub-task Components: atlas-core Reporter: David Radley Assignee: David Radley Introduce relationshipDef: 1) Creation 2) Get by name 3) Return relationshipdefs as part of a get typeDefs call. At this stage the relationshiDef is defined as a Vertex in the graph. getbyguid is not working update has not been tested implementation needs to be changed to an edge. Needs to be similar to existing composition relationships in the types graph. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1768) Create common types for Open Metadata
[ https://issues.apache.org/jira/browse/ATLAS-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16032727#comment-16032727 ] David Radley commented on ATLAS-1768: - [~mandy_chessell] [~Stefhan] How about defining the category parent as composition - as the indicator that we do not allow deletion of a parent category with children. |in order to delete a parent category - the children need to be moved away from under it. I see this as encouraging explicit management of the children to enable the delete and the composition as the way we see categories and term. If this is too restrictive for our use cases then I am fine with the aggregation. all the best, David > Create common types for Open Metadata > - > > Key: ATLAS-1768 > URL: https://issues.apache.org/jira/browse/ATLAS-1768 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: 0.9-incubating >Reporter: Mandy Chessell >Assignee: Mandy Chessell > Labels: OpenMetadata > > This JIRA describes a proposal for standard types for open metadata entities > and relationships. For example, glossaries, database definitions, rules, > policies, ... > The value of having standard definitions for metadata is to enable type safe > APIs and business level UIs plus be able to exchange metadata between > different instances of metadata repositories. > The implementation of these common types is divided into 8 areas: > * Area 0 - for extensions to Apache Atlas's base model > * Area 1 - for definitions of the data-related assets we are governing and > using > * Area 2 - for a glossary of meanings and semantic relationships > * Area 3 - for information about asset use, crowd-sourced definitions and > collaboration around the data-related assets > * Area 4 - for governance such as policies, rules and classifications > * Area 5 - for reference models and reference data > * Area 6 - for metadata discovery processes (see > https://issues.apache.org/jira/browse/ATLAS-1748) > * Area 7 - for lineage > Adaptation and flexibility are key in metadata environments so these common > definitions must be extensible - and we still need to support the ad hoc > definition of new types in Atlas. > Apache Atlas supports meta-types that are used in the definition of new > types. These are currently enumeration, struct, classification and entity. > JIRA https://issues.apache.org/jira/browse/ATLAS-1690 adds relationships to > this list. The open metadata models make use of all of these meta-types. > These are represented by sterotypes on the classes of the open metadata > definitions. > The Atlas wiki has the models as a set of linked pages which are probably the > easiest way to view the models. > Start here: > https://cwiki.apache.org/confluence/display/ATLAS/Building+out+the+Apache+Atlas+Typesystem -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1821) Classification propagation from entity to a derivative or child entity
[ https://issues.apache.org/jira/browse/ATLAS-1821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026274#comment-16026274 ] David Radley commented on ATLAS-1821: - [~jonesn] The example was one [~Stefhan] voiced in a recent call on this subject. This was the example of a case where he needed the Glossary term security level not to flow to the child asset. > Classification propagation from entity to a derivative or child entity > -- > > Key: ATLAS-1821 > URL: https://issues.apache.org/jira/browse/ATLAS-1821 > Project: Atlas > Issue Type: Improvement > Components: atlas-core, atlas-webui >Reporter: Srikanth Venkat > Fix For: 0.9-incubating > > > User Story: > As a data steward, I need a scalable way to quickly and efficiently propagate > classification across the information supply chain to support efficient > searches and classification based security for compliance and audit purposes. > This requires: > 1. Classifications for derivative entities should be inherited from the > originator and to child entities from parent. > For example, if a Hive column is classified "Confidential" then resulting > column created from a CTAS operation should also be tagged "Confidential" to > maintain the classification of the original entity. In the case where 2 or > more entities are composed, the derivative entity should have the union of > all classifications of each source entity. > 2. Business Terms: > a. Child business terms should inherit the classifications associated with > the parent term. > b. The option to propagate classification to child business terms in a > hierarchy should be provided > c. Ability to update the propagated tags manually via UI or through the API > d. Tagging a term should propagate to data assets that are already attached > to that business term as well > 3. Data assets > a. For all supported data asset types in Atlas, if a derivative asset is > created it should inherit the tags and attributes from the original asset. > b. the option to propagate tags to child entities should be provided (e.g. if > you tag a folder in HDFS optionally tag all the files within it) > c. Ability to update the propagated tags manually via UI or through the API > d. Tagging a parent object should be inherited after child creation > dynamically (unless a flag is set not to do this) > e. Derived data assets should have the tags of the original data asset. > Conflict resolution - if there are different values for attributes on tags > (classifications) on upstream or parent entities used to derive a data asset > then user needs to be prompted for action to resolve the conflict. Once > resolved, the resolved value should be carried forth to derived assets. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1768) Create common types for Open Metadata
[ https://issues.apache.org/jira/browse/ATLAS-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026139#comment-16026139 ] David Radley commented on ATLAS-1768: - [~mandy_chessell] Thanks for your quick responses. I am left with 2 thoughts : I am wondering why a sub category life cycle is not tied to the parents. What is the thinking here? I thought it would be a true tree; so we would stop the deletion of a parent if it had children or have explicit delete subtree API. I understand that the child categories could be moved to top level categories if the parent is deleted; this seems odd, as categories half way down a hierarchy may not make a lot of sense as top level categories. If this end state is desired - I suggest the child category should be explicitly moved to the top prior and this move not occur as part of the parent category deletion. Also for soft deletes if the children categories are deleted prior to the parent - this leaves the graph reflecting the before state more accurately as the parent pointer would still point to the old parent. I am not seeing where Classifications have cardinality in the current Atlas model; I only see cardinality around attributeDefs (and soon to be relationshipDefs). I think adding cardinality to ClassificationDef would be useful - the Atlas code could then police this constraint. > Create common types for Open Metadata > - > > Key: ATLAS-1768 > URL: https://issues.apache.org/jira/browse/ATLAS-1768 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: 0.9-incubating >Reporter: Mandy Chessell >Assignee: Mandy Chessell > Labels: VirtualDataConnector > > This JIRA describes a proposal for standard types for open metadata entities > and relationships. For example, glossaries, database definitions, rules, > policies, ... > The value of having standard definitions for metadata is to enable type safe > APIs and business level UIs plus be able to exchange metadata between > different instances of metadata repositories. > The implementation of these common types is divided into 8 areas: > * Area 0 - for extensions to Apache Atlas's base model > * Area 1 - for definitions of the data-related assets we are governing and > using > * Area 2 - for a glossary of meanings and semantic relationships > * Area 3 - for information about asset use, crowd-sourced definitions and > collaboration around the data-related assets > * Area 4 - for governance such as policies, rules and classifications > * Area 5 - for reference models and reference data > * Area 6 - for metadata discovery processes (see > https://issues.apache.org/jira/browse/ATLAS-1748) > * Area 7 - for lineage > Adaptation and flexibility are key in metadata environments so these common > definitions must be extensible - and we still need to support the ad hoc > definition of new types in Atlas. > Apache Atlas supports meta-types that are used in the definition of new > types. These are currently enumeration, struct, classification and entity. > JIRA https://issues.apache.org/jira/browse/ATLAS-1690 adds relationships to > this list. The open metadata models make use of all of these meta-types. > These are represented by sterotypes on the classes of the open metadata > definitions. > The Atlas wiki has the models as a set of linked pages which are probably the > easiest way to view the models. > Start here: > https://cwiki.apache.org/confluence/display/ATLAS/Building+out+the+Apache+Atlas+Typesystem -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1410) V2 Glossary API
[
https://issues.apache.org/jira/browse/ATLAS-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025052#comment-16025052
]
David Radley commented on ATLAS-1410:
-
[~mandy_chessell] Looks really good. Some thoughts:
- 210 I wonder if language should be a code table value - or more generally an
valid value from reference data
-210 I am wondering about usage. Should this also be a code table - it seems
more structural than the description
-220 I suggest the supercategory to the subcategory be a composition (filled in
diamond) relationship.
- 230 I think the GlossaryCategory role name should be categories rather than
category
- 240 I wonder about the "to" and "from" ends of the related term as they imply
a direction - for a SYNONYM and TRANSLATION there is no direction. It is almost
like synonyms and transactions should be in a synonym group or translation
group respectively. Maybe we introduce an equivalence group concept, where
everything in the group is related to everything else in the group. This would
help for tag propagation for these terms.
I don't think we have a way in the current Atlas model to constrain the number
of classifications to 0..1.
> V2 Glossary API
> ---
>
> Key: ATLAS-1410
> URL: https://issues.apache.org/jira/browse/ATLAS-1410
> Project: Atlas
> Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: Atlas Glossary V2 proposal v1.0.pdf, Atlas Glossary V2
> proposal v1.1.pdf, Atlas Glossary V2 proposal v1.2.pdf, Atlas Glossary V2
> proposal v1.3.pdf, Atlas Glossary V2 proposal v1.4.pdf
>
>
> The BaseResourceDefinition uses the AttributeDefintion class from typesystem.
> There are newer more funcitonal versions of this capability in the atlas-intg
> project. This Jira is changing over the glossary implementation to the newer
> entity / type classes.
> Instread of the instanceProperties and collectionProperties in the
> BaseResourceDefintions we should use something in this sort of style :
> "
> AtlasEntityDef deptTypeDef =
> AtlasTypeUtil.createClassTypeDef(DEPARTMENT_TYPE,
> "Department"+_description, ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> new AtlasAttributeDef("employees",
> String.format("array<%s>", "Person"), true,
> AtlasAttributeDef.Cardinality.SINGLE, 0, 1,
> false, false,
>
> Collections.emptyList()));
> AtlasEntityDef personTypeDef =
> AtlasTypeUtil.createClassTypeDef("Person", "Person"+_description,
> ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> AtlasTypeUtil.createOptionalAttrDef("address", "Address"),
> AtlasTypeUtil.createOptionalAttrDef("birthday", "date"),
> AtlasTypeUtil.createOptionalAttrDef("hasPets", "boolean"),
> AtlasTypeUtil.createOptionalAttrDef("numberOfCars", "byte"),
> AtlasTypeUtil.createOptionalAttrDef("houseNumber", "short"),
> AtlasTypeUtil.createOptionalAttrDef("carMileage", "int"),
> AtlasTypeUtil.createOptionalAttrDef("age", "float"),
> "
> For the parent child relationships with glossary categories and terms we
> should be able to have the type system manage edge deletion. As part of this,
> we will need to investigate whether we could get rid of the disconnect and
> connect methods added in ATLAS-1186
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (ATLAS-1821) Classification propagation from entity to a derivative or child entity
[ https://issues.apache.org/jira/browse/ATLAS-1821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024460#comment-16024460 ] David Radley commented on ATLAS-1821: - I think having the classifications propagate is powerful. I am concerned that we need to restrict which classifications propagate across which relationships. If we want to pursue a declarative approach to this, I would like to understand the process by which conflicts are resolved and by who and how we know that these conflict resolutions meet with the companies governance policies. We have a use case where the glossary is set up with a Glossary Term National Insurance number, which is tagged as confidential. It is mapped to a masked column and an unmasked column. The masked column can be public, but the unmasked column need to get the Terms security classification. This would not be a conflict. I suggest a rules based approach be used instead. In this case a governance team could define a set of rules around how classifications flow , including special cases. Maybe something like : for all Glossary terms that have assigned assets, flow the terms confidentiality level classification to the assigned asset, except in the case where if the assigned asset is masked - then classify it as public We are then in a position to author rules that encapsulate best governance practices and play a part to enforce governance standards. > Classification propagation from entity to a derivative or child entity > -- > > Key: ATLAS-1821 > URL: https://issues.apache.org/jira/browse/ATLAS-1821 > Project: Atlas > Issue Type: Improvement > Components: atlas-core, atlas-webui >Reporter: Srikanth Venkat > Fix For: 0.9-incubating > > > User Story: > As a data steward, I need a scalable way to quickly and efficiently propagate > classification across the information supply chain to support efficient > searches and classification based security for compliance and audit purposes. > This requires: > 1. Classifications for derivative entities should be inherited from the > originator and to child entities from parent. > For example, if a Hive column is classified "Confidential" then resulting > column created from a CTAS operation should also be tagged "Confidential" to > maintain the classification of the original entity. In the case where 2 or > more entities are composed, the derivative entity should have the union of > all classifications of each source entity. > 2. Business Terms: > a. Child business terms should inherit the classifications associated with > the parent term. > b. The option to propagate classification to child business terms in a > hierarchy should be provided > c. Ability to update the propagated tags manually via UI or through the API > d. Tagging a term should propagate to data assets that are already attached > to that business term as well > 3. Data assets > a. For all supported data asset types in Atlas, if a derivative asset is > created it should inherit the tags and attributes from the original asset. > b. the option to propagate tags to child entities should be provided (e.g. if > you tag a folder in HDFS optionally tag all the files within it) > c. Ability to update the propagated tags manually via UI or through the API > d. Tagging a parent object should be inherited after child creation > dynamically (unless a flag is set not to do this) > e. Derived data assets should have the tags of the original data asset. > Conflict resolution - if there are different values for attributes on tags > (classifications) on upstream or parent entities used to derive a data asset > then user needs to be prompted for action to resolve the conflict. Once > resolved, the resolved value should be carried forth to derived assets. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16019214#comment-16019214 ] David Radley commented on ATLAS-1690: - Hey [~sarath.ku...@gmail.com], I tried to push up my changes to the new branch - but it fails with an authentication error, I reset my password and still get the same error. It looks like I do not have authority to push to this branch, many thanks , David. > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas_RelationDef_Json_Structure_v1.pdf, Atlas > Relationships proposal v1.0.pdf, Atlas Relationships proposal v1.1.pdf, Atlas > Relationships proposal v1.2.pdf, Atlas Relationships proposal v1.3.pdf, Atlas > Relationships proposal v1.4.pdf, Atlas Relationships proposal v1.5.pdf, Atlas > Relationships proposal v1.6.pdf, Atlas Relationships proposal v1.7.pdf, Atlas > Relationships proposal v1.8.pdf, Atlas Relationships proposal v1.9.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1690: Attachment: Atlas Relationships proposal v1.9.pdf We had a meeting yesterday with various companies participating to talk about the implementation / design of Jira on top level relationships. We reached consensus that we wanted to include composition relationships as well as association and aggregation relationships. We also talked about tag propagation, suggesting it would be useful to have an enumeration defined in the RelationshipDef indicating that which way tags should flow. Thinking about this I wonder if this is too blunt an instrument. For example if I define a has-a relationship in the RelationshipDef - then we do not want to always propagate all tags in all circumstances . - we may sometimes want to propagate some tags but not others. I have updated the design document to make it more coherent and added a proposal to extend the endpoint in the relationshipDef to include a tags-to-propagate list of ClassificationDef names, these would be the classifications that would be propagated from the entity of that type across the relationship at runtime. > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas_RelationDef_Json_Structure_v1.pdf, Atlas > Relationships proposal v1.0.pdf, Atlas Relationships proposal v1.1.pdf, Atlas > Relationships proposal v1.2.pdf, Atlas Relationships proposal v1.3.pdf, Atlas > Relationships proposal v1.4.pdf, Atlas Relationships proposal v1.5.pdf, Atlas > Relationships proposal v1.6.pdf, Atlas Relationships proposal v1.7.pdf, Atlas > Relationships proposal v1.8.pdf, Atlas Relationships proposal v1.9.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1690: Attachment: Atlas Relationships proposal v1.8.pdf Updated to add tag propagation in response to recent Jira discussions and added lower level design. > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas_RelationDef_Json_Structure_v1.pdf, Atlas > Relationships proposal v1.0.pdf, Atlas Relationships proposal v1.1.pdf, Atlas > Relationships proposal v1.2.pdf, Atlas Relationships proposal v1.3.pdf, Atlas > Relationships proposal v1.4.pdf, Atlas Relationships proposal v1.5.pdf, Atlas > Relationships proposal v1.6.pdf, Atlas Relationships proposal v1.7.pdf, Atlas > Relationships proposal v1.8.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1782) Document deprecations
[ https://issues.apache.org/jira/browse/ATLAS-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010256#comment-16010256 ] David Radley commented on ATLAS-1782: - [~Jan Hentschel] I am thinking the most important deprecations are the APIs. Is there an agreed deprecation period for Atlas. I wonder whether the community agrees with documented a deprecation policy; for example an API labeled as deprecated may be removed in the next release. > Document deprecations > - > > Key: ATLAS-1782 > URL: https://issues.apache.org/jira/browse/ATLAS-1782 > Project: Atlas > Issue Type: Improvement >Reporter: Jan Hentschel >Priority: Minor > > Currently none of the deprecated classes document how its deprecation should > be handled, e.g. *AtlasClient* does not mention in which version the object > was deprecated, in which version the object will be removed and how it can be > exchanged. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16007924#comment-16007924 ] David Radley commented on ATLAS-1690: - Hi [~madhan.neethiraj] and [~cmgrote], This is interesting, it seems to me that if we are going to embrace using tags in context specific ways and we feel tag propagation decisions should not be made by the relationship author then we do not want to hard bake tag propagation into the Atlas store. I am hearing that you feel there are use cases where the relationship author or updater would be responsible for tag propagation. Can I just check how you are thinking about the tag propagation implementation in your scenarios : 1) when a table/column is classified as PII, any lineage from this table/view/column should also be automatically be classified as PII. This means we need a relationship between an entity defined attribute to another entity defined attribute in the other entity. The current proposal does not allow this as the relationship specifies attribute names for each end, that are not defined in the entity. The current relationship proposal adds a new attribute of type 'the other end'. I think we would need a new tag propagating relationship - or more generally a mapping. 2) when a term is classified as PII, all entities that are associated with the term also should automatically be classified as PII. So I am thinking that an asset / entity will have an assigned terms attribute. The GlossaryTerm would have a assignedEntities attribute. These attributes would be added by virtue of the relationship. If the Glossary Term was tagged PII, we need special Atlas logic so assignedEntities propagates to the entities assignedTerms - then special logic for the assignedTerms tag is picked up by the entity itself. 3) when a term is classified as PII, all terms that are synonyms of this term (and all the entities associated with the synonym terms) also should automatically be classified the same. We would need special Atlas logic for the synonym case that propagates the PII tag from the synonym relationship to the terms themselves. In summary I see 2 main tag propagation scenarios, and propose a way forward for each: - mapping between existing entity attributes, that could be used as the basis of propagation. I suggest we introduce the top level concept of a mapping separately from this relationships Jira. - the need for additional logic in Atlas to propagate tags across glossary term to term or term to asset relationships. I can see this is useful. So I suggest that I add in tag_propagation_hint enum including NONE. The actual propagation will depend on specific relationship types. When we implement the glossary, we will use this hint to propagate the tags in glossary specific Atlas logic. Further enhancements can occur as we bring in logic around the classification level use case and enhanced Ranger integration; Ranger will need to be able to override the hinted classification. Outside of the glossary case - tags will not be propagated - even if the tag-propagation-hint is set on a relationship. Give this thinking, it makes sense for the tag_propagation_hint to be on only on the Glossary Relationship types and not on the top level relationships. Madhan and [~mandy_chessell] does this make sense? > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas_RelationDef_Json_Structure_v1.pdf, Atlas > Relationships proposal v1.0.pdf, Atlas Relationships proposal v1.1.pdf, Atlas > Relationships proposal v1.2.pdf, Atlas Relationships proposal v1.3.pdf, Atlas > Relationships proposal v1.4.pdf, Atlas Relationships proposal v1.5.pdf, Atlas > Relationships proposal v1.6.pdf, Atlas Relationships proposal v1.7.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002261#comment-16002261 ] David Radley commented on ATLAS-1690: - Hi [~madhan.neethiraj] Great feedback - thanks for your thoughtful and open response. I will change the array to and endpoint1 and 2. I think that is clearer. I am keen that we propagate tags, this is very powerful. I thought I would explain how we could do classifications and then see how this option fits in. The classifications our customers are working with include confidentiality. The confidentiality scheme might have C1,C2, C3 and C4 levels. C1 might be public and C3 top secret. Different companies name these Cn levels differently. But in all these cases there is an order, C4 being the highest classification level. Though it is possible to have more complex classifications schemes- many / most cases can work with this list sort of classification schema. A particular glossary term or column might be associated with one of these classifications. We were thinking that the Classification levels (C1, C2 etc) would be new system Types (I suggest an EntityDef) A classification use is the relationship between the classification level and the thing it is classifying. By default the classification will be that from the level , but a rule can run and increase the classification; this would be calculated (and stored?) in the relationship instance. So to address your proposal: - I think the propagated classifications would be derived at query time and could be useful -do we need an effective classification? I am not convinced with the proposed mechanism. - Your example around tables and columns and PII assumes that PII is a binary flag (or one tag), I am suggesting that this is not the way that classifications are normally implemented - these should be an ordered list of levels. I see in some of your recent demos you use v1 terms to implement these classification levels for this classification ordering. If a table is classified as public and has a PII column , we would not want the public classification to override the PII column. As a query brings together 2 public fields, line name and salary, the combination becomes PII, in this case we need the rule to drive this. This implementation would encourage the use of bidirectional relationships to be implemented purely to propagate tags. I suggest many propagations would not be on one relationship, but could flow much further would be to all has-a terms - following all the has-a links. I am also concerned that the role who authors the relationship is not the right role to make classification propagation decisions. I wonder whether a smarter approach would be to tag the relationship as "propagate-1-to-2" (hopefully something more meaningful like propogate-table-to-column") and Ranger picks up this hint. Ranger could decide to run a simple rule of propagating all the tags from 1 to 2 or a more complex rule taking other conditions into account. I suggest that we explicitly implement these classification levels and uses, I hope there is a simple case where there are some classifications that should be propagated for all consumer cases, and rules can run to override the classifications and we can find a way of doing this using a governance role and we could make this work. Maybe some supplied Ranger rules and tags that Atlas used out of the box. GDPR rules and tags would be a good use case here. > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas_RelationDef_Json_Structure_v1.pdf, Atlas > Relationships proposal v1.0.pdf, Atlas Relationships proposal v1.1.pdf, Atlas > Relationships proposal v1.2.pdf, Atlas Relationships proposal v1.3.pdf, Atlas > Relationships proposal v1.4.pdf, Atlas Relationships proposal v1.5.pdf, Atlas > Relationships proposal v1.6.pdf, Atlas Relationships proposal v1.7.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1765) Self-Service Catalog Search and Data Preview
[ https://issues.apache.org/jira/browse/ATLAS-1765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16000646#comment-16000646 ] David Radley commented on ATLAS-1765: - Hi [~eostic] Nice use cases. I wonder if (for each of the use cases it would be worth adding "so that..."). My guess at the first one is: "List out all assets (usually columns) that have not yet been assigned a Term so that all assets are exposed as glossary terms, allowing assets to be managed/found/governed using a more natural language rather than having to use their unintuitive technical names " This would help me understand the motivations behind the search. > Self-Service Catalog Search and Data Preview > > > Key: ATLAS-1765 > URL: https://issues.apache.org/jira/browse/ATLAS-1765 > Project: Atlas > Issue Type: New Feature > Components: atlas-webui >Affects Versions: 0.9-incubating >Reporter: Mandy Chessell >Assignee: Mandy Chessell > Labels: Self-Service-UIs, VirtualDataConnector > > This JIRA covers the development of the catalog search and preview of data > for data scientists and business users. It supports the search of the Atlas > metadata repository, display of search results, additional filtering and > drill down into details of the data sources, including a data preview option > if the end user has access permission. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16000487#comment-16000487 ] David Radley commented on ATLAS-1690: - Hi [~suma.shivaprasad], As always these are great questions. 3. I will put together a Jira design doc to detail this out. It seems to me that we need the relationships piece then glossary then the classification piece. As the real power will be when we can tag a business term and have that propagate through. I think that the propagation piece sits more naturally in a second access layer around the base entity services. I would like to concentrate on the relationship implementation, then glossary, then classification. Do you feel it would be more useful for the community if I were to produce a design doc for the classification rather than prioritize the relationship implementation 4. If top level relationship is used to model all relationships - I would agree with you. This proposal is for the top level relationships to be just for the bidirectional and aggregation cases; so there is minimal migration impact for existing users of composition relationships. Are you proposing we model all relationships as a top level relationship object? My suggestion is to keep the scope down for this initial check in. Then we could more composition into top level relationships as a follow on piece - if we need to. I think keeping composition relationships in the entities seems pretty natural. > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas_RelationDef_Json_Structure_v1.pdf, Atlas > Relationships proposal v1.0.pdf, Atlas Relationships proposal v1.1.pdf, Atlas > Relationships proposal v1.2.pdf, Atlas Relationships proposal v1.3.pdf, Atlas > Relationships proposal v1.4.pdf, Atlas Relationships proposal v1.5.pdf, Atlas > Relationships proposal v1.6.pdf, Atlas Relationships proposal v1.7.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997971#comment-15997971 ] David Radley commented on ATLAS-1690: - Thank you very much for the feedback [~sarath.ku...@gmail.com] and [~cmgrote] . I am very much looking forward to your responses. To respond to the Sarath's points : 1. I do not think we should have "to" and "from". This implies a direction. For bidirection associations we do not want to imply there is any direction. Aggregations have a containership relationship - I think the important piece here is the container side of things not the direction. 2. I am wondering about the use case for a relationship to have more than one type. We can allow more than one type here if there is a compelling use case. 3. Interesting idea. You are right - tag propagation is important. As these are not bidirectional relationships, there is not an implied direction-it is not obvious which way the propagation should flow. I could see us wanting to do this sort of thing for composition (which are not modelled by a top level relationship). I suspect it is less likely that for 2 independent entities we would want to propagate tags using flags in the types. Ideally propagation of tags should be defined in Ranger rules - as we are likely to want different tag propagation in different contexts. Can I suggest we deal with classification in a subsequent design? We can then cover Semantic classification which will actually be a relationship. 4. Yes this is the same as Suma's suggestion of a category; I was thinking of "bidirectional”, “aggregation” - as association could be directional and contains could be composition . I will put this in, as we can then put this value into the edge - so we can identify which edges we need to expand into attributes. Are you OK with these amendments? I am thinking we still need an isContains flag to indicate the container end of a relationship. On your examples 1. I think a collection to member relationship would be an aggregation. 2. and 3. are compositions. There is a case to model composition in the top level relationship as well. I suggest we consider that later to reduce the migration impact of this change. 4 I have not looked at lineage. Responding to Chis's comments I can see that tag propagation direction is important; I am not sure the type itself should have these indicators. In the wider VDC architecture proposed in the Atlas wiki, the OMRS would not be responsible for tag propagation. A second layer -the OMAS layer exposes data in a particular shape for consumption in a context, eg around assets or the glossary or catalog search. Each of these OMAS layers are likely to have different tag propagation requirements. I am therefore reticent to have tag propagation at this lower layer. By not propagating at the OMRS layer classifications are defined once and are not duplicated. As above, I suggest we enhance the classifications design after the glossary. > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas_RelationDef_Json_Structure_v1.pdf, Atlas > Relationships proposal v1.0.pdf, Atlas Relationships proposal v1.1.pdf, Atlas > Relationships proposal v1.2.pdf, Atlas Relationships proposal v1.3.pdf, Atlas > Relationships proposal v1.4.pdf, Atlas Relationships proposal v1.5.pdf, Atlas > Relationships proposal v1.6.pdf, Atlas Relationships proposal v1.7.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1690: Attachment: Atlas Relationships proposal v1.7.pdf corrected terminology around cardinality > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas Relationships proposal v1.0.pdf, Atlas > Relationships proposal v1.1.pdf, Atlas Relationships proposal v1.2.pdf, Atlas > Relationships proposal v1.3.pdf, Atlas Relationships proposal v1.4.pdf, Atlas > Relationships proposal v1.5.pdf, Atlas Relationships proposal v1.6.pdf, Atlas > Relationships proposal v1.7.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1690: Attachment: Atlas Relationships proposal v1.6.pdf I feel there is now some consensus on this design. On this basis I am going to code based on this specification. > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas Relationships proposal v1.0.pdf, Atlas > Relationships proposal v1.1.pdf, Atlas Relationships proposal v1.2.pdf, Atlas > Relationships proposal v1.3.pdf, Atlas Relationships proposal v1.4.pdf, Atlas > Relationships proposal v1.5.pdf, Atlas Relationships proposal v1.6.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1764) Design and implement Atlas Collections
[ https://issues.apache.org/jira/browse/ATLAS-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994407#comment-15994407 ] David Radley commented on ATLAS-1764: - Hi [~sarath.ku...@gmail.com], This is a great Jira. Some up front thoughts: - I assume a collection could have a type . For example, we could have a collection of Hive tables. - I am not sure what constraints and attributes you are envisaging are useful for a collection. Maybe whether the collection is ordered might be a useful constraint or maximum size. - I am not sure what a classification on a collection means. It would be good to have a use case for this. - Is a collection going to be defined using a CollectionDef or type in EntityDef . If suggest it be an EntityDef type if that is possible. - Can we have an attribute pointing to a collection? - I wonder if we can have collections in collections and allow set based operations like unions. > Design and implement Atlas Collections > -- > > Key: ATLAS-1764 > URL: https://issues.apache.org/jira/browse/ATLAS-1764 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: 0.9-incubating >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian > Labels: features > > Design and implement Atlas Collections - A first class element in Atlas to > group related entities together and perform operations on these collections - > associate tags, add/remove entities... > Operations: > --- > a. Create collection(s) with attributes, constraints > b. Update existing collection(s) > c. Delete collection(s) - soft delete, hard delete > d. Retrieve collections by id, name > e. Add entity(s) to collection(s) > f. Remove entity(s) from collection(s) > g. Associate classification(s) to collection(s) > h. Disassociate classification(s) from collection(s) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993032#comment-15993032 ] David Radley commented on ATLAS-1690: - Thank you very much for your insightful reviews [~mandy_chessell] and [~suma.shivaprasad]. Response to Mandy's review Pg2 "Metadata repositories store metadata. The context of a metadata object is dictated by its relationships." I am not sure these sentences tell the complete story. Maybe something like "The Apache Atlas metadata repository stores metadata objects and their relationships. The relationships between the metadata objects are as important as the metadata objects themselves. They explain how the data landscape is structured and how the components within it relate to the business and the governance requirements, ownership and other interested parties. The relationships in Apache Atlas today provide support for containment (or part-of) relationships. This is necessary to describe sub-components of a component - for example, a Hive Column is a sub-component of a Hive Table. With these types of relationships, the lifetime of the sub-components is tied to their parent component. So for example, if a hive table is deleted, then all of its columns should also be deleted. This design is looking to add support for a new type of relationship between metadata objects that have independent lifetimes. In fact the creation of these relationships are actually an auditable action that can impact how data is discovered, understood, secured, managed and removed. Such relationships include when Glossary ..." <> Pg2 "If these links are made incorrectly (purposely or otherwise) data can be inappropriately exposed." This comment is out of place - it is only true if the relationship is involved in access control. A more general comment could be "If these links are made incorrectly (purposely or otherwise) data may be inappropriately used or governed." <> Font of JSON example on page 4 is inconsistent - harder to read than necessary. <> pg5 - "Relationship constraints" - first time mentioned this term - should be introduced in examples above.<> pg5 - "This name will help us name an association and its associated classification." Not sure what classification means in this sentence. Also need a description of why an association needs a name (I am thinking of this as a Type name - is that right? The name is important because the creation of these types of relationships are a deliberate act of governance and we need to be able to describe their use - and govern their lifecycle. <> pg 5 "“Address” and “Person”; a person has addresses, and addresses have people living in them. In this case, there is no obvious direction, so a bidirectional relationship is natural way of associating these concepts; the alternative would be 2 directional relationships that would not be kept in sync." Please use a metadata description - this is confusing to talk about data relationships. <> pg 5 "There are 2 main styles of relationships, tight and loose relationships." Why have new names been for these when at the top the doc states it is using UML names? Also the names are misleading. There is nothing loose about the association between a glossary term and a database column. <> pg 6 "In the case of tight relationships, the top entity and its children are governed as one, as the lifecycles of the children are tied to the parent. " It is true that the lifecycles are linked but it does not mean the governance is tied - for example, the confidentiality classification of a table may be different from the different columns it is made up of. Governance rules may be defined to act on specific columns and not on a table as a whole. <> pg 6/7 - RelationshipDef example - please use metadata examples not data examples - it is confusing because you would never define types for customer and account in Atlas.<> pg7 "The entity instances use Atlas object ids pointing to the relationship instance (which has a guid)." This needs further explanation and an example. pg 8 "Read" - what are the parameters on read - is this a single relationship operation?<> pg 8 "Aggregation implies that here is containment " I know what you mean but aggregation and containment are different things in UML and so this statement is not logically correct.<> p8 "A natural way to specify aggregation would be to have an isContainer Boolean flag, defaulting to false and specified on one of the endpoints in the relationship." Should say this flag can only be set on one end. <> pg8 - aggregations example - please use metadata example such as category to term <> pg8 - observations - a relationship described by a relationshipDef can not be mandatory. The isOptional flag is obsolete. Can we remove it?<> Response to Suma's review. 1. What does re
[jira] [Updated] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1690: Attachment: Atlas Relationships proposal v1.5.pdf Updated to respond to Suma and Mandys feedback > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas Relationships proposal v1.0.pdf, Atlas > Relationships proposal v1.1.pdf, Atlas Relationships proposal v1.2.pdf, Atlas > Relationships proposal v1.3.pdf, Atlas Relationships proposal v1.4.pdf, Atlas > Relationships proposal v1.5.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1757) Proposal to update graph DB
[ https://issues.apache.org/jira/browse/ATLAS-1757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988523#comment-15988523 ] David Radley commented on ATLAS-1757: - That makes sense Graham. Have you looked at whether the dependent products like Cassandra and HBase are different versions and may require different config between Titan 0, Titan 1 and Janusgraph. I wonder what the migration impact really is for an Atlas system; I would hope that Atlas hides much of the code level differences. I wonder how these migrations effect Cassanda configs underpinning an Atlas in production and how the TP level changes effect existing Atlas apps. > Proposal to update graph DB > --- > > Key: ATLAS-1757 > URL: https://issues.apache.org/jira/browse/ATLAS-1757 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: trunk >Reporter: Graham Wallis > > Given the formation of the JanusGraph open source project (under the Linux > Foundation) to continue the development and support of the Titan DB, should > we aim to deprecate Titan and move over to JanusGraph? > If we did this, we could keep the graph abstraction layer and use it to > support Titan 0, Titan 1 and JanusGraph. > Are there other graph databases that we should consider? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1755) UI - Add checkbox to exclude deleted entities in basic and fulltext search
[ https://issues.apache.org/jira/browse/ATLAS-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15988472#comment-15988472 ] David Radley commented on ATLAS-1755: - Please could you add the link the the review board patch - so it can easily be found. I suggest that the checkbox be the other way round. As discussed on the API Jira for this. the default behavior should be to get active entries. I suggest we have a UI checkbox called "Include historical entities", which by default is not set. I think this would be a more intuitive UI experience. > UI - Add checkbox to exclude deleted entities in basic and fulltext search > -- > > Key: ATLAS-1755 > URL: https://issues.apache.org/jira/browse/ATLAS-1755 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Reporter: Kalyani Kashikar >Assignee: Kalyani Kashikar > Attachments: ATLAS-1755.1.patch, ATLAS-1755.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1743) Issue with uniqueAttribute while creation of entities
[ https://issues.apache.org/jira/browse/ATLAS-1743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15978735#comment-15978735 ] David Radley commented on ATLAS-1743: - [~ayubkhan] that is interesting. That seems to contradict the w3 standard https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html 9.5 seems to indicate post is only for create in line with wikipedia. > Issue with uniqueAttribute while creation of entities > - > > Key: ATLAS-1743 > URL: https://issues.apache.org/jira/browse/ATLAS-1743 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.8-incubating, 0.9-incubating >Reporter: Sharmadha Sainath > > Atlas allows creation of entities with same value for unique attributes. > For example , > 1. Created an hdfs_path entity with > * name = dir1 > * qualifiedName = "userdir@cl1" > * path = /user/user1/dir > Entity is created successfully. > 2. Created another hdfs_path entity with > * name = dir2 > * qualifiedName = "userdir@cl1" > * path = /user/user2/dir > Qualified Name is uniqueAttribute and entity creation with same Qualified > Name is allowed. > Problem is when the 2nd hdfs_path entity is created , 1st entity is updated > with 2nd entity's values , hence 2nd entity creation is merely an update call > here instead of a create call because of same unique attribute. > There is no condition to check uniqueness of attributes while creation of > entities which is causing the problem. > If 1st hdfs_path entity is DELETED , then 2nd hdfs_path entity POST call > creates new entity. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1743) Issue with uniqueAttribute while creation of entities
[ https://issues.apache.org/jira/browse/ATLAS-1743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15978541#comment-15978541 ] David Radley commented on ATLAS-1743: - Hi there, [~ayubkhan]I agree this is how Atlas is designed. Though it does not seem quite right to me. See my review comment on https://reviews.apache.org/r/58333/. It seems that post should not be used for update. I suspect [~ssainath] expected the rest post to be a create only; as per normal Rest conventions. The confusion is that Atlas is exposing non standard Rest behavior. all the best, David I suggest the design is changed so that posts are only for create. Then creates with the same unique attributes should probably be rejected (though for big graphs this check could imply a big search - that would hit performance). This would be the normal way that entities are created. Puts can be used for update; the Rest standard does allow puts to be used for updates or creates - but normally I see Puts for update only. > Issue with uniqueAttribute while creation of entities > - > > Key: ATLAS-1743 > URL: https://issues.apache.org/jira/browse/ATLAS-1743 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.8-incubating, 0.9-incubating >Reporter: Sharmadha Sainath > > Atlas allows creation of entities with same value for unique attributes. > For example , > 1. Created an hdfs_path entity with > * name = dir1 > * qualifiedName = "userdir@cl1" > * path = /user/user1/dir > Entity is created successfully. > 2. Created another hdfs_path entity with > * name = dir2 > * qualifiedName = "userdir@cl1" > * path = /user/user2/dir > Qualified Name is uniqueAttribute and entity creation with same Qualified > Name is allowed. > Problem is when the 2nd hdfs_path entity is created , 1st entity is updated > with 2nd entity's values , hence 2nd entity creation is merely an update call > here instead of a create call because of same unique attribute. > There is no condition to check uniqueness of attributes while creation of > entities which is causing the problem. > If 1st hdfs_path entity is DELETED , then 2nd hdfs_path entity POST call > creates new entity. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1690: Attachment: Atlas Relationships proposal v1.4.pdf amended the json in the types so they were more natural extensions to the existing code. > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas Relationships proposal v1.0.pdf, Atlas > Relationships proposal v1.1.pdf, Atlas Relationships proposal v1.2.pdf, Atlas > Relationships proposal v1.3.pdf, Atlas Relationships proposal v1.4.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1690: Attachment: Atlas Relationships proposal v1.3.pdf simplify json > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas Relationships proposal v1.0.pdf, Atlas > Relationships proposal v1.1.pdf, Atlas Relationships proposal v1.2.pdf, Atlas > Relationships proposal v1.3.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1690: Attachment: Atlas Relationships proposal v1.2.pdf Amending in response to feedback from Graham Wallis. > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas Relationships proposal v1.0.pdf, Atlas > Relationships proposal v1.1.pdf, Atlas Relationships proposal v1.2.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1740) Amend types to be more intuitive when creating multiple cardinality attributes
David Radley created ATLAS-1740: --- Summary: Amend types to be more intuitive when creating multiple cardinality attributes Key: ATLAS-1740 URL: https://issues.apache.org/jira/browse/ATLAS-1740 Project: Atlas Issue Type: Bug Reporter: David Radley When specifying a multiple carnality attribute - for example hive-columns , Atlas has "typeName": "array", "cardinality": "SINGLE", This is confusing - I read this as a single array. I would expect MULTIPLE cardinality to be used. A more intuitive way would be to use: "typeName": "hive_column", "cardinality": "MULTIPLE", How we migrate the existing API should be considered here. I suggest clear design around when to use arrays and cardinality and what the difference - if any - is. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1734) Import API: Add Support to Update Attributes of Existing Types During Import
[ https://issues.apache.org/jira/browse/ATLAS-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15969859#comment-15969859 ] David Radley commented on ATLAS-1734: - [~ashutoshm] i am uncomfortable with an import being able to change the shape of an entity. Imports should be for creates and updates of entity instances. We would like Atlas to be a metadata repository for governance - so I suggest we need to control who updates the metadata. Updating schema- like information should not be done in an import. The import could flag that there are additional fields associated with entities, so that appropriate type changes could be made in a controlled way. The import should have controls to determine its behaviour in these sort of scenarios. I would suggest most companies would want to fail the complete import if a situation occurred like this - or ignore the unrecognized attributes. > Import API: Add Support to Update Attributes of Existing Types During Import > > > Key: ATLAS-1734 > URL: https://issues.apache.org/jira/browse/ATLAS-1734 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: trunk, 0.8-incubating >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry > Labels: patch > Fix For: trunk > > Attachments: > ATLAS-1734-Import-with-additional-attribute-processi.patch > > > *Background* > Existing version of Import API allows for importing types that are not > already present in the system being imported in. This causes import to fail > in the cases where the data being imported happens to have the additional > attribute. > *Solution* > During import, existing types are checked to determine if the types being > imported have additional attributes. If additional attributes exist, then the > existing type is updated with the new attributes. The import then proceeds. > *Impact Assessment* > - Import API: > -- Type import: Additional capability (mentioned above). > -- Entity creation and processing: No impact. > - Export API: No impact. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1733) No SIGKILL in python signal library on Windows
[ https://issues.apache.org/jira/browse/ATLAS-1733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15969856#comment-15969856 ] David Radley commented on ATLAS-1733: - [~grahamwallis] The Jira is still in open state - I think you want to resolve this Jira as fixed. > No SIGKILL in python signal library on Windows > -- > > Key: ATLAS-1733 > URL: https://issues.apache.org/jira/browse/ATLAS-1733 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.9-incubating > Environment: Windows 7 >Reporter: Graham Wallis >Assignee: Graham Wallis >Priority: Minor > Attachments: ATLAS-1733.patch > > Original Estimate: 0.5h > Remaining Estimate: 0.5h > > [I am new to Atlas and this is my first JIRA issue so apologies if I have > omitted any necessary information or steps.] > There is no SIGKILL signal in the python signal library on Windows, so the > import in atlas_stop.py fails. > There are two ways I can suggest that we fix it: > 1) introduce platform-specific code into atlas_stop.py > 2) wrap the python os.kill function > My preference is for the former as it is a small and localized change. > In either case the use of SIGTERM should be sufficient on Windows, as the > existing python os.kill() function will open the process and terminate it, > which is supposed to be equivalent to a SIGKILL (-9) on Linux. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1410) V2 Glossary API
[
https://issues.apache.org/jira/browse/ATLAS-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Radley updated ATLAS-1410:
Attachment: Atlas Glossary V2 proposal v1.4.pdf
Added a document to remove the classification details and remove the details of
the relationship. These will be designed separately. I have added more detail
around how semantic relationships could be implemented.
> V2 Glossary API
> ---
>
> Key: ATLAS-1410
> URL: https://issues.apache.org/jira/browse/ATLAS-1410
> Project: Atlas
> Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: Atlas Glossary V2 proposal v1.0.pdf, Atlas Glossary V2
> proposal v1.1.pdf, Atlas Glossary V2 proposal v1.2.pdf, Atlas Glossary V2
> proposal v1.3.pdf, Atlas Glossary V2 proposal v1.4.pdf
>
>
> The BaseResourceDefinition uses the AttributeDefintion class from typesystem.
> There are newer more funcitonal versions of this capability in the atlas-intg
> project. This Jira is changing over the glossary implementation to the newer
> entity / type classes.
> Instread of the instanceProperties and collectionProperties in the
> BaseResourceDefintions we should use something in this sort of style :
> "
> AtlasEntityDef deptTypeDef =
> AtlasTypeUtil.createClassTypeDef(DEPARTMENT_TYPE,
> "Department"+_description, ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> new AtlasAttributeDef("employees",
> String.format("array<%s>", "Person"), true,
> AtlasAttributeDef.Cardinality.SINGLE, 0, 1,
> false, false,
>
> Collections.emptyList()));
> AtlasEntityDef personTypeDef =
> AtlasTypeUtil.createClassTypeDef("Person", "Person"+_description,
> ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> AtlasTypeUtil.createOptionalAttrDef("address", "Address"),
> AtlasTypeUtil.createOptionalAttrDef("birthday", "date"),
> AtlasTypeUtil.createOptionalAttrDef("hasPets", "boolean"),
> AtlasTypeUtil.createOptionalAttrDef("numberOfCars", "byte"),
> AtlasTypeUtil.createOptionalAttrDef("houseNumber", "short"),
> AtlasTypeUtil.createOptionalAttrDef("carMileage", "int"),
> AtlasTypeUtil.createOptionalAttrDef("age", "float"),
> "
> For the parent child relationships with glossary categories and terms we
> should be able to have the type system manage edge deletion. As part of this,
> we will need to investigate whether we could get rid of the disconnect and
> connect methods added in ATLAS-1186
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1690: Attachment: Atlas Relationships proposal v1.1.pdf Updated doc in response to Stefhan's feedback. > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas Relationships proposal v1.0.pdf, Atlas > Relationships proposal v1.1.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967442#comment-15967442 ] David Radley commented on ATLAS-1690: - Thank you very much [~Stefhan] for the quick thorough feedback. REsoponding to you feedback in <> tags : Page 1: •“Metadata stores store metadata” re-write this to “Metadata repository stores metadata” as it is less confusing. <> •Suggestion to replace ‘metadata entity’ with ‘metadata object’, as the term object is more generic. <> •“Relationships include when Glossary Terms are linked to each other, assets mapped to Glossary Terms and asset metadata linked together.” Are there types of relationships between metadata objects that are excluded from this scope? <> Page 2: •“As a 1 When this document refers to Glossary Term – it is referring to those described in Jira 1410. Page 2 of 9 Glossary Term is linked to an asset, we need to be aware that there are 3 owners, one for the Glossary Term one for the asset and one for the relationship. This document proposes a design that allows relationships to be considered in this way.” Does this state that we consider relationships as ‘first class citizens’? As such that the relationship between two objects becomes an object on its own, with a unique identifier and potentially additional metadata? <> •“When this document refers to Glossary Term – it is referring to those described in Jira 1410.” This exact sentence was already mentioned as footnote on page 1. <> Page 5: -“Relationship constraints can have an attribute name and a reverse attribute name, but there is no label for the relationship itself – the relationship name.” What about synonym relationships? <> What would then be the attribute name and the reverse attribute name? <> -With regards to the Bidirectional relationships example. It would be: ‘A person has a home address’/’A home address is of a person’. The ‘has a’ / ‘is of’ are two directional relationships that would always need to be kept in sync. <> ‘A home address is a address’. A more accurate example of a bidirectional relationship would be ‘First name is synonym of Given name’ / ‘Given name is synonym of First name’ <> -With regards of the styles of relationships, perhaps its good idea to adopt the terminology of: oHierarchical relationships (inheritance) oEquivalence relationships (synonyms/is a) oAssociative relationships (related to / loosely coupled) https://twobenches.wordpress.com/2008/06/04/semantic-relationships-in-a-thesaurus/ For inspiration see: Morville, Peter and Rosenfeld, Louis (2002) http://yunus.hacettepe.edu.tr/~tonta/courses/fall2010/bby607/IAWWW.pdf Page number 203, page 225, Definition ‘thesaurus’: A controlled vocabulary in which equivalence, hierarchical, and associative relationships are identified for purposes of improved retrieval.* * Guidelines for the Construction, Format, and Management of Monolingual Thesauri. ANSI/NISO Z39.19–1993 (R1998). See also Figure 9-12. Semantic relationships in a thesaurus (Page number 204, page 226). See also Semantic Relationships (page number 215, page 237) <> Page 7: -Delete functionality. This is crucial as it will allow to cope with changes in the metadata objects, and relationships provided by other metadata repositories (e.g. other atlas instances). -With regards to the discussion point on endpoints. Endpoints should not be updated / modified when working on relationships. The relationship should point to the object and subject identifier without impacting those objects in any way. A separate mechanism should be responsible for interpreting the relationships and if needed adjusting/updating the associated objects. E.g. collapsing the assigned classification into the subject or one of the endpoints of the subject (e.g. assigned information view). <>. Page 9: -On ‘1 to 1 relationships’: I do not understand why this would be an issue, if you would define the relationship object independently of the endpoints. In that way, both endpoints need to exist (otherwise, you will not even be able to select them) before you can create the relationship. Only the relationship object will then be created pointing to the identifiers of both the subject and object. <> > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas Relationshi
[jira] [Commented] (ATLAS-1726) Creating type name with space causes exceptions while doing DSL search
[
https://issues.apache.org/jira/browse/ATLAS-1726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967366#comment-15967366
]
David Radley commented on ATLAS-1726:
-
[~sarath.ku...@gmail.com] This is a good catch.
I notice http://www.json.org/ talks of valid json strings as any unicode
character apart from " or \. I am not sure whether our Jackson json support
introduces more restrictions.
I wonder do we :
1) support blanks in type names. If we do presumably we should support blanks
in the other type related structures like AttributeDef names. We may not want
to support an uni-code characters as type names, leading blank characters in
type names, leading numbers. Whether we support non-english characters is
another question, the advantage of sticking to English characters is that it
helps debugging; but there should then be a way to globalise these strings.
2) We do not support type names with blanks in and are much more restrictive
in the characters - for example see
https://db.apache.org/derby/docs/10.1/ref/crefsqlj1003454.html.
In both cases we should explicitly document/police exactly which characters are
supported and any encoding restrictions. We should also police document the
maximum length. It would be great to see a set of unit tests to tests all the
valid and invalids here.
> Creating type name with space causes exceptions while doing DSL search
> --
>
> Key: ATLAS-1726
> URL: https://issues.apache.org/jira/browse/ATLAS-1726
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating, 0.9-incubating
>Reporter: Sharmadha Sainath
>Assignee: Sarath Subramanian
>
> 1.Created a type "type 1" successfully
> 2.Searched the type "type 1" in Basic search which worked fine.
> 3.Same type when selected in drop down list of types in DSL , failed with 500
> internal server error with exception :
> {code}
> org.apache.atlas.query.Expressions$UnresolvedException: Unresolved datatype.
> Can not resolve due to unresolved children, expression: type where 1
> {code}
> 4. If typename is provided as `type 1` in "Search by Query" text box (with
> backquotes) like mentioned in https://issues.apache.org/jira/browse/ATLAS-263
> , it works fine.
> User might perceive that the type is not registered properly due to the
> exception.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (ATLAS-1459) Atlas V2 types to manage bidirectional relationships
[ https://issues.apache.org/jira/browse/ATLAS-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley resolved ATLAS-1459. - Resolution: Duplicate I am doing the work for this in Jira 1690 > Atlas V2 types to manage bidirectional relationships > - > > Key: ATLAS-1459 > URL: https://issues.apache.org/jira/browse/ATLAS-1459 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Fix For: 0.8-incubating > > > Currently Atlas can use mappedFromRef constraint and the refAttribute param > in an attribute to point to a child type that points back to the parent . > See 0030-hive-model.json for an example. > In an operational environment where we want to update these attributes - we > want the corresponding attribute (related using the constraints) to be > updated. > I suggest that we add an inSyncRefAttribute in addition to the existing > refAttribute. If inSyncRefAttribute constraint is specified then the > attribute pointed to is kept in sync with our attribute. On creates, updates > and deletes. This will be supported for List and non List attributes. > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1690) Introduce top level relationships
[ https://issues.apache.org/jira/browse/ATLAS-1690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1690: Attachment: Atlas Relationships proposal v1.0.pdf Initial design proposal for relationships. > Introduce top level relationships > - > > Key: ATLAS-1690 > URL: https://issues.apache.org/jira/browse/ATLAS-1690 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: Atlas Relationships proposal v1.0.pdf > > > Introduce top level relationships including support for > -many to many relationships > - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1693) Improve Classifications to introduce Classification Schemes
[ https://issues.apache.org/jira/browse/ATLAS-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1693: Summary: Improve Classifications to introduce Classification Schemes (was: Improve Classifications to introduce Classificaiton Schemes) > Improve Classifications to introduce Classification Schemes > --- > > Key: ATLAS-1693 > URL: https://issues.apache.org/jira/browse/ATLAS-1693 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley > Labels: VirtualDataConnector > > Improve Classifications to introduce Classification Schemes. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1713) V2 API to delete a type requires type definition to given as body
[
https://issues.apache.org/jira/browse/ATLAS-1713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15954802#comment-15954802
]
David Radley commented on ATLAS-1713:
-
I agree - the type API should accept the type name or the guid on the REST
endpoint,
Some questions
- The interesting piece is when do you fail a delete of a type.
- I suggest that we should only allow a delete of a type that has no associated
active instances. In this way a single type delete cannot take out many
entities.
- I assume we will soft delete a type - so it is still present in the graph.
- if there are relationships to / from other types - then potentially those
other types are invalidated by the deletion of this type. Another type may
reference or have a non-optional relationship to this type.
I suggest a design doc be created for this item to ensure that the metadata
repository deals with the potential loss of integrity after a type delete.
> V2 API to delete a type requires type definition to given as body
> --
>
> Key: ATLAS-1713
> URL: https://issues.apache.org/jira/browse/ATLAS-1713
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Sharmadha Sainath
>Priority: Minor
>
> V2 API
> {code}
> api/atlas/v2/types/typedefs
> {code}
> requires body data when doing a HTTP DELETE request .
> Since type has both name and guid , it could be given as path instead of
> sending the whole type definition as body.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (ATLAS-1700) Add constraints to JAVA API
[ https://issues.apache.org/jira/browse/ATLAS-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1700: Description: Add constraints to JAVA API for the Glossary. There are some constraints that can be captured using the constraints in the type model. For the Glossary API, there are restrictions that cannot be captured in the type model. I am thinking of constraints like: - policing that we do not allow a category to be updated to have a parent as one of its descendants. - policing that we do not allow a category from another glossary to be added under our category - policing that we do not allow a category to be updated, such that the category tree contains categories from more than one glossary. - restricting terms under a category to have unique names (we need to get consensus as to whether this is something we would like) These sorts of restrictions need to policed in the Java code - as there is no place for them in the model. For the Glossary, we are looking to add a new OMAS layer around the base Atlas entity APIs. was: Add constraints to JAVA API for the Glossary. There are some constraints that can be captured using the constraints in the type model. For the Glossary API, there are restrictions that cannot be captured in the type model. I am thinking of constraints like: - policing that we do not allow a category to be updated to have a parent as one of its descendants. - policing that we do not allow a category from another glossary to be added under out category - policing that we do not allow a category to be updated such that the category tree contains categories from more than one glossary. - restricting terms under a category to have unique names (we need to get consensus as to whether this is something we would like) These sorts of restrictions would need to policed in the Java code - as there is no place for them in the model. For the Glossary, we are looking to add a new OMAS layer around the base Atlas entity APIs. > Add constraints to JAVA API > --- > > Key: ATLAS-1700 > URL: https://issues.apache.org/jira/browse/ATLAS-1700 > Project: Atlas > Issue Type: Sub-task >Reporter: David Radley > Labels: VirtualDataConnector > > Add constraints to JAVA API for the Glossary. There are some constraints that > can be captured using the constraints in the type model. For the Glossary > API, there are restrictions that cannot be captured in the type model. I am > thinking of constraints like: > - policing that we do not allow a category to be updated to have a parent as > one of its descendants. > - policing that we do not allow a category from another glossary to be added > under our category > - policing that we do not allow a category to be updated, such that the > category tree contains categories from more than one glossary. > - restricting terms under a category to have unique names (we need to get > consensus as to whether this is something we would like) > > These sorts of restrictions need to policed in the Java code - as there is no > place for them in the model. For the Glossary, we are looking to add a new > OMAS layer around the base Atlas entity APIs. > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1700) Add constraints to JAVA API
[ https://issues.apache.org/jira/browse/ATLAS-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1700: Description: Add constraints to JAVA API for the Glossary. There are some constraints that can be captured using the constraints in the type model. For the Glossary API, there are restrictions that cannot be captured in the type model. I am thinking of constraints like: - policing that we do not allow a category to be updated to have a parent as one of its descendants. - policing that we do not allow a category from another glossary to be added under out category - policing that we do not allow a category to be updated such that the category tree contains categories from more than one glossary. - restricting terms under a category to have unique names (we need to get consensus as to whether this is something we would like) These sorts of restrictions would need to policed in the Java code - as there is no place for them in the model. For the Glossary, we are looking to add a new OMAS layer around the base Atlas entity APIs. was:Add constraints to JAVA API > Add constraints to JAVA API > --- > > Key: ATLAS-1700 > URL: https://issues.apache.org/jira/browse/ATLAS-1700 > Project: Atlas > Issue Type: Sub-task >Reporter: David Radley > Labels: VirtualDataConnector > > Add constraints to JAVA API for the Glossary. There are some constraints that > can be captured using the constraints in the type model. For the Glossary > API, there are restrictions that cannot be captured in the type model. I am > thinking of constraints like: > - policing that we do not allow a category to be updated to have a parent as > one of its descendants. > - policing that we do not allow a category from another glossary to be added > under out category > - policing that we do not allow a category to be updated such that the > category tree contains categories from more than one glossary. > - restricting terms under a category to have unique names (we need to get > consensus as to whether this is something we would like) > > These sorts of restrictions would need to policed in the Java code - as there > is no place for them in the model. For the Glossary, we are looking to add a > new OMAS layer around the base Atlas entity APIs. > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1700) Add constraints to JAVA API
[ https://issues.apache.org/jira/browse/ATLAS-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15953096#comment-15953096 ] David Radley commented on ATLAS-1700: - [~apoorvnaik] Thanks Apporv - I have added more detail - it should make more sense now. > Add constraints to JAVA API > --- > > Key: ATLAS-1700 > URL: https://issues.apache.org/jira/browse/ATLAS-1700 > Project: Atlas > Issue Type: Sub-task >Reporter: David Radley > Labels: VirtualDataConnector > > Add constraints to JAVA API for the Glossary. There are some constraints that > can be captured using the constraints in the type model. For the Glossary > API, there are restrictions that cannot be captured in the type model. I am > thinking of constraints like: > - policing that we do not allow a category to be updated to have a parent as > one of its descendants. > - policing that we do not allow a category from another glossary to be added > under out category > - policing that we do not allow a category to be updated such that the > category tree contains categories from more than one glossary. > - restricting terms under a category to have unique names (we need to get > consensus as to whether this is something we would like) > > These sorts of restrictions would need to policed in the Java code - as there > is no place for them in the model. For the Glossary, we are looking to add a > new OMAS layer around the base Atlas entity APIs. > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (ATLAS-1704) amend Atlas simple security to add glossary
[ https://issues.apache.org/jira/browse/ATLAS-1704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley resolved ATLAS-1704. - Resolution: Duplicate This was raised by mistake - I am closing this out - thanks for pointing this out Hermanth. > amend Atlas simple security to add glossary > - > > Key: ATLAS-1704 > URL: https://issues.apache.org/jira/browse/ATLAS-1704 > Project: Atlas > Issue Type: Sub-task >Reporter: David Radley > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1704) amend Atlas simple security to add glossary
David Radley created ATLAS-1704: --- Summary: amend Atlas simple security to add glossary Key: ATLAS-1704 URL: https://issues.apache.org/jira/browse/ATLAS-1704 Project: Atlas Issue Type: Sub-task Reporter: David Radley -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1705) Update Atlas UI to use V2 Glossary
David Radley created ATLAS-1705: --- Summary: Update Atlas UI to use V2 Glossary Key: ATLAS-1705 URL: https://issues.apache.org/jira/browse/ATLAS-1705 Project: Atlas Issue Type: Sub-task Reporter: David Radley -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1703) amend Atlas simple security to add glossary
David Radley created ATLAS-1703: --- Summary: amend Atlas simple security to add glossary Key: ATLAS-1703 URL: https://issues.apache.org/jira/browse/ATLAS-1703 Project: Atlas Issue Type: Sub-task Reporter: David Radley -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1410) V2 Glossary API
[
https://issues.apache.org/jira/browse/ATLAS-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949427#comment-15949427
]
David Radley commented on ATLAS-1410:
-
Hi [~Stefhan] and [~clyned] . Thanks you for your feedback. From recent
discussions, prompted by your feedback, we are thinking we need:
- relationships to be top level relationships. I have raised this as a subtask.
- relationships should have modelling flags to indicate composite , names of
each side and the relationship, cardinality and the like.
We think that if the association ends are not named then we can default the
names to has-a etc. I think this is a nice compromise to give some default
relationship names but also encourage custom names and to use the modelling
flags to see what the real meaning is.
- We could introduce default names like has-an. I think has-a and has-an are a
bit confusing as in English an is used when the noun starts with a vowel.
> V2 Glossary API
> ---
>
> Key: ATLAS-1410
> URL: https://issues.apache.org/jira/browse/ATLAS-1410
> Project: Atlas
> Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: Atlas Glossary V2 proposal v1.0.pdf, Atlas Glossary V2
> proposal v1.1.pdf, Atlas Glossary V2 proposal v1.2.pdf, Atlas Glossary V2
> proposal v1.3.pdf
>
>
> The BaseResourceDefinition uses the AttributeDefintion class from typesystem.
> There are newer more funcitonal versions of this capability in the atlas-intg
> project. This Jira is changing over the glossary implementation to the newer
> entity / type classes.
> Instread of the instanceProperties and collectionProperties in the
> BaseResourceDefintions we should use something in this sort of style :
> "
> AtlasEntityDef deptTypeDef =
> AtlasTypeUtil.createClassTypeDef(DEPARTMENT_TYPE,
> "Department"+_description, ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> new AtlasAttributeDef("employees",
> String.format("array<%s>", "Person"), true,
> AtlasAttributeDef.Cardinality.SINGLE, 0, 1,
> false, false,
>
> Collections.emptyList()));
> AtlasEntityDef personTypeDef =
> AtlasTypeUtil.createClassTypeDef("Person", "Person"+_description,
> ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> AtlasTypeUtil.createOptionalAttrDef("address", "Address"),
> AtlasTypeUtil.createOptionalAttrDef("birthday", "date"),
> AtlasTypeUtil.createOptionalAttrDef("hasPets", "boolean"),
> AtlasTypeUtil.createOptionalAttrDef("numberOfCars", "byte"),
> AtlasTypeUtil.createOptionalAttrDef("houseNumber", "short"),
> AtlasTypeUtil.createOptionalAttrDef("carMileage", "int"),
> AtlasTypeUtil.createOptionalAttrDef("age", "float"),
> "
> For the parent child relationships with glossary categories and terms we
> should be able to have the type system manage edge deletion. As part of this,
> we will need to investigate whether we could get rid of the disconnect and
> connect methods added in ATLAS-1186
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (ATLAS-1702) documentation
David Radley created ATLAS-1702: --- Summary: documentation Key: ATLAS-1702 URL: https://issues.apache.org/jira/browse/ATLAS-1702 Project: Atlas Issue Type: Sub-task Reporter: David Radley -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1701) Swagger
David Radley created ATLAS-1701: --- Summary: Swagger Key: ATLAS-1701 URL: https://issues.apache.org/jira/browse/ATLAS-1701 Project: Atlas Issue Type: Sub-task Reporter: David Radley -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1699) Add REST API
[ https://issues.apache.org/jira/browse/ATLAS-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1699: Request participants: (was: ) Summary: Add REST API (was: Add constraints to JAVA API) > Add REST API > > > Key: ATLAS-1699 > URL: https://issues.apache.org/jira/browse/ATLAS-1699 > Project: Atlas > Issue Type: Sub-task >Reporter: David Radley > Labels: VirtualDataConnector > > Add constraints to JAVA API -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1699) Add constraints to JAVA API
David Radley created ATLAS-1699: --- Summary: Add constraints to JAVA API Key: ATLAS-1699 URL: https://issues.apache.org/jira/browse/ATLAS-1699 Project: Atlas Issue Type: Sub-task Reporter: David Radley Add constraints to JAVA API -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1700) Add constraints to JAVA API
David Radley created ATLAS-1700: --- Summary: Add constraints to JAVA API Key: ATLAS-1700 URL: https://issues.apache.org/jira/browse/ATLAS-1700 Project: Atlas Issue Type: Sub-task Reporter: David Radley Add constraints to JAVA API -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1698) Create OMAS Glossary project, new types and Java API
David Radley created ATLAS-1698: --- Summary: Create OMAS Glossary project, new types and Java API Key: ATLAS-1698 URL: https://issues.apache.org/jira/browse/ATLAS-1698 Project: Atlas Issue Type: Sub-task Reporter: David Radley -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1693) Improve Classifications to introduce Classificaiton Schemes
David Radley created ATLAS-1693: --- Summary: Improve Classifications to introduce Classificaiton Schemes Key: ATLAS-1693 URL: https://issues.apache.org/jira/browse/ATLAS-1693 Project: Atlas Issue Type: Improvement Reporter: David Radley Improve Classifications to introduce Classification Schemes. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1692) Introduce the ability to relate a type to any type
David Radley created ATLAS-1692: --- Summary: Introduce the ability to relate a type to any type Key: ATLAS-1692 URL: https://issues.apache.org/jira/browse/ATLAS-1692 Project: Atlas Issue Type: Improvement Reporter: David Radley Introduce a hierarchy of types. So every type inherits from a base type. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1690) Introduce top level relationships
David Radley created ATLAS-1690: --- Summary: Introduce top level relationships Key: ATLAS-1690 URL: https://issues.apache.org/jira/browse/ATLAS-1690 Project: Atlas Issue Type: Improvement Reporter: David Radley Assignee: David Radley Introduce top level relationships including support for -many to many relationships - relationship names including the name for both ends and the relationship. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Reopened] (ATLAS-1459) Atlas V2 types to manage bidirectional relationships
[ https://issues.apache.org/jira/browse/ATLAS-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley reopened ATLAS-1459: - Assignee: David Radley Re opening - as we need true bidirectional relationships. > Atlas V2 types to manage bidirectional relationships > - > > Key: ATLAS-1459 > URL: https://issues.apache.org/jira/browse/ATLAS-1459 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Fix For: 0.8-incubating > > > Currently Atlas can use mappedFromRef constraint and the refAttribute param > in an attribute to point to a child type that points back to the parent . > See 0030-hive-model.json for an example. > In an operational environment where we want to update these attributes - we > want the corresponding attribute (related using the constraints) to be > updated. > I suggest that we add an inSyncRefAttribute in addition to the existing > refAttribute. If inSyncRefAttribute constraint is specified then the > attribute pointed to is kept in sync with our attribute. On creates, updates > and deletes. This will be supported for List and non List attributes. > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1689) Virtual Data connector
David Radley created ATLAS-1689: --- Summary: Virtual Data connector Key: ATLAS-1689 URL: https://issues.apache.org/jira/browse/ATLAS-1689 Project: Atlas Issue Type: Improvement Reporter: David Radley Virtual Data Connector - this is a project to create a meta-data driven data lake in which Apache Atlas is the meta-data store. Create an open, metadata-driven virtual connector to data repositories – a Data Virtualization Platform that will satisfy the following goals: Give users one place to access each and every Data Lake repository (search and discover) Give users one search interface for both data and metadata inside the Data Lake (search and discover) To provide one place to look for and provision data for all kinds of end users To provide one place to build and maintain data security for all data repositories inside the Data Lake -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1410) V2 Glossary API
[
https://issues.apache.org/jira/browse/ATLAS-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Radley updated ATLAS-1410:
Attachment: Atlas Glossary V2 proposal v1.3.pdf
I have removed scenario 4 and collections and tidied up parts of the document
in response to feedback.
> V2 Glossary API
> ---
>
> Key: ATLAS-1410
> URL: https://issues.apache.org/jira/browse/ATLAS-1410
> Project: Atlas
> Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: Atlas Glossary V2 proposal v1.0.pdf, Atlas Glossary V2
> proposal v1.1.pdf, Atlas Glossary V2 proposal v1.2.pdf, Atlas Glossary V2
> proposal v1.3.pdf
>
>
> The BaseResourceDefinition uses the AttributeDefintion class from typesystem.
> There are newer more funcitonal versions of this capability in the atlas-intg
> project. This Jira is changing over the glossary implementation to the newer
> entity / type classes.
> Instread of the instanceProperties and collectionProperties in the
> BaseResourceDefintions we should use something in this sort of style :
> "
> AtlasEntityDef deptTypeDef =
> AtlasTypeUtil.createClassTypeDef(DEPARTMENT_TYPE,
> "Department"+_description, ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> new AtlasAttributeDef("employees",
> String.format("array<%s>", "Person"), true,
> AtlasAttributeDef.Cardinality.SINGLE, 0, 1,
> false, false,
>
> Collections.emptyList()));
> AtlasEntityDef personTypeDef =
> AtlasTypeUtil.createClassTypeDef("Person", "Person"+_description,
> ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> AtlasTypeUtil.createOptionalAttrDef("address", "Address"),
> AtlasTypeUtil.createOptionalAttrDef("birthday", "date"),
> AtlasTypeUtil.createOptionalAttrDef("hasPets", "boolean"),
> AtlasTypeUtil.createOptionalAttrDef("numberOfCars", "byte"),
> AtlasTypeUtil.createOptionalAttrDef("houseNumber", "short"),
> AtlasTypeUtil.createOptionalAttrDef("carMileage", "int"),
> AtlasTypeUtil.createOptionalAttrDef("age", "float"),
> "
> For the parent child relationships with glossary categories and terms we
> should be able to have the type system manage edge deletion. As part of this,
> we will need to investigate whether we could get rid of the disconnect and
> connect methods added in ATLAS-1186
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (ATLAS-1410) V2 Glossary API
[ https://issues.apache.org/jira/browse/ATLAS-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15936571#comment-15936571 ] David Radley commented on ATLAS-1410: - Responding to [~eostic] Use Case 1 (page 5), with references also to comments on page 7 "It is important that duplicate Glossary Term names can be defined in the glossary, each with their own context..". That "context" is important, and in fact, should be what makes the Term unique. It might be a "duplicate" in the highest sense of the word, for the whole "collection" of Terms in a glossary, but parentage is indeed important, and keeps things separate. Enterprises can't always agree on Term names or their categorization. One company might, in an insurance example, spend the time to clarify their terms and be sure to have "Automobile Accident Claim" along with "Homeowners Claim" in the same Glossary, but another site might just as easily have the same Term "Claim", existing many times.once in a "Personal Lines/Claims/Auto/Accident" category and another in "Personal Lines/Claims/Homeowners" category. <> This is a complex topic, however so it might be beneficial for the model and the APIs to allow full duplication within the higher Glossary level, without requiring parentage definition, leaving it to the "implementer" of any GUI to support (or not) some further level of identitybut it should be strongly recommended. Use Case 9. Classifying terms rather than assets sounds very natural, but shouldn't be a "requirement". Is this implying that direct asset classification wouldn't be permitted? <>There will be times when the classifications are only applicable to assets<>, because a Term does not yet exist, or because the type of classification isn't fine grained enough, or other equally creative reasons. Page 7. Kinds of Glossary Terms. It isn't clear why there is a need for different formal "types" of mutually exclusive Terms. Relationships determine the "use" of a particular Term, and if it is important for a consumer to have, for their users and model, a set of "Semantic" terms vs "Classifying" terms, it can be done other ways, such as by putting the Terms into their own separate categories or parental structures. <> Page 10 and 11 this discussion is more clear now, since the removal of the idea of an "Attribute Term". On page 11 specifically, it may be beneficial to also show how "has-a" can cascade. Customer "has_a" address, and then address could also "has_a" City, State and Zip (and so forth...). <> Page 13. Great that you brought up the need for custom relationships. We need to ensure that this capability as "hoped for", remains intact. ; ) Page 14. Perhaps it needs more explanation, but I found the definition of Has-type and Types to be a bit confusing. "Has-types/is-a-type-of" seems more natural.and that perhaps these could be combined into one. <> Page 14. Synonyms. Perhaps needs more explanation? Synonyms are difficult to have any kind of "owner". They are all peers in a "collection" of similar concepts. Having one owner, in the model itself, could create issues if/when that owner is deleted. <> Page 14. Antonymsneeds further definition so that it is explained separately from Synonyms. In this case, there could be many Terms that are opposites, but they themselves are not necessarily antonyms of each other. This one seems ok to have an "owner" concept. <> Page 14 Homonyms. This one is more like Synonyms, where they can be peers of each other. Page 15. Preferred Term. Great concept. Especially important for enterprises that are overloading the glossary to meet a lot of their governance objectives, but still want to retain the idea that "this term" is "the one to use" for specific alternate name, or priority reference purposes. Specifically critical to scenarios where Terms are seen as a "replacement for names in retrieval requests or reporting tool interfaces". <> Page 15. Collections. Very important concept — but is it part of the Glossary specification? <> ...or should it be reviewed at a much higher Atlas perspective? Certainly the glossary could have a set of Terms, qualified in some way as a "Collection Glossary" and then more generically use "assigned assets" [including other Terms] as a generic relationship but it maybe that this is overloading the Glossary too much > V2 Glossary API > --- > > Key: ATLAS-1410 > URL: https://issues.apache.org/jira/browse/ATLAS-1410 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Attachments: Atlas Glossary V2 proposal v1.0.pdf, Atlas Glossary V2 > proposal v1.1.pdf, Atlas Glossary V2 proposal v1.2.pdf > > > The BaseResourceDefinition uses the AttributeDefintion class from typesyste
[jira] [Updated] (ATLAS-1410) V2 Glossary API
[
https://issues.apache.org/jira/browse/ATLAS-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Radley updated ATLAS-1410:
Attachment: Atlas Glossary V2 proposal v1.2.pdf
I have updated the document - based on the feedback received so far. Thanks for
all the feedback to date.
> V2 Glossary API
> ---
>
> Key: ATLAS-1410
> URL: https://issues.apache.org/jira/browse/ATLAS-1410
> Project: Atlas
> Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: Atlas Glossary V2 proposal v1.0.pdf, Atlas Glossary V2
> proposal v1.1.pdf, Atlas Glossary V2 proposal v1.2.pdf
>
>
> The BaseResourceDefinition uses the AttributeDefintion class from typesystem.
> There are newer more funcitonal versions of this capability in the atlas-intg
> project. This Jira is changing over the glossary implementation to the newer
> entity / type classes.
> Instread of the instanceProperties and collectionProperties in the
> BaseResourceDefintions we should use something in this sort of style :
> "
> AtlasEntityDef deptTypeDef =
> AtlasTypeUtil.createClassTypeDef(DEPARTMENT_TYPE,
> "Department"+_description, ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> new AtlasAttributeDef("employees",
> String.format("array<%s>", "Person"), true,
> AtlasAttributeDef.Cardinality.SINGLE, 0, 1,
> false, false,
>
> Collections.emptyList()));
> AtlasEntityDef personTypeDef =
> AtlasTypeUtil.createClassTypeDef("Person", "Person"+_description,
> ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> AtlasTypeUtil.createOptionalAttrDef("address", "Address"),
> AtlasTypeUtil.createOptionalAttrDef("birthday", "date"),
> AtlasTypeUtil.createOptionalAttrDef("hasPets", "boolean"),
> AtlasTypeUtil.createOptionalAttrDef("numberOfCars", "byte"),
> AtlasTypeUtil.createOptionalAttrDef("houseNumber", "short"),
> AtlasTypeUtil.createOptionalAttrDef("carMileage", "int"),
> AtlasTypeUtil.createOptionalAttrDef("age", "float"),
> "
> For the parent child relationships with glossary categories and terms we
> should be able to have the type system manage edge deletion. As part of this,
> we will need to investigate whether we could get rid of the disconnect and
> connect methods added in ATLAS-1186
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (ATLAS-1410) V2 Glossary API
[
https://issues.apache.org/jira/browse/ATLAS-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15932746#comment-15932746
]
David Radley commented on ATLAS-1410:
-
Thanks you very much for the feedback [~Stefhan]. some responses
Page 5:
Use case 1: Changed as discussed
Use case 2 and Page 6:
<>
Page 7: <>
Unclear why there is a need to have different term types. <>
Page 8 "A Classification points to one entity and can have many associated
term." I don't think i fully understand this statement. It would be wiser to
have the classification point to one or more terms and that the term will point
to one or more entities. To be further discussed. <>
Also it should be possible to have multiple classification pointing to the same
object. <>
Page 9 "The classification associated with the term should not be automatically
cascaded by Atlas to the assigned assets." Agree that Atlas does not
necessarily needs to do the cascading because logic might need to be involved.
<> However, the result might need to be made available in Atlas
and shown in a distinct way. If Atlas is seen as the single source of truth
then it must be possible for a end user to see from solely Atlas that a
classification is 'Derived from'. How that derivation has occurred can happen
by a different service. <>
changes will be in the next document.
> V2 Glossary API
> ---
>
> Key: ATLAS-1410
> URL: https://issues.apache.org/jira/browse/ATLAS-1410
> Project: Atlas
> Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: Atlas Glossary V2 proposal v1.0.pdf, Atlas Glossary V2
> proposal v1.1.pdf
>
>
> The BaseResourceDefinition uses the AttributeDefintion class from typesystem.
> There are newer more funcitonal versions of this capability in the atlas-intg
> project. This Jira is changing over the glossary implementation to the newer
> entity / type classes.
> Instread of the instanceProperties and collectionProperties in the
> BaseResourceDefintions we should use something in this sort of style :
> "
> AtlasEntityDef deptTypeDef =
> AtlasTypeUtil.createClassTypeDef(DEPARTMENT_TYPE,
> "Department"+_description, ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> new AtlasAttributeDef("employees",
> String.format("array<%s>", "Person"), true,
> AtlasAttributeDef.Cardinality.SINGLE, 0, 1,
> false, false,
>
> Collections.emptyList()));
> AtlasEntityDef personTypeDef =
> AtlasTypeUtil.createClassTypeDef("Person", "Person"+_description,
> ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> AtlasTypeUtil.createOptionalAttrDef("address", "Address"),
> AtlasTypeUtil.createOptionalAttrDef("birthday", "date"),
> AtlasTypeUtil.createOptionalAttrDef("hasPets", "boolean"),
> AtlasTypeUtil.createOptionalAttrDef("numberOfCars", "byte"),
> AtlasTypeUtil.createOptionalAttrDef("houseNumber", "short"),
> AtlasTypeUtil.createOptionalAttrDef("carMileage", "int"),
> AtlasTypeUtil.createOptionalAttrDef("age", "float"),
> "
> For the parent child relationships with glossary categories and terms we
> should be able to have the type system manage edge deletion. As part of this,
> we will need to investigate whether we could get rid of the disconnect and
> connect methods added in ATLAS-1186
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (ATLAS-1410) V2 Glossary API
[
https://issues.apache.org/jira/browse/ATLAS-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15932552#comment-15932552
]
David Radley commented on ATLAS-1410:
-
Responding to [~jonesn]
6- Ok so a term belongs in a glossary, but can be categorized by one in another
. I understand this from an object perspective but trying to think of an
example as to why that is needed? I guess I'm not clear on the meaning of
having multiple glossaries that are interlinked << David this allows us to
separate terms into subject areas. It also allows duplicate term names to occur
by putting them in different glossaries; something Stefhan is keen on. Also
glossaries are the owning entity for terms, so there is no need for a single
owning category. >>
8- Are you saying that additional attribute values can be stored with the
classification object? I'm thinking here of the example tag based policies
covered at section 8.2 of
https://cwiki.apache.org/confluence/display/RANGER/Tag+Based+Policies where
"EXPIRES_ON" is referred to << David the classification type would be
subclassed and new attributes added>>
> V2 Glossary API
> ---
>
> Key: ATLAS-1410
> URL: https://issues.apache.org/jira/browse/ATLAS-1410
> Project: Atlas
> Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: Atlas Glossary V2 proposal v1.0.pdf, Atlas Glossary V2
> proposal v1.1.pdf
>
>
> The BaseResourceDefinition uses the AttributeDefintion class from typesystem.
> There are newer more funcitonal versions of this capability in the atlas-intg
> project. This Jira is changing over the glossary implementation to the newer
> entity / type classes.
> Instread of the instanceProperties and collectionProperties in the
> BaseResourceDefintions we should use something in this sort of style :
> "
> AtlasEntityDef deptTypeDef =
> AtlasTypeUtil.createClassTypeDef(DEPARTMENT_TYPE,
> "Department"+_description, ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> new AtlasAttributeDef("employees",
> String.format("array<%s>", "Person"), true,
> AtlasAttributeDef.Cardinality.SINGLE, 0, 1,
> false, false,
>
> Collections.emptyList()));
> AtlasEntityDef personTypeDef =
> AtlasTypeUtil.createClassTypeDef("Person", "Person"+_description,
> ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> AtlasTypeUtil.createOptionalAttrDef("address", "Address"),
> AtlasTypeUtil.createOptionalAttrDef("birthday", "date"),
> AtlasTypeUtil.createOptionalAttrDef("hasPets", "boolean"),
> AtlasTypeUtil.createOptionalAttrDef("numberOfCars", "byte"),
> AtlasTypeUtil.createOptionalAttrDef("houseNumber", "short"),
> AtlasTypeUtil.createOptionalAttrDef("carMileage", "int"),
> AtlasTypeUtil.createOptionalAttrDef("age", "float"),
> "
> For the parent child relationships with glossary categories and terms we
> should be able to have the type system manage edge deletion. As part of this,
> we will need to investigate whether we could get rid of the disconnect and
> connect methods added in ATLAS-1186
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (ATLAS-1410) V2 Glossary API
[ https://issues.apache.org/jira/browse/ATLAS-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15928409#comment-15928409 ] David Radley commented on ATLAS-1410: - Thank you [~zimnymc] for your feedback. It is great. Here are my responses ad. Use Case use case 1 It shouldn't be possible to define two terms with exactly the same name. <> It can be possible to do it only through synonyms if definition stays the same. <> If we have different definition then we also must have different name for each term. If we will allow same naming we will probably enormously stress glossary integrity. <> use cases 2 and 3 I agree that Categories are needed to give more control over terms organization but I think I need a bit more thinking if categories should help in creating hierarchies. It might be the case but then we should allow terms to only be leaves and every kind of grouping should be done via category. This would mean that categories should also have classifiers. <> use case 7 Do you mean collections ? <> use case 11 this sounds a bit too high level and would probably be nice to describe it in more details <> I'm explicitly missing two things: 1. ability to inherit classifiers <> 2. are there any models between terms and assets or is it only about term to asset ? we might want to include couple of levels of models (like LDM and/or PDM for particular technology) at least one is already there - by connecting terms to other terms we are creating concepts which should be visualized in some way for easier navigation <> ad. discussion point on p. 5 yes, that's how I also see it - Taxonomy is the name of the hierarchy of Glossary Categories but does this mean that Taxonomy is a name of Glossary instance ?> ad. Glossary Terms and Glossary Categories discussion point - can there be a term without Category ? if not will there always be at least one prime category for each Glossary ? if yes what is the difference then between Glossary and prime category ? is there any at all ? <> point for discussion - should it be allowed that term from one Glossary is inside Category from another Glossary ? I think we should not allow this kind of situations as those increase the risk of loosing integrity for particular Glossary. <> I'd say that there should be a copy of that term done to the other Glossary with some kind of a marker "inspired by". Otherwise we will create tight connection between two Glossaries and their maintenance will be more difficult (e.g. upgrades). <> ad. Glossary Term identification and names Glossary Term names might not be unique in a Glossary. For example, there could be 2 definitions of customer. - just NO :-) <> "we do not allow 2 Glossary Terms of the same name inheriting from a parent Glossary Term" - so we do allow or not ? or I missed something ? I need an example for this one to properly understand it. ad. Glossary Term context I'd like to create clear distinction between what is here meant by context and the term business context (being a term to term relations that create business context) - I just don't like using word "context" for both. <> ad. page 9, example In general I do agree with the line of thinking but I have a question: both customer and attributes are terms right ? if so then is "has-a" relationship the best one to do term-to-term assignments ? <> ad. Owning relationships this "Concept Glossary Terms own Attribute Glossary Terms." I've some doubts about (see above remark for page 9, example) I not saying not go there, I just want to explore it more to understand it better <> ad. Discussion point – maybe we should consider defining the Glossary Term attributes using the type system rather than relationships - yes we should ad. Discussion point: we could add homophones as well – if there was a need. I don't think there is a need now to do that. <> ad. Discussion point preferred-term attribute could be stored in the entity, AtlasObjectId or classification. I suggest storing it in the entity. I agree. ad. Discussion point: We will enable collection types to be created. Additionally, we may want to consider including a Collection type that has one attribute called contents with multiple values of the top-level type. Do you mean nesting Collections ? <> ad. Discussion point Introduction of bidirectional relationships, could be done separately from the Glossary enhancement. We may take a step-by-step approach but I'd say we need this from nearly the very beginning. <> ad. Discussion point: We may wish to take a more revolutionary approach and allow relationships to be defined as top level a
[jira] [Commented] (ATLAS-1410) V2 Glossary API
[
https://issues.apache.org/jira/browse/ATLAS-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15928083#comment-15928083
]
David Radley commented on ATLAS-1410:
-
Many thanks [~jonesn].
Responding to your comments for pages
6 : the intent is that a term is owned by one glossary but can be categorized
by categories from any glossary. Do you think I need to be more explicit in the
text about this?
7 : Very valid concerns relating to globalization, which I suggest we deal with
this separately, as per my exclusions at the top of the document.
We have talked of display attributes on the dev list. I have not looked into
TitanDbs encoding, whether this is effected by which store is used, whether
String data type in Titan supports unicode or UTF-8 and how this fits with
indexes.
p8 - in figure 3 "hive column" is meant to be an instance - so could be worth
using an example like "employee salary" or similar to avoid confusion with type
definitions. <>Also on this page it would be worth comparing to
the v1 implementation. The association there between the column (entity) & term
(trait) is the trait instance, which also carries additional information -
parameters. That’s how we might capture the level of SPI, whilst I think with
this new design that is done through the hierarchy of glossary terms <>. An example may help? or just a link to page 16. Question
for other reviewers - is this sufficient (I think it's simpler, but do we lose
additional attributes?) <>
13 : yes there is scope to add new semantics relationships.
I agree on your search comment
16/17 agreed.
On Ranger Tag sync. I am suggesting we continue to expose classifications as
tags. Now V2 Classifications are enhanced by
* having a guid (as the name cannot be relied upon to be unique)
* having an associated Glossary Terms, including the classifying term.
I hope this is sufficient to meet the needs of tag sync; or do you think more
is required?
> V2 Glossary API
> ---
>
> Key: ATLAS-1410
> URL: https://issues.apache.org/jira/browse/ATLAS-1410
> Project: Atlas
> Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: Atlas Glossary V2 proposal v1.0.pdf, Atlas Glossary V2
> proposal v1.1.pdf
>
>
> The BaseResourceDefinition uses the AttributeDefintion class from typesystem.
> There are newer more funcitonal versions of this capability in the atlas-intg
> project. This Jira is changing over the glossary implementation to the newer
> entity / type classes.
> Instread of the instanceProperties and collectionProperties in the
> BaseResourceDefintions we should use something in this sort of style :
> "
> AtlasEntityDef deptTypeDef =
> AtlasTypeUtil.createClassTypeDef(DEPARTMENT_TYPE,
> "Department"+_description, ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> new AtlasAttributeDef("employees",
> String.format("array<%s>", "Person"), true,
> AtlasAttributeDef.Cardinality.SINGLE, 0, 1,
> false, false,
>
> Collections.emptyList()));
> AtlasEntityDef personTypeDef =
> AtlasTypeUtil.createClassTypeDef("Person", "Person"+_description,
> ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> AtlasTypeUtil.createOptionalAttrDef("address", "Address"),
> AtlasTypeUtil.createOptionalAttrDef("birthday", "date"),
> AtlasTypeUtil.createOptionalAttrDef("hasPets", "boolean"),
> AtlasTypeUtil.createOptionalAttrDef("numberOfCars", "byte"),
> AtlasTypeUtil.createOptionalAttrDef("houseNumber", "short"),
> AtlasTypeUtil.createOptionalAttrDef("carMileage", "int"),
> AtlasTypeUtil.createOptionalAttrDef("age", "float"),
> "
> For the parent child relationships with glossary categories and terms we
> should be able to have the type system manage edge deletion. As part of this,
> we will need to investigate whether we could get rid of the disconnect and
> connect methods added in ATLAS-1186
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (ATLAS-1410) V2 Glossary API
[ https://issues.apache.org/jira/browse/ATLAS-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905259#comment-15905259 ] David Radley commented on ATLAS-1410: - Responses to comments Page numbers would help to tie these comments to the document. <> Page 2 - Asset type - defined in terms of itself. How are they used? or is this not relevant to this paper? <> Page 2 - Why do we need to know about V1 and V2? I think it is because the current interfaces works with V1 and the new one will work with V2 - it would be helpful to state this explicitly. <> Page 4 - bullets 4-5 - has-a and is-a relationships are semantic relationships. <> Page 4 - missing from list - ability to associate a semantic meaning to a classification (v2), trait (v1)? <> Page 4 - Missing from the list - "typed-by" relationship to associate terms that include meaning in context with terms that describe more pure objects. For example Home Address is typed by Address.<> Page 5 - Figure 1 - I am not comfortable with terms being owned by categories. I think each terms should be owned by a glossary and linked into 0, 1 or more categories as appropriate. This creates a much simpler deletion rule for the API/End user - particularly when you look at Figure 2 where terms are owned by multiple categories. IE, delete term from its glossary and it is deleted. In the proposed design, it raises such questions as "Is the term deleted when unlinked from all categories - or the first category it is linked to?" <> Page 6 - Figure 3 - I need more detail to understand the "classifies" relationship and how it relates to a classification. It seems redundant. Would you not relate a term to a classification which is in itself semantically classified by its definition term? Page 6 - Bullet 6) - What is the alternative to using Gremlin queries? <> Page 6 - Bullet 7) - is this an incomplete sentence - or does the paragraph that follows supposed to be a nested bullet list? Assuming it is a follow on point. My confusion is that I do not understand why the term/category hierarchy is relevant to the enhancement of classifications? The Classification object is defining the type of classification and its meaning is coming from the term? <> Is this suggesting that the relationships between classifications is coming from the term relationships in the same way we do thin in IGC today? <> If so it may help to show an example? <> Page 7 - Figure 4 and 5 - what is the difference between "Classification" and "Classification Relationship"? <> Page 7 - Maybe strange examples - the Glossaries would be for different subject areas - for example, there may be a marketing glossary, a customer care glossary, a banking glossary. These may be used for associating meaning to data assets (ie data assets). there may also be glossaries for different regulations, or standard governance approaches, and these may include terms that can be used to describe classification for data that drive operational governance? <> Page 8 - I am not sure what the proposed enhancements are - it just seems to list the problems with the current model. All relationships in metadata are bi-directional. It should be the default. This mechanism seems complicated. Really need to define relationships independent of entities so we can define attributes on these relationships. The Classification is actually an example of an independently defined relationship that includes the GUID of the 2 entities it connects. This should be the common style of relationship. <> Page 9 - on discussion point - a Taxonomy is a hierarchy of categories that the terms are placed in - I thought this was included in the proposal and we do need this for organising terms so that people can find them - and the category hierarchies (taxonomies) help to provide context to terms too. Also, the semantic relationships discussed would mean we could support a simple ontology. <> Page 9 - Fully-qualified name - What a grandparent or parent term? What does a fully qualified name mean and when is it used? The unique name is its GUID. Its path name (there may be many) is the navigation to the term through the category hierarchies. <> Page 9 - why do Atlas terms need to follow the schema in defined at this link - https://www.ibm.com/support/knowledgecenter/en/SSN364_8.8.0/com.ibm.ima.using/comp/vocab/terms_prop.html? it seem to imply a lifecycle which is not included in this proposal and a very specific modelling of the IBM industry models that have mandatory fields that are not always applicable to all glossaries. I think this doc should describe the schema of the glossary term explicitly and explain the fields.<> page 10 - Figure 7 shows the navigation relationships and 1 way. We need to be able to navigate from the hive table to its classification to support the GAF. <> Page 11 - Figure 8 - Atlas entities
[jira] [Updated] (ATLAS-1410) V2 Glossary API
[
https://issues.apache.org/jira/browse/ATLAS-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Radley updated ATLAS-1410:
Attachment: Atlas Glossary V2 proposal v1.1.pdf
> V2 Glossary API
> ---
>
> Key: ATLAS-1410
> URL: https://issues.apache.org/jira/browse/ATLAS-1410
> Project: Atlas
> Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: Atlas Glossary V2 proposal v1.0.pdf, Atlas Glossary V2
> proposal v1.1.pdf
>
>
> The BaseResourceDefinition uses the AttributeDefintion class from typesystem.
> There are newer more funcitonal versions of this capability in the atlas-intg
> project. This Jira is changing over the glossary implementation to the newer
> entity / type classes.
> Instread of the instanceProperties and collectionProperties in the
> BaseResourceDefintions we should use something in this sort of style :
> "
> AtlasEntityDef deptTypeDef =
> AtlasTypeUtil.createClassTypeDef(DEPARTMENT_TYPE,
> "Department"+_description, ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> new AtlasAttributeDef("employees",
> String.format("array<%s>", "Person"), true,
> AtlasAttributeDef.Cardinality.SINGLE, 0, 1,
> false, false,
>
> Collections.emptyList()));
> AtlasEntityDef personTypeDef =
> AtlasTypeUtil.createClassTypeDef("Person", "Person"+_description,
> ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> AtlasTypeUtil.createOptionalAttrDef("address", "Address"),
> AtlasTypeUtil.createOptionalAttrDef("birthday", "date"),
> AtlasTypeUtil.createOptionalAttrDef("hasPets", "boolean"),
> AtlasTypeUtil.createOptionalAttrDef("numberOfCars", "byte"),
> AtlasTypeUtil.createOptionalAttrDef("houseNumber", "short"),
> AtlasTypeUtil.createOptionalAttrDef("carMileage", "int"),
> AtlasTypeUtil.createOptionalAttrDef("age", "float"),
> "
> For the parent child relationships with glossary categories and terms we
> should be able to have the type system manage edge deletion. As part of this,
> we will need to investigate whether we could get rid of the disconnect and
> connect methods added in ATLAS-1186
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (ATLAS-1187) Rename Trait to Classification
[ https://issues.apache.org/jira/browse/ATLAS-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley resolved ATLAS-1187. - Resolution: Fixed Fixed in V2 API > Rename Trait to Classification > -- > > Key: ATLAS-1187 > URL: https://issues.apache.org/jira/browse/ATLAS-1187 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley > > Classification would be more descriptive of the capability currently called > trait in Atlas (which appear to have come from Ranger terminology). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (ATLAS-1344) Allow types to be created that support parent child hierarchies.
[ https://issues.apache.org/jira/browse/ATLAS-1344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley resolved ATLAS-1344. - Resolution: Duplicate > Allow types to be created that support parent child hierarchies. > > > Key: ATLAS-1344 > URL: https://issues.apache.org/jira/browse/ATLAS-1344 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > > enhance the Type system to allow entities to be created in parent child > hierarchies. > I notice that AttributeDefinition currently only support COLLECTION, SET , > OPTIONAL and REQUIRED. There is no collection with a lower bound of 0- which > would be required for a parent child type relationship. > Also it seems to me that it would be ideal to create an attribute with the > guid of the parent / child and then derive the other one from the edge. I may > have misunderstood but reverseAttributeName seems to use the name of the > attribute not the guid. > I suggest that iscomposite true and false is supported. > When we have this in place, we could use this mechanism for Glossary > Categories > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (ATLAS-1458) 2nd loop Loop in Atlas using V2 entity API CONSTRAINT_TYPE_MAPPED_FROM_REF pointing to its own type
[ https://issues.apache.org/jira/browse/ATLAS-1458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley resolved ATLAS-1458. - Resolution: Duplicate > 2nd loop Loop in Atlas using V2 entity API CONSTRAINT_TYPE_MAPPED_FROM_REF > pointing to its own type > --- > > Key: ATLAS-1458 > URL: https://issues.apache.org/jira/browse/ATLAS-1458 > Project: Atlas > Issue Type: Bug >Reporter: David Radley >Assignee: David Radley > Attachments: 01-test.json, 02-test.json, 03-test.json, > QuickStartNoFKUpdatechildafterparentloop.java > > > similar to 1452. I am trying to knit a parent to a child entity. Both the > parent and child are the same type. I have supplied the app that shows this > loop. This loop is of the form: > xception in thread "main" java.lang.StackOverflowError > at sun.util.calendar.ZoneInfo.getOffsets(ZoneInfo.java:271) > at sun.util.calendar.ZoneInfo.getOffsets(ZoneInfo.java:248) > at > java.util.GregorianCalendar.computeFields(GregorianCalendar.java:2276) > at > java.util.GregorianCalendar.computeFields(GregorianCalendar.java:2248) > at java.util.Calendar.setTimeInMillis(Calendar.java:1140) > at java.util.Calendar.setTime(Calendar.java:1106) > at java.text.SimpleDateFormat.format(SimpleDateFormat.java:955) > at java.text.SimpleDateFormat.format(SimpleDateFormat.java:948) > at java.text.DateFormat.format(DateFormat.java:336) > at > org.apache.atlas.model.instance.AtlasStruct.dumpDateField(AtlasStruct.java:257) > at > org.apache.atlas.model.instance.AtlasEntity.toString(AtlasEntity.java:176) > at > org.apache.atlas.model.instance.AtlasEntity.toString(AtlasEntity.java:209) > at java.lang.String.valueOf(String.java:2849) > at java.lang.StringBuilder.append(StringBuilder.java:128) > at java.util.AbstractCollection.toString(AbstractCollection.java:458) > at java.lang.String.valueOf(String.java:2849) > at java.lang.StringBuilder.append(StringBuilder.java:128) > at > org.apache.atlas.model.instance.AtlasStruct.dumpObjects(AtlasStruct.java:243) > at > org.apache.atlas.model.instance.AtlasStruct.toString(AtlasStruct.java:137) > at > org.apache.atlas.model.instance.AtlasEntity.toString(AtlasEntity.java:180) > at > org.apache.atlas.model.instance.AtlasEntity.toString(AtlasEntity.java:209) > at java.lang.String.valueOf(String.java:2849) > at java.lang.StringBuilder.append(StringBuilder.java:128) > at > org.apache.atlas.model.instance.AtlasStruct.dumpObjects(AtlasStruct.java:243) > at > org.apache.atlas.model.instance.AtlasStruct.toString(AtlasStruct.java:137) > at > org.apache.atlas.model.instance.AtlasEntity.toString(AtlasEntity.java:180) > at > org.apache.atlas.model.instance.AtlasEntity.toString(AtlasEntity.java:209) > at java.lang.String.valueOf(String.java:2849) > at java.lang.StringBuilder.append(StringBuilder.java:128) > at java.util.AbstractCollection.toString(AbstractCollection.java:458) > at java.lang.String.valueOf(String.java:2849) > at java.lang.StringBuilder.append(StringBuilder.java:128) > at > org.apache.atlas.model.instance.AtlasStruct.dumpObjects(AtlasStruct.java:243) > at > org.apache.atlas.model.instance.AtlasStruct.toString(AtlasStruct.java:137) > at > org.apache.atlas.model.instance.AtlasEntity.toString(AtlasEntity.java:180) > at > org.apache.atlas.model.instance.AtlasEntity.toString(AtlasEntity.java:209) > at java.lang.String.valueOf(String.java:2849) > at java.lang.StringBuilder.append(StringBuilder.java:128) > at > org.apache.atlas.model.instance.AtlasStruct.dumpObjects(AtlasStruct.java:243) > at > org.apache.atlas.model.instance.AtlasStruct.toString(AtlasStruct.java:137) > at > org.apache.atlas.model.instance.AtlasEntity.toString(AtlasEntity.java:180) > at > org.apache.atlas.model.instance.AtlasEntity.toString(AtlasEntity.java:209) > at java.lang.String.valueOf(String.java:2849) > at java.lang.StringBuilder.append(StringBuilder.java:128) > at java.util.AbstractCollection.toString(AbstractCollection.java:458) > at java.lang.String.valueOf(String.java:2849) > at java.lang.StringBuilder.append(StringBuilder.java:128) > at > org.apache.atlas.model.instance.AtlasStruct.dumpObjects(AtlasStruct.java:243) > at > org.apache.atlas.model.instance.AtlasStruct.toString(AtlasStruct.java:137) > at > org.apache.atlas.model.instance.AtlasEntity.toString(AtlasEntity.java:180) > at > org.apache.atlas.model.instance.AtlasEntity.toString(AtlasEntity.java:209) > at java.lang.String.valueOf(String.java:2849) > at java.la
[jira] [Resolved] (ATLAS-1457) Loop in Atlas using V2 entity API CONSTRAINT_TYPE_MAPPED_FROM_REF pointing to its own type
[ https://issues.apache.org/jira/browse/ATLAS-1457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley resolved ATLAS-1457. - Resolution: Duplicate > Loop in Atlas using V2 entity API CONSTRAINT_TYPE_MAPPED_FROM_REF pointing to > its own type > -- > > Key: ATLAS-1457 > URL: https://issues.apache.org/jira/browse/ATLAS-1457 > Project: Atlas > Issue Type: Bug >Reporter: David Radley > Attachments: QuickStartLoop.java > > > I am attempting to model parent child relationships with the V2 Entity API. > I define a type that has > - a single optional parent attribute of our type. > - a children optional List CONSTRAINT_TYPE_MAPPED_FROM_REF pointing to its > own type and a parent attribute. > When I create entities and try to knit them together as parent and child > Atlas loops. I have included a test program I used to create this error. > The console shows the loop: > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToClassReference(GraphToTypedInstanceMapper.java:191) > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToCollectionEntry(GraphToTypedInstanceMapper.java:248) > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToArrayInstance(GraphToTypedInstanceMapper.java:222) > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToAttribute(GraphToTypedInstanceMapper.java:142) > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToInstance(GraphToTypedInstanceMapper.java:116) > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapGraphToTypedInstance(GraphToTypedInstanceMapper.java:93) > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToClassReference(GraphToTypedInstanceMapper.java:191) > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToCollectionEntry(GraphToTypedInstanceMapper.java:248) > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToArrayInstance(GraphToTypedInstanceMapper.java:222) > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToAttribute(GraphToTypedInstanceMapper.java:142) > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToInstance(GraphToTypedInstanceMapper.java:116) > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapGraphToTypedInstance(GraphToTypedInstanceMapper.java:93) > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToClassReference(GraphToTypedInstanceMapper.java:191) > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToCollectionEntry(GraphToTypedInstanceMapper.java:248) > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToArrayInstance(GraphToTypedInstanceMapper.java:222) > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToAttribute(GraphToTypedInstanceMapper.java:142) > at > org.apache.atlas.repository.graph.GraphToTypedInstanceMapper.mapVertexToInstance(GraphToTypedInstanceMapper.java:116) > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (ATLAS-1459) Atlas V2 types to manage bidirectional relationships
[ https://issues.apache.org/jira/browse/ATLAS-1459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley resolved ATLAS-1459. - Resolution: Duplicate Fix Version/s: 0.8-incubating > Atlas V2 types to manage bidirectional relationships > - > > Key: ATLAS-1459 > URL: https://issues.apache.org/jira/browse/ATLAS-1459 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley > Fix For: 0.8-incubating > > > Currently Atlas can use mappedFromRef constraint and the refAttribute param > in an attribute to point to a child type that points back to the parent . > See 0030-hive-model.json for an example. > In an operational environment where we want to update these attributes - we > want the corresponding attribute (related using the constraints) to be > updated. > I suggest that we add an inSyncRefAttribute in addition to the existing > refAttribute. If inSyncRefAttribute constraint is specified then the > attribute pointed to is kept in sync with our attribute. On creates, updates > and deletes. This will be supported for List and non List attributes. > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1609) Atlas V2 API should error if it does not recognise part of the supplied structure.
David Radley created ATLAS-1609: --- Summary: Atlas V2 API should error if it does not recognise part of the supplied structure. Key: ATLAS-1609 URL: https://issues.apache.org/jira/browse/ATLAS-1609 Project: Atlas Issue Type: Bug Reporter: David Radley If an API is called misspells as parameter in a structure, the request is accepted and the caller is unaware, so subsequent behaviour is not as expected. For example: - on an attribute name when creating an entity - a keyword like constraint or Attributes -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1401) Document in detail how to set up Eclipse for Atlas dev environment.
[ https://issues.apache.org/jira/browse/ATLAS-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876019#comment-15876019 ] David Radley commented on ATLAS-1401: - This Eclipse document assumes that your Atlas environment is already setup; it is not talking about how to populate the Alas repository. I have set up Atlas using the 0.8 codebase - the Eclipse instructions are talking about this Atlas version. HDP uses Atlas 0.7. I have not setup Hive with my development environment. It looks like you run an import script (import-hive.sh) to bring in the Hive metadata- as documented here http://atlas.incubator.apache.org/Bridge-Hive.html. all the best, David > Document in detail how to set up Eclipse for Atlas dev environment. > --- > > Key: ATLAS-1401 > URL: https://issues.apache.org/jira/browse/ATLAS-1401 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Fix For: 0.8-incubating > > Attachments: rb54899(2).patch, rb54899(3).patch, rb54899.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1370) Generate API document with enunciate/swagger
[ https://issues.apache.org/jira/browse/ATLAS-1370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871740#comment-15871740 ] David Radley commented on ATLAS-1370: - [~apoorvnaik] This looks like a good change. Can we you check in the output into Git so that we can have V2 Rest API documented on the web site as well as the V1. I think you are suggesting that we should move up to the latest version of enunciate so we can put this in the 0.8 build and generate the API docs as part of the build - did I understand you correctly? > Generate API document with enunciate/swagger > > > Key: ATLAS-1370 > URL: https://issues.apache.org/jira/browse/ATLAS-1370 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.8-incubating >Reporter: Apoorv Naik >Assignee: Vimal Sharma > Fix For: 0.8-incubating > > > Currently there's no UI that lists down all the API implemented within Atlas. > The tech docs are a good source of understanding Atlas APIs but I feel a > visual tool (something like an built-in postman client) would be a good > addition for developers to setup atlas and start playing around with the APIs. > Swagger has really good capabilities and there are a couple of other tools > too. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1551) Auto reverse reference update in V1/legacy code path
[ https://issues.apache.org/jira/browse/ATLAS-1551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871735#comment-15871735 ] David Radley commented on ATLAS-1551: - [~dkantor] I think this is a great change to get in. Sorry for the late comments. I wondered: - when you add a reverse edge - and want to delete the existing one , we coudl check to see if it is already soft deleted - is it possible to manually update the reverse reference so the reference get out of sync? > Auto reverse reference update in V1/legacy code path > > > Key: ATLAS-1551 > URL: https://issues.apache.org/jira/browse/ATLAS-1551 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.8-incubating >Reporter: David Kantor >Assignee: David Kantor > Attachments: rb56287.patch > > > Review board request for V1 changes: https://reviews.apache.org/r/56287/ -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1566) Use Jackson ObjectMapper for serialization and deserailization
[ https://issues.apache.org/jira/browse/ATLAS-1566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871666#comment-15871666 ] David Radley commented on ATLAS-1566: - I think this makes sense. Just be aware for types that have attributes pointing to themselves like parent child relationships or other loops, we use object_ids. It looks like Jackson does have annotations to help avoid this sort of thing with: @JsonManagedReference and @JsonBackReference; we do not need to use these as we use the object_ids. I suggest including unit tests to test these loop scenarios (1 to 1 and 1 to many; to own type and other types). > Use Jackson ObjectMapper for serialization and deserailization > -- > > Key: ATLAS-1566 > URL: https://issues.apache.org/jira/browse/ATLAS-1566 > Project: Atlas > Issue Type: Improvement >Reporter: Vimal Sharma >Assignee: Vimal Sharma > Fix For: 0.8-incubating > > Attachments: ATLAS-1566.1.patch > > > Currently, GSON is used for serialization and deserialization of AtlasType > POJOs at REST layer. ObjectMapper from Jackson library is used at other > places in the Atlas code. Use ObjectMapper in AtlasType to make the > serialization/deserialization consistent across Atlas code. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1564) EntityResource v1 should start routing its calls to v2 EntityREST
[ https://issues.apache.org/jira/browse/ATLAS-1564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871476#comment-15871476 ] David Radley commented on ATLAS-1564: - Some thoughts: Can I suggest that you do not route the V1 Glossary calls to V2. We have JIRA 1410 for a V2 version of the glossary. At some stage we should deprecate the V1 API; or make a statement to say people should use the V2 APIs. I notice there is code to convert V1 shape entities to V2 shapes. It sounds like to make this change you would need to change the V1 entity shapes in the graph to be V2 then skin the V2 APIs with the V1 JAVA, REST API, notification and search wrappers (did i miss any)? As the bridges have been changed to V2 - I am not sure how many people could stay on V1. I think the effort should focus on migration strategies; is this a requirement raised by an Atlas customer? > EntityResource v1 should start routing its calls to v2 EntityREST > - > > Key: ATLAS-1564 > URL: https://issues.apache.org/jira/browse/ATLAS-1564 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.8-incubating >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian > > With the v2 implementation of EntityREST ready, we must start routing all v1 > entity CRUD calls to start using v2 APIs in EntityREST. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1410) V2 Glossary API
[
https://issues.apache.org/jira/browse/ATLAS-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Radley updated ATLAS-1410:
Attachment: Atlas Glossary V2 proposal v1.0.pdf
Here is a high level design document for the glossary, please feedback whether
you have concerns with this approach.
Detailed feedback on the document is also welcome.
> V2 Glossary API
> ---
>
> Key: ATLAS-1410
> URL: https://issues.apache.org/jira/browse/ATLAS-1410
> Project: Atlas
> Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: Atlas Glossary V2 proposal v1.0.pdf
>
>
> The BaseResourceDefinition uses the AttributeDefintion class from typesystem.
> There are newer more funcitonal versions of this capability in the atlas-intg
> project. This Jira is changing over the glossary implementation to the newer
> entity / type classes.
> Instread of the instanceProperties and collectionProperties in the
> BaseResourceDefintions we should use something in this sort of style :
> "
> AtlasEntityDef deptTypeDef =
> AtlasTypeUtil.createClassTypeDef(DEPARTMENT_TYPE,
> "Department"+_description, ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> new AtlasAttributeDef("employees",
> String.format("array<%s>", "Person"), true,
> AtlasAttributeDef.Cardinality.SINGLE, 0, 1,
> false, false,
>
> Collections.emptyList()));
> AtlasEntityDef personTypeDef =
> AtlasTypeUtil.createClassTypeDef("Person", "Person"+_description,
> ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> AtlasTypeUtil.createOptionalAttrDef("address", "Address"),
> AtlasTypeUtil.createOptionalAttrDef("birthday", "date"),
> AtlasTypeUtil.createOptionalAttrDef("hasPets", "boolean"),
> AtlasTypeUtil.createOptionalAttrDef("numberOfCars", "byte"),
> AtlasTypeUtil.createOptionalAttrDef("houseNumber", "short"),
> AtlasTypeUtil.createOptionalAttrDef("carMileage", "int"),
> AtlasTypeUtil.createOptionalAttrDef("age", "float"),
> "
> For the parent child relationships with glossary categories and terms we
> should be able to have the type system manage edge deletion. As part of this,
> we will need to investigate whether we could get rid of the disconnect and
> connect methods added in ATLAS-1186
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (ATLAS-1410) V2 Glossary API
[
https://issues.apache.org/jira/browse/ATLAS-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Radley reassigned ATLAS-1410:
---
Assignee: David Radley
> V2 Glossary API
> ---
>
> Key: ATLAS-1410
> URL: https://issues.apache.org/jira/browse/ATLAS-1410
> Project: Atlas
> Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
>
> The BaseResourceDefinition uses the AttributeDefintion class from typesystem.
> There are newer more funcitonal versions of this capability in the atlas-intg
> project. This Jira is changing over the glossary implementation to the newer
> entity / type classes.
> Instread of the instanceProperties and collectionProperties in the
> BaseResourceDefintions we should use something in this sort of style :
> "
> AtlasEntityDef deptTypeDef =
> AtlasTypeUtil.createClassTypeDef(DEPARTMENT_TYPE,
> "Department"+_description, ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> new AtlasAttributeDef("employees",
> String.format("array<%s>", "Person"), true,
> AtlasAttributeDef.Cardinality.SINGLE, 0, 1,
> false, false,
>
> Collections.emptyList()));
> AtlasEntityDef personTypeDef =
> AtlasTypeUtil.createClassTypeDef("Person", "Person"+_description,
> ImmutableSet.of(),
> AtlasTypeUtil.createRequiredAttrDef("name", "string"),
> AtlasTypeUtil.createOptionalAttrDef("address", "Address"),
> AtlasTypeUtil.createOptionalAttrDef("birthday", "date"),
> AtlasTypeUtil.createOptionalAttrDef("hasPets", "boolean"),
> AtlasTypeUtil.createOptionalAttrDef("numberOfCars", "byte"),
> AtlasTypeUtil.createOptionalAttrDef("houseNumber", "short"),
> AtlasTypeUtil.createOptionalAttrDef("carMileage", "int"),
> AtlasTypeUtil.createOptionalAttrDef("age", "float"),
> "
> For the parent child relationships with glossary categories and terms we
> should be able to have the type system manage edge deletion. As part of this,
> we will need to investigate whether we could get rid of the disconnect and
> connect methods added in ATLAS-1186
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (ATLAS-328) UI: Swagger API Documentation support
[ https://issues.apache.org/jira/browse/ATLAS-328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15860972#comment-15860972 ] David Radley commented on ATLAS-328: Hi Nigel, I had a quick look. It looks like https://github.com/sdaschner/jaxrs-analyzer-maven-plugin could be interesting. This is analysing jaxrs to produce the Swagger json. I am not sure if we can use this code. It seems the more standard way would be to put Swagger annotations where we need them in the code. Either way we would need a servlet to display the Swagger. > UI: Swagger API Documentation support > - > > Key: ATLAS-328 > URL: https://issues.apache.org/jira/browse/ATLAS-328 > Project: Atlas > Issue Type: Task >Reporter: roshni gadiyar >Assignee: Jean-Baptiste Onofré >Priority: Minor > Labels: Atlas-UI > > Swagger API Documentation support. > https://github.com/swagger-api/swagger-samples/tree/master/java/java-jersey2-guice -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1401) Document in detail how to set up Eclipse for Atlas dev environment.
[ https://issues.apache.org/jira/browse/ATLAS-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1401: Attachment: rb54899(3).patch > Document in detail how to set up Eclipse for Atlas dev environment. > --- > > Key: ATLAS-1401 > URL: https://issues.apache.org/jira/browse/ATLAS-1401 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Attachments: rb54899(2).patch, rb54899(3).patch, rb54899.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1401) Document in detail how to set up Eclipse for Atlas dev environment.
[ https://issues.apache.org/jira/browse/ATLAS-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1401: Attachment: rb54899(2).patch > Document in detail how to set up Eclipse for Atlas dev environment. > --- > > Key: ATLAS-1401 > URL: https://issues.apache.org/jira/browse/ATLAS-1401 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Attachments: rb54899(2).patch, rb54899.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1401) Document in detail how to set up Eclipse for Atlas dev environment.
[ https://issues.apache.org/jira/browse/ATLAS-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1401: Attachment: (was: rb54899(1).patch) > Document in detail how to set up Eclipse for Atlas dev environment. > --- > > Key: ATLAS-1401 > URL: https://issues.apache.org/jira/browse/ATLAS-1401 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Attachments: rb54899.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1401) Document in detail how to set up Eclipse for Atlas dev environment.
[ https://issues.apache.org/jira/browse/ATLAS-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15858092#comment-15858092 ] David Radley commented on ATLAS-1401: - pointer to the review https://reviews.apache.org/r/54899/ > Document in detail how to set up Eclipse for Atlas dev environment. > --- > > Key: ATLAS-1401 > URL: https://issues.apache.org/jira/browse/ATLAS-1401 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Attachments: rb54899(1).patch, rb54899.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1401) Document in detail how to set up Eclipse for Atlas dev environment.
[ https://issues.apache.org/jira/browse/ATLAS-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Radley updated ATLAS-1401: Attachment: rb54899(1).patch > Document in detail how to set up Eclipse for Atlas dev environment. > --- > > Key: ATLAS-1401 > URL: https://issues.apache.org/jira/browse/ATLAS-1401 > Project: Atlas > Issue Type: Improvement >Reporter: David Radley >Assignee: David Radley > Attachments: rb54899(1).patch, rb54899.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (ATLAS-1526) update AtlasAttribute with methods to derive legacy flags isComposite/reverseAttribute
[ https://issues.apache.org/jira/browse/ATLAS-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15857765#comment-15857765 ] David Radley commented on ATLAS-1526: - [~madhan.neethiraj] I suggest that this is put out in a small design document, spelling out the V1 situations, the recent changes and the new proposal. In that way, the wider community could be more readily engaged to feedback. I think the way we implement relationships is key basic capability that deserves this level of scrutiny, all the best, David. > update AtlasAttribute with methods to derive legacy flags > isComposite/reverseAttribute > -- > > Key: ATLAS-1526 > URL: https://issues.apache.org/jira/browse/ATLAS-1526 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 0.8-incubating >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj > Fix For: 0.8-incubating > > Attachments: ATLAS-1526.3.patch, ATLAS-1526.4.patch, > ATLAS-1526.patch, Atlas-legacy-to-new-ModelMapping.pdf > > > The logic to map legacy attribute flags isComposite/reverseAttribute to its > equivalent constraints in the new data model is spread across multiple > locations. Consolidating this logic to a single class will make it easier to > manage. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
