[jira] [Commented] (ATLAS-2215) UI: Editing array of Structs behave as array of entity references
[ https://issues.apache.org/jira/browse/ATLAS-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995309#comment-16995309 ] Logan Rickert commented on ATLAS-2215: -- [~madhan] I searched through the backlog and found this ticket https://issues.apache.org/jira/browse/ATLAS-2763 I will link your comment in the comments of the ticket. > UI: Editing array of Structs behave as array of entity references > - > > Key: ATLAS-2215 > URL: https://issues.apache.org/jira/browse/ATLAS-2215 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Priority: Major > > Reproduction steps: > * create a struct with a set of attributes > * create an entity type with an attribute defined as an array of the > previously defined struct > * Attempt to create an instance of the entity and set the struct value. It > will behave as if the type is an array of entities and not a free form field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-3560) Not checking for relationshipDefs attributes when defining entities
Logan Rickert created ATLAS-3560: Summary: Not checking for relationshipDefs attributes when defining entities Key: ATLAS-3560 URL: https://issues.apache.org/jira/browse/ATLAS-3560 Project: Atlas Issue Type: Bug Components: atlas-webui Affects Versions: 2.0.0, 1.0.0, 2.1.0 Reporter: Logan Rickert When creating an entity definition, it is possible to specify an attribute key that will later be overwritten/shadowed through relationshipDef. See [~madhan]'s comment for more information: https://issues.apache.org/jira/browse/ATLAS-2215?focusedCommentId=16995162=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16995162 Related: https://issues.apache.org/jira/browse/ATLAS-2763 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (ATLAS-2215) UI: Editing array of Structs behave as array of entity references
[ https://issues.apache.org/jira/browse/ATLAS-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995309#comment-16995309 ] Logan Rickert edited comment on ATLAS-2215 at 12/13/19 3:37 AM: [~madhan] I searched through the backlog and found this ticket https://issues.apache.org/jira/browse/ATLAS-2763 I will link your comment in the comments of the ticket. Created a ticket to track this issue: https://issues.apache.org/jira/browse/ATLAS-3560 was (Author: logan.rickert): [~madhan] I searched through the backlog and found this ticket https://issues.apache.org/jira/browse/ATLAS-2763 I will link your comment in the comments of the ticket. Do you think there should still be a separate issue made or should ATLAS-2763 work? > UI: Editing array of Structs behave as array of entity references > - > > Key: ATLAS-2215 > URL: https://issues.apache.org/jira/browse/ATLAS-2215 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Priority: Major > > Reproduction steps: > * create a struct with a set of attributes > * create an entity type with an attribute defined as an array of the > previously defined struct > * Attempt to create an instance of the entity and set the struct value. It > will behave as if the type is an array of entities and not a free form field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-2215) UI: Editing array of Structs behave as array of entity references
[ https://issues.apache.org/jira/browse/ATLAS-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995264#comment-16995264 ] Madhan Neethiraj commented on ATLAS-2215: - [~logan.rickert] - we should have a JIRA to track this issue (of relationship attribute overwritting/shadowing another attribute). Can you please file one? > UI: Editing array of Structs behave as array of entity references > - > > Key: ATLAS-2215 > URL: https://issues.apache.org/jira/browse/ATLAS-2215 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Priority: Major > > Reproduction steps: > * create a struct with a set of attributes > * create an entity type with an attribute defined as an array of the > previously defined struct > * Attempt to create an instance of the entity and set the struct value. It > will behave as if the type is an array of entities and not a free form field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (ATLAS-2215) UI: Editing array of Structs behave as array of entity references
[
https://issues.apache.org/jira/browse/ATLAS-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995162#comment-16995162
]
Madhan Neethiraj edited comment on ATLAS-2215 at 12/13/19 1:17 AM:
---
[~logan.rickert] - an attribute named {{schema}} is _injected_ into entity-type
{{DataSet}}, by relationship-type {{avro_schema_associatedEntities}} in
1000-Hadoop/1065-avro_model.json.
The attribute {{DataSetTestC.schema}} is overwritten/shadowed by this
relationship attribute, hence the failure. Ideally, creation of DataSetTestC
should have failed due to this duplicate attribute name.
{noformat}
{
"name": "avro_schema_associatedEntities",
"serviceType": "avro",
"typeVersion": "1.0",
"relationshipCategory": "ASSOCIATION",
"relationshipLabel": " __avro_schema.associatedEntities",
"endDef1": {
"type": "avro_schema",
"name": "associatedEntities",
"isContainer": false,
"cardinality": "SET",
"isLegacyAttribute": true
},
"endDef2": {
"type":"DataSet",
"name":"schema",
"isContainer": false,
"cardinality": "SET"
}
},
{noformat}
was (Author: madhan.neethiraj):
[~logan.rickert] - an attribute named {{schema}} is _injected_ into entity-type
{{DataSet}}, by following relationship-type {{avro_schema_associatedEntities}}
in 1000-Hadoop/1065-avro_model.json.
The attribute {{DataSetTestC.schema}} in is overwritten/shadowed by this
relationship attribute, hence the failure. Ideally, creation of DataSetTestC
should have failed due to this duplicate attribute name.
{noformat}
{
"name": "avro_schema_associatedEntities",
"serviceType": "avro",
"typeVersion": "1.0",
"relationshipCategory": "ASSOCIATION",
"relationshipLabel": " __avro_schema.associatedEntities",
"endDef1": {
"type": "avro_schema",
"name": "associatedEntities",
"isContainer": false,
"cardinality": "SET",
"isLegacyAttribute": true
},
"endDef2": {
"type":"DataSet",
"name":"schema",
"isContainer": false,
"cardinality": "SET"
}
},
{noformat}
> UI: Editing array of Structs behave as array of entity references
> -
>
> Key: ATLAS-2215
> URL: https://issues.apache.org/jira/browse/ATLAS-2215
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
>Priority: Major
>
> Reproduction steps:
> * create a struct with a set of attributes
> * create an entity type with an attribute defined as an array of the
> previously defined struct
> * Attempt to create an instance of the entity and set the struct value. It
> will behave as if the type is an array of entities and not a free form field.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ATLAS-2763) Using entity attribute name `schema` causes errors
[ https://issues.apache.org/jira/browse/ATLAS-2763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995313#comment-16995313 ] Logan Rickert commented on ATLAS-2763: -- I believe that this issue is the cause of ATLAS-2215. [~madhan] gave a good explanation of the cause here: https://issues.apache.org/jira/browse/ATLAS-2215?focusedCommentId=16995162=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16995162 > Using entity attribute name `schema` causes errors > --- > > Key: ATLAS-2763 > URL: https://issues.apache.org/jira/browse/ATLAS-2763 > Project: Atlas > Issue Type: Bug >Affects Versions: 1.0.0 >Reporter: Connie Chen >Priority: Major > > It seems like there's something in `AtlasEntityGraphDiscoveryV2` that is > breaking when we have an entity attribute name called `schema` that is a > reference, and it is not able to use the `AtlasObjectId` that we pass to it > via the v2 client. > Error messages and stacktrace here: > http://mail-archives.apache.org/mod_mbox/atlas-user/201806.mbox/%3CCAKLJkYxqOyuGrgtYqaZcvkEhCKJgTaPaFqDKaYPEmxdZdEUwYg%40mail.gmail.com%3E -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (ATLAS-2215) UI: Editing array of Structs behave as array of entity references
[ https://issues.apache.org/jira/browse/ATLAS-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995309#comment-16995309 ] Logan Rickert edited comment on ATLAS-2215 at 12/13/19 3:01 AM: [~madhan] I searched through the backlog and found this ticket https://issues.apache.org/jira/browse/ATLAS-2763 I will link your comment in the comments of the ticket. Do you think there should still be a separate issue made or should ATLAS-2763 work? was (Author: logan.rickert): [~madhan] I searched through the backlog and found this ticket https://issues.apache.org/jira/browse/ATLAS-2763 I will link your comment in the comments of the ticket. > UI: Editing array of Structs behave as array of entity references > - > > Key: ATLAS-2215 > URL: https://issues.apache.org/jira/browse/ATLAS-2215 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Priority: Major > > Reproduction steps: > * create a struct with a set of attributes > * create an entity type with an attribute defined as an array of the > previously defined struct > * Attempt to create an instance of the entity and set the struct value. It > will behave as if the type is an array of entities and not a free form field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 71902: ATLAS-3558: Improve lineage performance using in-memory traversal
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71902/ --- (Updated Dec. 12, 2019, 3:20 p.m.) Review request for atlas, Ashutosh Mestry, Aadarsh Jajodia, keval bhatt, Sridhar K, Le Ma, Mandar Ambawane, mayank jain, Nixon Rodrigues, Sameer Shaikh, and Sarath Subramanian. Changes --- addressed review comments. Bugs: ATLAS-3558 https://issues.apache.org/jira/browse/ATLAS-3558 Repository: atlas Description --- Lineage in atlas uses graph query to compute lineage across entities (inputs, outputs or both). Lineage rendering performance have degraded after using janusgraph version 0.4.0 On investigation, lineage graph query initialization and execution using gremlin script engine has been found to be the bottleneck. Alternate in-memory computation of lineage has improved performance by many folds (~90% improvement). This Jira is about adding alternate in-memory computation of lineage. "atlas.use.graph.query.for.lineage" property can be used to toggle between graph query and in-memory computation of lineage. The default option will be in-memory. Diffs (updated) - intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 979bd0ae3 repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java 9a020468d Diff: https://reviews.apache.org/r/71902/diff/2/ Changes: https://reviews.apache.org/r/71902/diff/1-2/ Testing --- Precommit: https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1581/console Manually validated lineage rendering works fine for simple, complex and circular lineages. Thanks, Sarath Subramanian
[jira] [Updated] (ATLAS-3559) Add attribute 'displayName' in Asset typeDef
[ https://issues.apache.org/jira/browse/ATLAS-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil Bonte updated ATLAS-3559: Attachment: (was: ATLAS-3559-Add-attribute-displayName-in-Asset-typeDe.patch) > Add attribute 'displayName' in Asset typeDef > > > Key: ATLAS-3559 > URL: https://issues.apache.org/jira/browse/ATLAS-3559 > Project: Atlas > Issue Type: Improvement >Reporter: Nikhil Bonte >Assignee: Nikhil Bonte >Priority: Major > Attachments: > ATLAS-3559-Add-attribute-displayName-in-Asset-typeDe.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3559) Add attribute 'displayName' in Asset typeDef
[ https://issues.apache.org/jira/browse/ATLAS-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil Bonte updated ATLAS-3559: Attachment: ATLAS-3559-Add-attribute-displayName-in-Asset-typeDe.patch > Add attribute 'displayName' in Asset typeDef > > > Key: ATLAS-3559 > URL: https://issues.apache.org/jira/browse/ATLAS-3559 > Project: Atlas > Issue Type: Improvement >Reporter: Nikhil Bonte >Assignee: Nikhil Bonte >Priority: Major > Attachments: > ATLAS-3559-Add-attribute-displayName-in-Asset-typeDe.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-3559) Add displayName attribute in Asset typeDef
Nikhil Bonte created ATLAS-3559: --- Summary: Add displayName attribute in Asset typeDef Key: ATLAS-3559 URL: https://issues.apache.org/jira/browse/ATLAS-3559 Project: Atlas Issue Type: Improvement Reporter: Nikhil Bonte Assignee: Nikhil Bonte -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3559) Add attribute 'displayName' in Asset typeDef
[ https://issues.apache.org/jira/browse/ATLAS-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil Bonte updated ATLAS-3559: Summary: Add attribute 'displayName' in Asset typeDef (was: Add displayName attribute in Asset typeDef) > Add attribute 'displayName' in Asset typeDef > > > Key: ATLAS-3559 > URL: https://issues.apache.org/jira/browse/ATLAS-3559 > Project: Atlas > Issue Type: Improvement >Reporter: Nikhil Bonte >Assignee: Nikhil Bonte >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-2215) UI: Editing array of Structs behave as array of entity references
[ https://issues.apache.org/jira/browse/ATLAS-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995143#comment-16995143 ] Madhan Neethiraj commented on ATLAS-2215: - [~logan.rickert] - can you please add details of the struct-def, entity-json? These will help in investigating the issue. > UI: Editing array of Structs behave as array of entity references > - > > Key: ATLAS-2215 > URL: https://issues.apache.org/jira/browse/ATLAS-2215 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Priority: Major > > Reproduction steps: > * create a struct with a set of attributes > * create an entity type with an attribute defined as an array of the > previously defined struct > * Attempt to create an instance of the entity and set the struct value. It > will behave as if the type is an array of entities and not a free form field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-2215) UI: Editing array of Structs behave as array of entity references
[
https://issues.apache.org/jira/browse/ATLAS-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995154#comment-16995154
]
Logan Rickert commented on ATLAS-2215:
--
[~madhan] Thank you for replying. I think I have narrowed down the issue some
more while creating some examples for you. I've only been testing with the key
value 'schema' because that is what was used in the only example JSON I could
find for structures with entities. Changing the key value to something other
than 'schema' (in this case 'testArrayB') appears to not give the error. It's
possible that 'schema' is a reserved key value and was just not known to me.
[http://localhost:21000/api/atlas/v2/types/typedefs]
{code:java}
// Creating entities from this def works as expected
{
"entityDefs" : [
{
"superTypes" : [ "DataSet" ],
"category" : "ENTITY",
"name" : "DataSetTestA",
"description" : "description",
"typeVersion" : "1.0",
"attributeDefs" : [
{
"name" : "testArrayA",
"typeName" : "array",
"isOptional" : true,
"cardinality" : "SINGLE",
"valuesMinCount" : 1,
"valuesMaxCount" : 1,
"isUnique" : false,
"isIndexable" : false
} ]
}
]
}
{code}
{code:java}
// Creating an entity from this def also works. I was surprised by this because
I had only tried with the key 'schema' when making an array type. Using a key
other than schema (in this case testArrayB) did not give me an error.
"structDefs": [
{
"category": "STRUCT",
"name": "testStructB",
"description": "description",
"typeVersion": "1.0",
"attributeDefs": [
{
"name": "testAttribute",
"typeName": "string",
"isOptional": false,
"cardinality": "SINGLE",
"valuesMinCount": 1,
"valuesMaxCount": 1,
"isUnique": false,
"isIndexable": false
}
]
}
],
"entityDefs" : [
{
"superTypes" : [ "DataSet" ],
"category" : "ENTITY",
"name" : "DataSetTestB",
"description" : "description",
"typeVersion" : "1.0",
"attributeDefs" : [
{
"name" : "testArrayB",
"typeName" : "array",
"isOptional" : true,
"cardinality" : "SINGLE",
"valuesMinCount" : 1,
"valuesMaxCount" : 1,
"isUnique" : false,
"isIndexable" : false
}
]
}
]
}
{code}
{code:java}
// Creating an entity from this def will fail with an error
{
"structDefs": [
{
"category": "STRUCT",
"name": "testStructC",
"description": "column definition for schema",
"typeVersion": "1.0",
"attributeDefs": [
{
"name": "testAttribute",
"typeName": "string",
"isOptional": false,
"cardinality": "SINGLE",
"valuesMinCount": 1,
"valuesMaxCount": 1,
"isUnique": false,
"isIndexable": false
}
]
}
],
"entityDefs" : [
{
"superTypes" : [ "DataSet" ],
"category" : "ENTITY",
"name" : "DataSetTestC",
"description" : "description",
"typeVersion" : "1.0",
"attributeDefs" : [
{
"name" : "schema",
"typeName" : "array",
"isOptional" : true,
"cardinality" : "SINGLE",
"valuesMinCount" : 1,
"valuesMaxCount" : 1,
"isUnique" : false,
"isIndexable" : false
}
]
}
]
}
{code}
[http://localhost:21000/api/atlas/v2/entity/bulk]
{code:java}
// This post works
{
"entities": [
{
"typeName": "DataSetTestA",
"createdBy": "admin",
"guid": -100,
"attributes": {
"qualifiedName": "qualifiedName123",
"uri": "url123",
"name": "name123",
"testArrayA":[
"test", "test2", "test3"
]
}
}
]
}
{code}
{code:java}
// This post works
{
"entities": [
{
"typeName": "DataSetTestB",
"createdBy": "admin",
"guid": -100,
"attributes": {
"qualifiedName": "qualifiedName456",
"uri": "url456",
"name": "name456",
"testArrayB":[
{"testAttribute": "testValue1"},
{"testAttribute": "testValue2"}
]
}
}
]
}
{code}
{code:java}
// This post does NOT work. "errorMessage": "ObjectId is not valid
AtlasObjectId{guid='null', typeName='null', uniqueAttributes={}}"
{
"entities": [
{
"typeName": "DataSetTestC",
"createdBy": "admin",
"guid": -100,
"attributes": {
"qualifiedName": "qualifiedName789",
"uri": "url789",
"name": "name789",
"schema":[
{"testAttribute": "testValue1"},
{"testAttribute": "testValue2"}
]
}
}
[jira] [Commented] (ATLAS-2215) UI: Editing array of Structs behave as array of entity references
[
https://issues.apache.org/jira/browse/ATLAS-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995162#comment-16995162
]
Madhan Neethiraj commented on ATLAS-2215:
-
[~logan.rickert] - an attribute named {{schema}} is _injected_ into entity-type
{{DataSet}}, by following relationship-type {{avro_schema_associatedEntities}}
in 1000-Hadoop/1065-avro_model.json.
The attribute {{DataSetTestC.schema}} in is overwritten/shadowed by this
relationship attribute, hence the failure. Ideally, creation of DataSetTestC
should have failed due to this duplicate attribute name.
{noformat}
{
"name": "avro_schema_associatedEntities",
"serviceType": "avro",
"typeVersion": "1.0",
"relationshipCategory": "ASSOCIATION",
"relationshipLabel": " __avro_schema.associatedEntities",
"endDef1": {
"type": "avro_schema",
"name": "associatedEntities",
"isContainer": false,
"cardinality": "SET",
"isLegacyAttribute": true
},
"endDef2": {
"type":"DataSet",
"name":"schema",
"isContainer": false,
"cardinality": "SET"
}
},
{noformat}
> UI: Editing array of Structs behave as array of entity references
> -
>
> Key: ATLAS-2215
> URL: https://issues.apache.org/jira/browse/ATLAS-2215
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
>Priority: Major
>
> Reproduction steps:
> * create a struct with a set of attributes
> * create an entity type with an attribute defined as an array of the
> previously defined struct
> * Attempt to create an instance of the entity and set the struct value. It
> will behave as if the type is an array of entities and not a free form field.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ATLAS-2215) UI: Editing array of Structs behave as array of entity references
[ https://issues.apache.org/jira/browse/ATLAS-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16995132#comment-16995132 ] Logan Rickert commented on ATLAS-2215: -- Does anyone know if this issue still stands as open? Is this ticket being worked on? I am not using the UI but the REST API. I am running Atlas 2.1.0 and when I create an entity through the V2 REST API, it fails for the reason stated in the ticket. If no one is working on it, I would like to attempt to fix the issue. > UI: Editing array of Structs behave as array of entity references > - > > Key: ATLAS-2215 > URL: https://issues.apache.org/jira/browse/ATLAS-2215 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 1.0.0 >Reporter: Pierre Padovani >Priority: Major > > Reproduction steps: > * create a struct with a set of attributes > * create an entity type with an attribute defined as an array of the > previously defined struct > * Attempt to create an instance of the entity and set the struct value. It > will behave as if the type is an array of entities and not a free form field. -- This message was sent by Atlassian Jira (v8.3.4#803005)
