[jira] [Commented] (ATLAS-1990) Regression :Traits Attribute that References Another Entity
[
https://issues.apache.org/jira/browse/ATLAS-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401712#comment-16401712
]
Mark Kiami commented on ATLAS-1990:
---
Thank you [~madhan.neethiraj] for kindly providing the links.
I think in my case, I will need to create a relationship to the entity instead
of a classification.
It would be good to have this documented in one easily accessible place.
> Regression :Traits Attribute that References Another Entity
> ---
>
> Key: ATLAS-1990
> URL: https://issues.apache.org/jira/browse/ATLAS-1990
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
> Environment: HDP2.6
>Reporter: Benjamin BONNET
>Priority: Major
>
> Hi,
> I defined a trait that includes one attribute. That attribute has type
> "hive_column", i.e. it references a hive_column entity.
> Using Altas 0.5 on a HDP 2.4, I could attach that trait to an entity so that
> it refers a column. But using Atlas 0.8 on a HDP 2.6 that does not work
> anymore (I get a NullPointerException).
> I tried both V1 and V2 REST API and got the same error.
> Here is a full sample to reproduce that issue.
> Trait definition :
> {noformat}
> {"enumTypes":[],
> "structTypes":[],
> "traitTypes":[{
>"superTypes":[],
>"hierarchicalMetaTypeName":"org.apache.atlas.typesystem.types.TraitType",
>"typeName":"Reference",
>"typeDescription":"reference test",
>"attributeDefinitions":[{"name":"ref",
>"dataTypeName":"hive_column",
> "multiplicity":"required",
> "isComposite":false,
> "isUnique":false,
> "isIndexable":true,
> "reverseAttributeName":null
> }
> ]}],
> "classTypes":[]}
> {noformat}
> Then I use curl to create that type (ref file contains the traits definition):
> {noformat}
> curl -v -u admin:admin -X POST -d "`cat ref`" -H 'Content-Type:
> application/json' 'http://127.0.0.1:21000/api/atlas/types'
> {noformat}
> Using that command, trait type is created OK on both 0.5 and 0.8.
> Now, I can tag an entity on Atlas 0.5 :
> {noformat}
> curl -v -u admin:admin -X POST -d
> '{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
>
> "typeName":"Reference","values":{"ref":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"9e2b0059-8e75-42ce-a32c-fe9a6157caed","version":0,"typeName":"hive_column"}}}'
> -H "Content-Type: application/json"
> "http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits;
> {noformat}
> I get a HTTP201 :
> < HTTP/1.1 201 Created
> < Date: Tue, 25 Jul 2017 09:11:23 GMT
> < Location:
> http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits/23d1c3e7-7ad9-47b6-ac54-1d507119d00d
> < Content-Type: application/json; charset=UTF-8
> < Transfer-Encoding: chunked
> < Server: Jetty(9.2.12.v20150709)
> <
> * Connection #0 to host 127.0.0.1 left intact
> * Closing connection #0
> {"requestId":"qtp999445351-15 -
> 6d4b21dd-af88-46f3-91f4-5b3b20b6840c","GUID":"23d1c3e7-7ad9-47b6-ac54-1d507119d00d"}
> And if I check my entity traits, I can see that tag.
> {noformat}
> curl -v -u admin:admin -X GET
> "http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits;
> * About to connect() to 127.0.0.1 port 21000 (#0)
> * Trying 127.0.0.1... connected
> * Connected to 127.0.0.1 (127.0.0.1) port 21000 (#0)
> * Server auth using Basic with user 'admin'
> > GET /api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits HTTP/1.1
> > Authorization: Basic YWRtaW46YWRtaW4=
> > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1
> > Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> > Host: 127.0.0.1:21000
> > Accept: */*
> >
> < HTTP/1.1 200 OK
> < Date: Tue, 25 Jul 2017 09:13:27 GMT
> < Content-Type: application/json; charset=UTF-8
> < Transfer-Encoding: chunked
> < Server: Jetty(9.2.12.v20150709)
> <
> * Connection #0 to host 127.0.0.1 left intact
> * Closing connection #0
> {"requestId":"qtp999445351-13 -
> 15a0ce26-484a-4181-9f5a-742227be5d09","GUID":"23d1c3e7-7ad9-47b6-ac54-1d507119d00d","results":["tag1","tag2","Reference","tag3"],"count":4}
> {noformat}
> So far, so good, but If I try to do the same thing Atlas 0.8, i.e. tagging an
> entity with that trait :
> {noformat}
> curl -v -u admin:admin -X POST -d
> '{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
>
>
[jira] [Comment Edited] (ATLAS-1990) Regression :Traits Attribute that References Another Entity
[
https://issues.apache.org/jira/browse/ATLAS-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398616#comment-16398616
]
Mark Kiami edited comment on ATLAS-1990 at 3/14/18 1:47 PM:
For the record, this is the simple scenario where it fails:
1. Create an ENTITY type "some_entity_type_1"
*/v2/types/typedefs*
{code:javascript}
{
"entityDefs": [
{
"attributeDefs": [
{
"name": "some_custom_name",
"typeName": "string",
"isOptional": false,
"cardinality": "SINGLE",
"isUnique": false,
"isIndexable": true
}
],
"category": "ENTITY",
"createdBy": "mark",
"name": "some_entity_type_1",
"description": "Some entity type for testing purposes"
}
]
}
{code}
2. Create an entity of that ENTITY type
*/v2/entity*
{code:javascript}
{
"entity": {
"status": "ACTIVE",
"createdBy": "mark",
"typeName": "some_entity_type_1",
"attributes": {
"some_custom_name": "This is some custom name for testing"
}
}
}
{code}
3. Atlas will provide a guid for that newly created entity, suppose it is
"123456"
4. Create a CLASSIFICATION type "some_classification_type_1" that has an
attribute of type ENTITY "some_entity_type_1"
*/v2/types/typedefs*
{code:javascript}
{
"classificationDefs": [
{
"attributeDefs": [
{
"name": "custom_classification_name",
"typeName": "some_entity_type_1",
"isOptional": false,
"cardinality": "SINGLE",
"isUnique": false,
"isIndexable": true
}
],
"category": "CLASSIFICATION",
"createdBy": "mark",
"name": "some_classification_type_1",
"description": "Some classification type"
}
]
}
{code}
5. Add a classification of type "some_classification_type_1" to a hive_column
with guid "123"
*/v2/entity/guid/123/classifications*
{code:javascript}
[
{
"typeName": "some_classification_type_1",
"attributes": {
"custom_classification_name": {
"guid": "123456",
"typeName": "some_entity_type_1"
}
}
}
]
{code}
Atlas fails with a null pointer exception as mentioned in my previous comment.
Is this a bug or a limitation of Atlas, it would be good to have tags that
reference complex entities.
was (Author: mkiami):
For the record, this is the simple scenario where it fails:
# Create an ENTITY type "some_entity_type_1"
*/v2/types/typedefs*
{code:javascript}
{
"entityDefs": [
{
"attributeDefs": [
{
"name": "some_custom_name",
"typeName": "string",
"isOptional": false,
"cardinality": "SINGLE",
"isUnique": false,
"isIndexable": true
}
],
"category": "ENTITY",
"createdBy": "mark",
"name": "some_entity_type_1",
"description": "Some entity type for testing purposes"
}
]
}
{code}
# Create an entity of that ENTITY type
*/v2/entity*
{code:javascript}
{
"entity": {
"status": "ACTIVE",
"createdBy": "mark",
"typeName": "some_entity_type_1",
"attributes": {
"some_custom_name": "This is some custom name for testing"
}
}
}
{code}
# Atlas will provide a guid for that newly created entity, suppose it is
"123456"
# Create a CLASSIFICATION type "some_classification_type_1" that has an
attribute of type ENTITY "some_entity_type_1"
*/v2/types/typedefs*
{code:javascript}
{
"classificationDefs": [
{
"attributeDefs": [
{
"name": "custom_classification_name",
"typeName": "some_entity_type_1",
"isOptional": false,
"cardinality": "SINGLE",
"isUnique": false,
"isIndexable": true
}
],
"category": "CLASSIFICATION",
"createdBy": "mark",
"name": "some_classification_type_1",
"description": "Some classification type"
}
]
}
{code}
# Add a classification of type "some_classification_type_1" to a hive_column
with guid "123"
*/v2/entity/guid/123/classifications*
{code:javascript}
[
{
"typeName": "some_classification_type_1",
"attributes": {
"custom_classification_name": {
"guid": "123456",
"typeName": "some_entity_type_1"
}
}
}
]
{code}
Atlas fails with a null pointer exception as mentioned in my previous comment.
Is this a bug or a limitation of Atlas, it would be good to have tags that
reference complex entities.
> Regression :Traits Attribute that References Another Entity
> ---
>
> Key:
[jira] [Commented] (ATLAS-1990) Regression :Traits Attribute that References Another Entity
[
https://issues.apache.org/jira/browse/ATLAS-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398616#comment-16398616
]
Mark Kiami commented on ATLAS-1990:
---
For the record, this is the simple scenario where it fails:
# Create an ENTITY type "some_entity_type_1"
*/v2/types/typedefs*
{code:javascript}
{
"entityDefs": [
{
"attributeDefs": [
{
"name": "some_custom_name",
"typeName": "string",
"isOptional": false,
"cardinality": "SINGLE",
"isUnique": false,
"isIndexable": true
}
],
"category": "ENTITY",
"createdBy": "mark",
"name": "some_entity_type_1",
"description": "Some entity type for testing purposes"
}
]
}
{code}
# Create an entity of that ENTITY type
*/v2/entity*
{code:javascript}
{
"entity": {
"status": "ACTIVE",
"createdBy": "mark",
"typeName": "some_entity_type_1",
"attributes": {
"some_custom_name": "This is some custom name for testing"
}
}
}
{code}
# Atlas will provide a guid for that newly created entity, suppose it is
"123456"
# Create a CLASSIFICATION type "some_classification_type_1" that has an
attribute of type ENTITY "some_entity_type_1"
*/v2/types/typedefs*
{code:javascript}
{
"classificationDefs": [
{
"attributeDefs": [
{
"name": "custom_classification_name",
"typeName": "some_entity_type_1",
"isOptional": false,
"cardinality": "SINGLE",
"isUnique": false,
"isIndexable": true
}
],
"category": "CLASSIFICATION",
"createdBy": "mark",
"name": "some_classification_type_1",
"description": "Some classification type"
}
]
}
{code}
# Add a classification of type "some_classification_type_1" to a hive_column
with guid "123"
*/v2/entity/guid/123/classifications*
{code:javascript}
[
{
"typeName": "some_classification_type_1",
"attributes": {
"custom_classification_name": {
"guid": "123456",
"typeName": "some_entity_type_1"
}
}
}
]
{code}
Atlas fails with a null pointer exception as mentioned in my previous comment.
Is this a bug or a limitation of Atlas, it would be good to have tags that
reference complex entities.
> Regression :Traits Attribute that References Another Entity
> ---
>
> Key: ATLAS-1990
> URL: https://issues.apache.org/jira/browse/ATLAS-1990
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
> Environment: HDP2.6
>Reporter: Benjamin BONNET
>Priority: Major
>
> Hi,
> I defined a trait that includes one attribute. That attribute has type
> "hive_column", i.e. it references a hive_column entity.
> Using Altas 0.5 on a HDP 2.4, I could attach that trait to an entity so that
> it refers a column. But using Atlas 0.8 on a HDP 2.6 that does not work
> anymore (I get a NullPointerException).
> I tried both V1 and V2 REST API and got the same error.
> Here is a full sample to reproduce that issue.
> Trait definition :
> {noformat}
> {"enumTypes":[],
> "structTypes":[],
> "traitTypes":[{
>"superTypes":[],
>"hierarchicalMetaTypeName":"org.apache.atlas.typesystem.types.TraitType",
>"typeName":"Reference",
>"typeDescription":"reference test",
>"attributeDefinitions":[{"name":"ref",
>"dataTypeName":"hive_column",
> "multiplicity":"required",
> "isComposite":false,
> "isUnique":false,
> "isIndexable":true,
> "reverseAttributeName":null
> }
> ]}],
> "classTypes":[]}
> {noformat}
> Then I use curl to create that type (ref file contains the traits definition):
> {noformat}
> curl -v -u admin:admin -X POST -d "`cat ref`" -H 'Content-Type:
> application/json' 'http://127.0.0.1:21000/api/atlas/types'
> {noformat}
> Using that command, trait type is created OK on both 0.5 and 0.8.
> Now, I can tag an entity on Atlas 0.5 :
> {noformat}
> curl -v -u admin:admin -X POST -d
> '{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
>
> "typeName":"Reference","values":{"ref":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"9e2b0059-8e75-42ce-a32c-fe9a6157caed","version":0,"typeName":"hive_column"}}}'
> -H "Content-Type: application/json"
> "http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits;
> {noformat}
> I get a HTTP201 :
> < HTTP/1.1 201 Created
> < Date: Tue, 25 Jul 2017
[jira] [Comment Edited] (ATLAS-1990) Regression :Traits Attribute that References Another Entity
[
https://issues.apache.org/jira/browse/ATLAS-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398516#comment-16398516
]
Mark Kiami edited comment on ATLAS-1990 at 3/14/18 1:07 PM:
This is a blocker in our project, I have a very similar issue described by
[~bbonnet], when tagging a "hive_column" entity
Any news regarding this bug?
Thanks!
was (Author: mkiami):
This is a blocker in our project, I have exactly the same issue described by
[~bbonnet]
Any news regarding this bug?
Thanks!
> Regression :Traits Attribute that References Another Entity
> ---
>
> Key: ATLAS-1990
> URL: https://issues.apache.org/jira/browse/ATLAS-1990
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
> Environment: HDP2.6
>Reporter: Benjamin BONNET
>Priority: Major
>
> Hi,
> I defined a trait that includes one attribute. That attribute has type
> "hive_column", i.e. it references a hive_column entity.
> Using Altas 0.5 on a HDP 2.4, I could attach that trait to an entity so that
> it refers a column. But using Atlas 0.8 on a HDP 2.6 that does not work
> anymore (I get a NullPointerException).
> I tried both V1 and V2 REST API and got the same error.
> Here is a full sample to reproduce that issue.
> Trait definition :
> {noformat}
> {"enumTypes":[],
> "structTypes":[],
> "traitTypes":[{
>"superTypes":[],
>"hierarchicalMetaTypeName":"org.apache.atlas.typesystem.types.TraitType",
>"typeName":"Reference",
>"typeDescription":"reference test",
>"attributeDefinitions":[{"name":"ref",
>"dataTypeName":"hive_column",
> "multiplicity":"required",
> "isComposite":false,
> "isUnique":false,
> "isIndexable":true,
> "reverseAttributeName":null
> }
> ]}],
> "classTypes":[]}
> {noformat}
> Then I use curl to create that type (ref file contains the traits definition):
> {noformat}
> curl -v -u admin:admin -X POST -d "`cat ref`" -H 'Content-Type:
> application/json' 'http://127.0.0.1:21000/api/atlas/types'
> {noformat}
> Using that command, trait type is created OK on both 0.5 and 0.8.
> Now, I can tag an entity on Atlas 0.5 :
> {noformat}
> curl -v -u admin:admin -X POST -d
> '{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
>
> "typeName":"Reference","values":{"ref":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"9e2b0059-8e75-42ce-a32c-fe9a6157caed","version":0,"typeName":"hive_column"}}}'
> -H "Content-Type: application/json"
> "http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits;
> {noformat}
> I get a HTTP201 :
> < HTTP/1.1 201 Created
> < Date: Tue, 25 Jul 2017 09:11:23 GMT
> < Location:
> http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits/23d1c3e7-7ad9-47b6-ac54-1d507119d00d
> < Content-Type: application/json; charset=UTF-8
> < Transfer-Encoding: chunked
> < Server: Jetty(9.2.12.v20150709)
> <
> * Connection #0 to host 127.0.0.1 left intact
> * Closing connection #0
> {"requestId":"qtp999445351-15 -
> 6d4b21dd-af88-46f3-91f4-5b3b20b6840c","GUID":"23d1c3e7-7ad9-47b6-ac54-1d507119d00d"}
> And if I check my entity traits, I can see that tag.
> {noformat}
> curl -v -u admin:admin -X GET
> "http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits;
> * About to connect() to 127.0.0.1 port 21000 (#0)
> * Trying 127.0.0.1... connected
> * Connected to 127.0.0.1 (127.0.0.1) port 21000 (#0)
> * Server auth using Basic with user 'admin'
> > GET /api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits HTTP/1.1
> > Authorization: Basic YWRtaW46YWRtaW4=
> > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1
> > Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> > Host: 127.0.0.1:21000
> > Accept: */*
> >
> < HTTP/1.1 200 OK
> < Date: Tue, 25 Jul 2017 09:13:27 GMT
> < Content-Type: application/json; charset=UTF-8
> < Transfer-Encoding: chunked
> < Server: Jetty(9.2.12.v20150709)
> <
> * Connection #0 to host 127.0.0.1 left intact
> * Closing connection #0
> {"requestId":"qtp999445351-13 -
> 15a0ce26-484a-4181-9f5a-742227be5d09","GUID":"23d1c3e7-7ad9-47b6-ac54-1d507119d00d","results":["tag1","tag2","Reference","tag3"],"count":4}
> {noformat}
> So far, so good, but If I try to do the same thing Atlas 0.8, i.e. tagging an
> entity with that trait :
> {noformat}
> curl -v -u admin:admin -X POST -d
> '{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
>
>
[jira] [Commented] (ATLAS-1990) Regression :Traits Attribute that References Another Entity
[
https://issues.apache.org/jira/browse/ATLAS-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398539#comment-16398539
]
Mark Kiami commented on ATLAS-1990:
---
Adding the stack trace since it fails on a different line in
EntityGraphMapper.java, maybe due to the version of Atlas, I am using HDP 2.6.4
/ Atlas 0.8.0
{code:java}
2018-03-13 21:10:09,849 ERROR - [pool-2-thread-10 -
79a5a742-bb68-4e8c-9a6c-1f3449f73356:] ~ graph rollback due to exception
(GraphTransactionInterceptor:71)
java.lang.NullPointerException
at
org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapObjectIdValue(EntityGraphMapper.java:458)
at
org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapCollectionElementsToVertex(EntityGraphMapper.java:658)
at
org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapArrayValue(EntityGraphMapper.java:582)
at
org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapToVertexByTypeCategory(EntityGraphMapper.java:332)
at
org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapAttribute(EntityGraphMapper.java:274)
at
org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapAttributes(EntityGraphMapper.java:235)
at
org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapClassification(EntityGraphMapper.java:968)
at
org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.addClassifications(EntityGraphMapper.java:930)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.addClassifications(AtlasEntityStoreV1.java:374)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1$$FastClassBySpringCGLIB$$80c00649.invoke()
...
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-03-13 21:10:09,850 ERROR - [pool-2-thread-10 -
79a5a742-bb68-4e8c-9a6c-1f3449f73356:] ~ Error handling a request:
bdd8454e0b7c9331 (ExceptionMapperUtil:32)
java.lang.NullPointerException
at
org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapObjectIdValue(EntityGraphMapper.java:458)
at
org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapCollectionElementsToVertex(EntityGraphMapper.java:658)
at
org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapArrayValue(EntityGraphMapper.java:582)
at
org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapToVertexByTypeCategory(EntityGraphMapper.java:332)
at
org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapAttribute(EntityGraphMapper.java:274)
at
org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapAttributes(EntityGraphMapper.java:235)
at
org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapClassification(EntityGraphMapper.java:968)
at
org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.addClassifications(EntityGraphMapper.java:930)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.addClassifications(AtlasEntityStoreV1.java:374)
at
org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1$$FastClassBySpringCGLIB$$80c00649.invoke()
...
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
> Regression :Traits Attribute that References Another Entity
> ---
>
> Key: ATLAS-1990
> URL: https://issues.apache.org/jira/browse/ATLAS-1990
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
> Environment: HDP2.6
>Reporter: Benjamin BONNET
>Priority: Major
>
> Hi,
> I defined a trait that includes one attribute. That attribute has type
> "hive_column", i.e. it references a hive_column entity.
> Using Altas 0.5 on a HDP 2.4, I could attach that trait to an entity so that
> it refers a column. But using Atlas 0.8 on a HDP 2.6 that does not work
> anymore (I get a NullPointerException).
> I tried both V1 and V2 REST API and got the same error.
> Here is a full sample to reproduce that issue.
> Trait definition :
> {noformat}
> {"enumTypes":[],
> "structTypes":[],
> "traitTypes":[{
>"superTypes":[],
>"hierarchicalMetaTypeName":"org.apache.atlas.typesystem.types.TraitType",
>"typeName":"Reference",
>"typeDescription":"reference test",
>"attributeDefinitions":[{"name":"ref",
>"dataTypeName":"hive_column",
> "multiplicity":"required",
> "isComposite":false,
> "isUnique":false,
> "isIndexable":true,
>
[jira] [Commented] (ATLAS-1990) Regression :Traits Attribute that References Another Entity
[
https://issues.apache.org/jira/browse/ATLAS-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16398516#comment-16398516
]
Mark Kiami commented on ATLAS-1990:
---
This is a blocker in our project, I have exactly the same issue described by
[~bbonnet]
Any news regarding this bug?
Thanks!
> Regression :Traits Attribute that References Another Entity
> ---
>
> Key: ATLAS-1990
> URL: https://issues.apache.org/jira/browse/ATLAS-1990
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 0.8-incubating
> Environment: HDP2.6
>Reporter: Benjamin BONNET
>Priority: Major
>
> Hi,
> I defined a trait that includes one attribute. That attribute has type
> "hive_column", i.e. it references a hive_column entity.
> Using Altas 0.5 on a HDP 2.4, I could attach that trait to an entity so that
> it refers a column. But using Atlas 0.8 on a HDP 2.6 that does not work
> anymore (I get a NullPointerException).
> I tried both V1 and V2 REST API and got the same error.
> Here is a full sample to reproduce that issue.
> Trait definition :
> {noformat}
> {"enumTypes":[],
> "structTypes":[],
> "traitTypes":[{
>"superTypes":[],
>"hierarchicalMetaTypeName":"org.apache.atlas.typesystem.types.TraitType",
>"typeName":"Reference",
>"typeDescription":"reference test",
>"attributeDefinitions":[{"name":"ref",
>"dataTypeName":"hive_column",
> "multiplicity":"required",
> "isComposite":false,
> "isUnique":false,
> "isIndexable":true,
> "reverseAttributeName":null
> }
> ]}],
> "classTypes":[]}
> {noformat}
> Then I use curl to create that type (ref file contains the traits definition):
> {noformat}
> curl -v -u admin:admin -X POST -d "`cat ref`" -H 'Content-Type:
> application/json' 'http://127.0.0.1:21000/api/atlas/types'
> {noformat}
> Using that command, trait type is created OK on both 0.5 and 0.8.
> Now, I can tag an entity on Atlas 0.5 :
> {noformat}
> curl -v -u admin:admin -X POST -d
> '{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
>
> "typeName":"Reference","values":{"ref":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"9e2b0059-8e75-42ce-a32c-fe9a6157caed","version":0,"typeName":"hive_column"}}}'
> -H "Content-Type: application/json"
> "http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits;
> {noformat}
> I get a HTTP201 :
> < HTTP/1.1 201 Created
> < Date: Tue, 25 Jul 2017 09:11:23 GMT
> < Location:
> http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits/23d1c3e7-7ad9-47b6-ac54-1d507119d00d
> < Content-Type: application/json; charset=UTF-8
> < Transfer-Encoding: chunked
> < Server: Jetty(9.2.12.v20150709)
> <
> * Connection #0 to host 127.0.0.1 left intact
> * Closing connection #0
> {"requestId":"qtp999445351-15 -
> 6d4b21dd-af88-46f3-91f4-5b3b20b6840c","GUID":"23d1c3e7-7ad9-47b6-ac54-1d507119d00d"}
> And if I check my entity traits, I can see that tag.
> {noformat}
> curl -v -u admin:admin -X GET
> "http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits;
> * About to connect() to 127.0.0.1 port 21000 (#0)
> * Trying 127.0.0.1... connected
> * Connected to 127.0.0.1 (127.0.0.1) port 21000 (#0)
> * Server auth using Basic with user 'admin'
> > GET /api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits HTTP/1.1
> > Authorization: Basic YWRtaW46YWRtaW4=
> > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1
> > Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> > Host: 127.0.0.1:21000
> > Accept: */*
> >
> < HTTP/1.1 200 OK
> < Date: Tue, 25 Jul 2017 09:13:27 GMT
> < Content-Type: application/json; charset=UTF-8
> < Transfer-Encoding: chunked
> < Server: Jetty(9.2.12.v20150709)
> <
> * Connection #0 to host 127.0.0.1 left intact
> * Closing connection #0
> {"requestId":"qtp999445351-13 -
> 15a0ce26-484a-4181-9f5a-742227be5d09","GUID":"23d1c3e7-7ad9-47b6-ac54-1d507119d00d","results":["tag1","tag2","Reference","tag3"],"count":4}
> {noformat}
> So far, so good, but If I try to do the same thing Atlas 0.8, i.e. tagging an
> entity with that trait :
> {noformat}
> curl -v -u admin:admin -X POST -d
> '{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
>
> "typeName":"Reference","values":{"ref":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"d5a507e6-0752-4922-a524-cac6e8606f7d","version":0,"typeName":"hive_column"}}}'
> -H "Content-Type: application/json"
>
[jira] [Updated] (ATLAS-2498) Soft and hard deletes, deleting a "DELETED" entity and cleanup of unused types
[
https://issues.apache.org/jira/browse/ATLAS-2498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Kiami updated ATLAS-2498:
--
Summary: Soft and hard deletes, deleting a "DELETED" entity and cleanup of
unused types (was: Soft and hard deletes)
> Soft and hard deletes, deleting a "DELETED" entity and cleanup of unused types
> --
>
> Key: ATLAS-2498
> URL: https://issues.apache.org/jira/browse/ATLAS-2498
> Project: Atlas
> Issue Type: Improvement
>Affects Versions: 0.8-incubating
>Reporter: Mark Kiami
>Priority: Major
>
> Hello,
> Since the documentation on this feature not clear, could you please answer or
> confirm the following:
> # When an entity is deleted, its status goes to "DELETED", this is a soft
> delete.
> # Does Atlas version 0.8.0 support hard deletes (HDP 2.6.4 ships with 0.8.0)?
> # Is the atlas.DeleteHandler.impl configuration key supported in this
> version?
> # Is it possible to delete a "DELETED" entity?
> I am asking this because I am stuck in the following scenario:
> * Create a new ENTITY type named "custom_type"
> * Create an entity of that type "entity_1_of_custom_type", its status is
> "ACTIVE"
> * Delete the entity "entity_1_of_custom_type", its status is "DELETED"
> * Delete the ENTITY type named "custom_type"
> * I get an error: \{"errorCode":"ATLAS-409-00-002","errorMessage":"Given
> type custom_type has references"}
> How is it possible in this case to cleanup both the DELETED entity and the
> unused type "custom_type"?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ATLAS-2498) Soft and hard deletes
Mark Kiami created ATLAS-2498:
-
Summary: Soft and hard deletes
Key: ATLAS-2498
URL: https://issues.apache.org/jira/browse/ATLAS-2498
Project: Atlas
Issue Type: Improvement
Affects Versions: 0.8-incubating
Reporter: Mark Kiami
Hello,
Since the documentation on this feature not clear, could you please answer or
confirm the following:
# When an entity is deleted, its status goes to "DELETED", this is a soft
delete.
# Does Atlas version 0.8.0 support hard deletes (HDP 2.6.4 ships with 0.8.0)?
# Is the atlas.DeleteHandler.impl configuration key supported in this version?
# Is it possible to delete a "DELETED" entity?
I am asking this because I am stuck in the following scenario:
* Create a new ENTITY type named "custom_type"
* Create an entity of that type "entity_1_of_custom_type", its status is
"ACTIVE"
* Delete the entity "entity_1_of_custom_type", its status is "DELETED"
* Delete the ENTITY type named "custom_type"
* I get an error: \{"errorCode":"ATLAS-409-00-002","errorMessage":"Given type
custom_type has references"}
How is it possible in this case to cleanup both the DELETED entity and the
unused type "custom_type"?
Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (ATLAS-541) Soft deletes
[ https://issues.apache.org/jira/browse/ATLAS-541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Kiami updated ATLAS-541: - Comment: was deleted (was: Could the following be confirmed please, as documentation on this feature is not clear: * Does Atlas version 0.8.0 support hard deletes (HDP 2.6.4 ships with 0.8.0)? * Is the atlas.DeleteHandler.impl configuration supported in this version? * Is it possible to delete a "DELETED" entity? Thanks!) > Soft deletes > > > Key: ATLAS-541 > URL: https://issues.apache.org/jira/browse/ATLAS-541 > Project: Atlas > Issue Type: New Feature >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Major > Fix For: 0.7-incubating > > > We don't have graph versioning currently and hard deletes are not acceptable > for data governance. This jira tracks the proposal for soft deletes which can > mark an entity as deleted and by default search should return only active > entities. However, there should be an option to retrieve deleted entities -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-541) Soft deletes
[ https://issues.apache.org/jira/browse/ATLAS-541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16397154#comment-16397154 ] Mark Kiami commented on ATLAS-541: -- Could the following be confirmed please, as documentation on this feature is not clear: * Does Atlas version 0.8.0 support hard deletes (HDP 2.6.4 ships with 0.8.0)? * Is the atlas.DeleteHandler.impl configuration supported in this version? * Is it possible to delete a "DELETED" entity? Thanks! > Soft deletes > > > Key: ATLAS-541 > URL: https://issues.apache.org/jira/browse/ATLAS-541 > Project: Atlas > Issue Type: New Feature >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Major > Fix For: 0.7-incubating > > > We don't have graph versioning currently and hard deletes are not acceptable > for data governance. This jira tracks the proposal for soft deletes which can > mark an entity as deleted and by default search should return only active > entities. However, there should be an option to retrieve deleted entities -- This message was sent by Atlassian JIRA (v7.6.3#76005)
