Re: Problems creating an entity of a type that has cyclic inheritance
"typeName":
"testInherited_MGPC",
"state": "ACTIVE",
"version": 0,
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Id"
},
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"
}
]
Command executed on bash in the cluster:
curl -i -k -X POST -H 'Content-Type: application/json' -H 'Accept:
application/json' -u 'https:///api/atlas/entities/' -d
@entry.json
Error from application.log of Atlas:
org.apache.atlas.typesystem.types.ValueConversionException$NullConversionException:
Null value not allowed for multiplicty {lower=1, upper=1, isUnique=false} .
Message Value expected f
or required attribute qualifiedName
at
org.apache.atlas.typesystem.types.ClassType.convert(ClassType.java:141)
at
org.apache.atlas.services.DefaultMetadataService.getTypedReferenceableInstance(DefaultMetadataService.java:365)
at
org.apache.atlas.services.DefaultMetadataService.deserializeClassInstances(DefaultMetadataService.java:342)
at
org.apache.atlas.services.DefaultMetadataService.createEntities(DefaultMetadataService.java:323)
at
org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:133)
…
Types:
/api/atlas/types/testInherited_MGPC:
{"typeName":"testInherited_MGPC","definition":{"enumTypes":[],"structTypes":[],"traitTypes":[],"classTypes":[{"superTypes":["DataSet_mc_v1_0","hive_table"],"hierarchicalMetaTypeName":"org.apache.atlas.typesystem.types.ClassType","typeName":"testInherited_MGPC","typeDescription":"Test
type created for Hive Atlas prototyping: This type inherits from
DataSet_mc_v1_0 and
hive_table","attributeDefinitions":[]}]},"requestId":"qtp366590980-71707 -
3462e295-2433-4ef6-b082-d43ef24d26ae"}
/api/atlas/types/DataSet_mc_v1_0
{"typeName":"DataSet_mc_v1_0","definition":{"enumTypes":[],"structTypes":[],"traitTypes":[],"classTypes":[{"superTypes":["DataSet"],"hierarchicalMetaTypeName":"org.apache.atlas.typesystem.types.ClassType","typeName":"DataSet_mc_v1_0","typeDescription":"MCloud
Dataset Version
1_0","attributeDefinitions":[{"name":"metaDataVersion","dataTypeName":"string","multiplicity":"required","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"dataOwner","dataTypeName":"string","multiplicity":"required","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"dataCustodian","dataTypeName":"string","multiplicity":"required","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"dataSteward","dataTypeName":"string","multiplicity":"required","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"dataPublisher","dataTypeName":"string","multiplicity":"required","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"shortTitle","dataTypeName":"string","multiplicity":"optional","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"approvedPurpose","dataTypeName":"string","multiplicity":"optional","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"informationClassification","dataTypeName":"string","multiplicity":"required","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"fileLocation","dataTypeName":&quo
Re: Problems creating an entity of a type that has cyclic inheritance
Rico Bergmann,
The type definitions look good and this shouldn’t cause the error you see. To
verify this usecase, I created entity-types CustomDataSet and CustomHiveTable –
as shown below; and was able to create an entity of type CustomHiveTable.
Please review the JSON you used to create the entity-instance. If all looks
good, please send the JSON contents to enable further troubleshooting.
Hope this helps.
Madhan
1. Create entity types ‘CustomDataSet’ and ‘CustomHiveTable’:
{
"entityDefs": [
{
"name": "CustomDataSet",
"category": "ENTITY",
"description": "Custom DataSet",
"superTypes": [
"DataSet"
],
"attributeDefs": []
},
{
"name": "CustomHiveTable",
"category": "ENTITY",
"description": "Custom HiveTable",
"superTypes": [
"CustomDataSet",
"hive_table"
],
"attributeDefs": []
}
]
}
2. Create an entity instance of type ‘CustomHiveTable’:
{
"entity": {
"guid": "-1",
"typeName": "CustomHiveTable",
"attributes": {
"name": "testtable",
"qualifiedName": "default.testtable@cl1",
"tableType": "MANAGED_TABLE",
"temporary": false,
"owner": "hive",
"db": {
"typeName": "hive_db",
"uniqueAttributes": {
"qualifiedName": "default@cl1"
}
},
"sd": {
"guid": "-3",
"typeName": "hive_storagedesc"
},
"columns": [
{
"guid": "-4",
"typeName": "hive_column"
},
{
"guid": "-5",
"typeName": "hive_column"
},
{
"guid": "-6",
"typeName": "hive_column"
}
]
}
},
"referredEntities": {
"-3": {
"guid": "-3",
"typeName": "hive_storagedesc",
"attributes": {
"qualifiedName": "default.testtable@cl1_storage",
"compressed": false,
"inputFormat": "org.apache.hadoop.mapred.TextInputFormat",
"location":
"hdfs://localhost.localdomain:8020/apps/hive/warehouse/testtable",
"numBuckets": -1,
"outputFormat":
"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat",
"storedAsSubDirectories": false,
"serdeInfo": {
"typeName": "hive_serde",
"attributes": {
"name": null,
"parameters": {
"serialization.format": "1"
},
"serializationLib":
"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"
}
},
"table": {
"guid": "-1",
"typeName": "CustomHiveTable"
}
}
},
"-4": {
"guid": "-4",
"typeName": "hive_column",
"attributes": {
"name": "id",
"type": "int",
"position": 0,
"qualifiedName": "default.testtable.id@cl1",
"table": {
"guid": "-1",
"typeName": "CustomHiveTable"
}
}
},
"-5": {
"guid": "-5",
"typeName": "hive_column",
"attributes": {
"name": "name",
"type": "string",
"position": 1,
"qualifiedName": "default.testtable.name@cl1",
"table": {
"guid": "-1",
"typeName": "CustomHiveTable"
}
}
},
"-6": {
"guid": "-6",
"typeName": "hive_column",
"attributes": {
"name": "dob",
"type": "date",
"position": 2,
"qualifiedName": "default.testtable1.dob@cl1",
"table": {
"guid": "-1",
"typeName": "CustomHiveTable"
}
}
}
}
}
From: Rico Bergmann
Reply-To: "user@atlas.apache.org"
Date: Wednesday, March 7, 2018 at 3:25 AM
To: "user@atlas.apache.org"
Subject: Problems creating an entity of a type that has cyclic inheritance
Hi!
I'm facing an issue when creating an entity of a custom defined type. The error
log says, that "qualifiedName" can not be null. But the "qualifiedName "
attribute is set in our JSON entity definition for sure.
May be the problem arrises because of cyclic dependencies in the inheritance
graph of the types. We defined a custom type inheriting from DataSet (let us
call it CustomDataset). Then we have another type "CustomHiveDataset" that
inherits from our CustomDataset and the built-in type hive_table. So for
example the type "Referenceable" (where qualifiedName is defined) is somehow
twice in the inheritance graph. Can this cause p
