Re: Problems creating an entity of a type that has cyclic inheritance
Rico,
For ‘testInherited_MGPC.db’ attribute, the value provided is the entire hive_db
entity. Instead, please use ‘Id’ to refer to the hive_db entity – as shown
below.
Also, “id” was missing for testInherited_MGPC entity. Similar to the value used
for the hive_db entity, please use an negative number as ‘id’.
Hope this helps.
Regards,
Madhan
[
{
"typeName": "hive_db",
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
"id": "-11893021824425525",
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"state": "ACTIVE",
"typeName": "hive_db",
"version": 0
},
"values": {
"clusterName": "cl1",
"description": "Default Hive database",
"location": "hdfs://mycluster/apps/hive/warehouse",
"name": "default",
"owner": "public",
"ownerType": 2,
"parameters": {},
"qualifiedName": "default@cl1"
},
"traitNames": [],
"traits": {}
},
{
"typeName": "testInherited_MGPC",
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id": {
"id": "-11893021824425524",
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"state": "ACTIVE",
"typeName": "testInherited_MGPC",
"version": 0
},
"values": {
"creationDate": "2018-03-05T11:30:30.45+01:00",
"dataCustodian": "name:B|firstName:N|MUID:M123456",
"dataOwner": "Sector:BT|SubDomain:MC",
"dataPublisher": "Sector:BT|SubDomain:R&D",
"dataSteward": "name:B|firstName:R|MUID:M654321",
"db": {
"id": "-11893021824425525",
"jsonClass":
"org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"state": "ACTIVE",
"typeName": "hive_db",
"version": 0
},
"fileLocation":
"hdfs://mycluster/user/rb/hive/emp2/run01/staged20007",
"firstCreation": "2018-03-05T11:30:30.45+01:00",
"informationClassification": "Internal",
"metaDataVersion": "v1_0",
"name": "RimsImportRun1Inherited",
"qualifiedName":
"testInherited_MGPC_RIMS_import_run01_staged2007",
"sourceSystem":
"sourceSystemName:IDMP|sourceSystemConnection:Oracle_JDBC_Driver"
}
"traitNames": [],
"traits": {}
}
]
From: Rico Bergmann
Reply-To: "user@atlas.apache.org"
Date: Monday, March 12, 2018 at 1:18 AM
To: "user@atlas.apache.org"
Subject: AW: Problems creating an entity of a type that has cyclic inheritance
Hi!
Thank you very much for your reply. Still I can not figure out the cause of the
problem …
I should mention that we are using Atlas 0.7 from HDP 2.5.5.
I attach the important parts here:
entry.json:
[
{
"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
"id":{
"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
"id":"-11893021824425525",
"version":0,
"typeName":"hive_db",
"state":"ACTIVE"
},
"typeName":"hive_db",
"values":{
"name":"default",
"loca
AW: Problems creating an entity of a type that has cyclic inheritance
ble":false,"reverseAttributeName":null},{"name":"shortTitle","dataTypeName":"string","multiplicity":"optional","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"approvedPurpose","dataTypeName":"string","multiplicity":"optional","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"informationClassification","dataTypeName":"string","multiplicity":"required","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"fileLocation","dataTypeName":"string","multiplicity":"required","isComposite":false,"isUnique":true,"isIndexable":false,"reverseAttributeName":null},{"name":"sourceSystem","dataTypeName":"string","multiplicity":"required","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"creationDate","dataTypeName":"string","multiplicity":"required","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"firstCreation","dataTypeName":"string","multiplicity":"required","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"geoJurisdiction","dataTypeName":"string","multiplicity":"optional","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"geoScope","dataTypeName":"string","multiplicity":"optional","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"retentionSchedule","dataTypeName":"string","multiplicity":"optional","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"replicationRetentionDate","dataTypeName":"string","multiplicity":"optional","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"archivingRetentionDate","dataTypeName":"string","multiplicity":"optional","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"ingestType","dataTypeName":"string","multiplicity":"optional","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"collectionStatus","dataTypeName":"string","multiplicity":"optional","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"updateFrequency","dataTypeName":"string","multiplicity":"optional","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"sector","dataTypeName":"string","multiplicity":"optional","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null},{"name":"subDomain","dataTypeName":"string","multiplicity":"optional","isComposite":false,"isUnique":false,"isIndexable":false,"reverseAttributeName":null}]}]},"requestId":"qtp366590980-71707
- 7669bc82-b491-4600-81e9-764a95087f5b"}
Any ideas what could cause the problem?
Thanks a lot!
Rico.
Von: Madhan Neethiraj [mailto:mad...@apache.org]
Gesendet: Montag, 12. März 2018 05:10
An: user@atlas.apache.org
Betreff: Re: Problems creating an entity of a type that has cyclic inheritance
Rico Bergmann,
The type definitions look good and this shouldn’t cause the error you see. To
verify this usecase, I created entity-types Custo
Re: Problems creating an entity of a type that has cyclic inheritance
Rico Bergmann,
The type definitions look good and this shouldn’t cause the error you see. To
verify this usecase, I created entity-types CustomDataSet and CustomHiveTable –
as shown below; and was able to create an entity of type CustomHiveTable.
Please review the JSON you used to create the entity-instance. If all looks
good, please send the JSON contents to enable further troubleshooting.
Hope this helps.
Madhan
1. Create entity types ‘CustomDataSet’ and ‘CustomHiveTable’:
{
"entityDefs": [
{
"name": "CustomDataSet",
"category": "ENTITY",
"description": "Custom DataSet",
"superTypes": [
"DataSet"
],
"attributeDefs": []
},
{
"name": "CustomHiveTable",
"category": "ENTITY",
"description": "Custom HiveTable",
"superTypes": [
"CustomDataSet",
"hive_table"
],
"attributeDefs": []
}
]
}
2. Create an entity instance of type ‘CustomHiveTable’:
{
"entity": {
"guid": "-1",
"typeName": "CustomHiveTable",
"attributes": {
"name": "testtable",
"qualifiedName": "default.testtable@cl1",
"tableType": "MANAGED_TABLE",
"temporary": false,
"owner": "hive",
"db": {
"typeName": "hive_db",
"uniqueAttributes": {
"qualifiedName": "default@cl1"
}
},
"sd": {
"guid": "-3",
"typeName": "hive_storagedesc"
},
"columns": [
{
"guid": "-4",
"typeName": "hive_column"
},
{
"guid": "-5",
"typeName": "hive_column"
},
{
"guid": "-6",
"typeName": "hive_column"
}
]
}
},
"referredEntities": {
"-3": {
"guid": "-3",
"typeName": "hive_storagedesc",
"attributes": {
"qualifiedName": "default.testtable@cl1_storage",
"compressed": false,
"inputFormat": "org.apache.hadoop.mapred.TextInputFormat",
"location":
"hdfs://localhost.localdomain:8020/apps/hive/warehouse/testtable",
"numBuckets": -1,
"outputFormat":
"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat",
"storedAsSubDirectories": false,
"serdeInfo": {
"typeName": "hive_serde",
"attributes": {
"name": null,
"parameters": {
"serialization.format": "1"
},
"serializationLib":
"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"
}
},
"table": {
"guid": "-1",
"typeName": "CustomHiveTable"
}
}
},
"-4": {
"guid": "-4",
"typeName": "hive_column",
"attributes": {
"name": "id",
"type": "int",
"position": 0,
"qualifiedName": "default.testtable.id@cl1",
"table": {
"guid": "-1",
"typeName": "CustomHiveTable"
}
}
},
"-5": {
"guid": "-5",
"typeName": "hive_column",
"attributes": {
"name": "name",
"type": "string",
"position": 1,
"qualifiedName": "default.testtable.name@cl1",
"table": {
"guid": "-1",
"typeName": "CustomHiveTa
Problems creating an entity of a type that has cyclic inheritance
Hi! I'm facing an issue when creating an entity of a custom defined type. The error log says, that "qualifiedName" can not be null. But the "qualifiedName " attribute is set in our JSON entity definition for sure. May be the problem arrises because of cyclic dependencies in the inheritance graph of the types. We defined a custom type inheriting from DataSet (let us call it CustomDataset). Then we have another type "CustomHiveDataset" that inherits from our CustomDataset and the built-in type hive_table. So for example the type "Referenceable" (where qualifiedName is defined) is somehow twice in the inheritance graph. Can this cause problems? Best, Rico Bergmann. This message and any attachment are confidential and may be privileged or otherwise protected from disclosure. If you are not the intended recipient, you must not copy this message or attachment or disclose the contents to any other person. If you have received this transmission in error, please notify the sender immediately and delete the message and any attachment from your system. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not accept liability for any omissions or errors in this message which may arise as a result of E-Mail-transmission or for damages resulting from any unauthorized changes of the content of this message and any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not guarantee that this message is free of viruses and does not accept liability for any damages caused by any virus transmitted therewith. Click http://www.merckgroup.com/disclaimer to access the German, French, Spanish and Portuguese versions of this disclaimer.
