[jira] [Commented] (ATLAS-342) Atlas is sending an ENTITY_CREATE event to the ATLAS_ENTITIES topic even if the entity exists already.
[ https://issues.apache.org/jira/browse/ATLAS-342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17065380#comment-17065380 ] ASF subversion and git services commented on ATLAS-342: --- Commit 784b606ddfa2149535b437483051a68859689a9d in atlas's branch refs/heads/dependabot/npm_and_yarn/dashboardv2/bootstrap-3.4.1 from mayanknj [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=784b606 ] ATLAS-342: Import Glossary Terms from csv/excel file into Glossary Signed-off-by: Sarath Subramanian > Atlas is sending an ENTITY_CREATE event to the ATLAS_ENTITIES topic even if > the entity exists already. > -- > > Key: ATLAS-342 > URL: https://issues.apache.org/jira/browse/ATLAS-342 > Project: Atlas > Issue Type: Bug >Reporter: Hemanth Yamijala >Assignee: Shwetha GS >Priority: Major > Fix For: 0.6-incubating > > Attachments: ATLAS-342.patch > > > * Configure Hive hook in Hive. > * Configure Atlas to talk to Kafka topics on a separately set Kafka instance. > * Run a script to create tables in Hive multiple times, for e.g. > {code} > for i in `seq 1 10`; do ./bin/hive -e "create table tbl$i (column${i}1 > string, column${i}2 int)"; done > {code} > * After the script completes, check the number of entities & list the actual > entities in the ATLAS_ENTITIES topic. > We can see one ENTITY_CREATE event for the Hive database for every table > created in Hive. For e.g.: > {code} > {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"e7ed4ad9-1fba-47ad-a089-1ff7e715c1ad","version":0,"typeName":"hive_db"},"typeName":"hive_db","values":{"name":"default","description":"Default > Hive > database","ownerType":{"value":"ROLE","ordinal":2},"qualifiedName":"primary.default","locationUri":"hdfs://localhost:9000/user/hive/warehouse","ownerName":"public","clusterName":"primary"},"traitNames":[],"traits":{}},"operationType":"ENTITY_CREATE","traits":[]} > {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"e7ed4ad9-1fba-47ad-a089-1ff7e715c1ad","version":0,"typeName":"hive_db"},"typeName":"hive_db","values":{"name":"default","description":"Default > Hive > database","ownerType":{"value":"ROLE","ordinal":2},"qualifiedName":"primary.default","locationUri":"hdfs://localhost:9000/user/hive/warehouse","ownerName":"public","clusterName":"primary"},"traitNames":[],"traits":{}},"operationType":"ENTITY_CREATE","traits":[]} > {code} > The expectation is that we don't have these extraneous events as there is no > change to the entity. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-342) Atlas is sending an ENTITY_CREATE event to the ATLAS_ENTITIES topic even if the entity exists already.
[ https://issues.apache.org/jira/browse/ATLAS-342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063796#comment-17063796 ] ASF subversion and git services commented on ATLAS-342: --- Commit 0eb4562dca581e77508fb04dfb5a13b331939800 in atlas's branch refs/heads/branch-2.0 from mayanknj [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=0eb4562 ] ATLAS-342: Import Glossary Terms from csv/excel file into Glossary Signed-off-by: Sarath Subramanian (cherry picked from commit 784b606ddfa2149535b437483051a68859689a9d) > Atlas is sending an ENTITY_CREATE event to the ATLAS_ENTITIES topic even if > the entity exists already. > -- > > Key: ATLAS-342 > URL: https://issues.apache.org/jira/browse/ATLAS-342 > Project: Atlas > Issue Type: Bug >Reporter: Hemanth Yamijala >Assignee: Shwetha GS >Priority: Major > Fix For: 0.6-incubating > > Attachments: ATLAS-342.patch > > > * Configure Hive hook in Hive. > * Configure Atlas to talk to Kafka topics on a separately set Kafka instance. > * Run a script to create tables in Hive multiple times, for e.g. > {code} > for i in `seq 1 10`; do ./bin/hive -e "create table tbl$i (column${i}1 > string, column${i}2 int)"; done > {code} > * After the script completes, check the number of entities & list the actual > entities in the ATLAS_ENTITIES topic. > We can see one ENTITY_CREATE event for the Hive database for every table > created in Hive. For e.g.: > {code} > {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"e7ed4ad9-1fba-47ad-a089-1ff7e715c1ad","version":0,"typeName":"hive_db"},"typeName":"hive_db","values":{"name":"default","description":"Default > Hive > database","ownerType":{"value":"ROLE","ordinal":2},"qualifiedName":"primary.default","locationUri":"hdfs://localhost:9000/user/hive/warehouse","ownerName":"public","clusterName":"primary"},"traitNames":[],"traits":{}},"operationType":"ENTITY_CREATE","traits":[]} > {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"e7ed4ad9-1fba-47ad-a089-1ff7e715c1ad","version":0,"typeName":"hive_db"},"typeName":"hive_db","values":{"name":"default","description":"Default > Hive > database","ownerType":{"value":"ROLE","ordinal":2},"qualifiedName":"primary.default","locationUri":"hdfs://localhost:9000/user/hive/warehouse","ownerName":"public","clusterName":"primary"},"traitNames":[],"traits":{}},"operationType":"ENTITY_CREATE","traits":[]} > {code} > The expectation is that we don't have these extraneous events as there is no > change to the entity. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-342) Atlas is sending an ENTITY_CREATE event to the ATLAS_ENTITIES topic even if the entity exists already.
[ https://issues.apache.org/jira/browse/ATLAS-342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063795#comment-17063795 ] ASF subversion and git services commented on ATLAS-342: --- Commit 784b606ddfa2149535b437483051a68859689a9d in atlas's branch refs/heads/master from mayanknj [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=784b606 ] ATLAS-342: Import Glossary Terms from csv/excel file into Glossary Signed-off-by: Sarath Subramanian > Atlas is sending an ENTITY_CREATE event to the ATLAS_ENTITIES topic even if > the entity exists already. > -- > > Key: ATLAS-342 > URL: https://issues.apache.org/jira/browse/ATLAS-342 > Project: Atlas > Issue Type: Bug >Reporter: Hemanth Yamijala >Assignee: Shwetha GS >Priority: Major > Fix For: 0.6-incubating > > Attachments: ATLAS-342.patch > > > * Configure Hive hook in Hive. > * Configure Atlas to talk to Kafka topics on a separately set Kafka instance. > * Run a script to create tables in Hive multiple times, for e.g. > {code} > for i in `seq 1 10`; do ./bin/hive -e "create table tbl$i (column${i}1 > string, column${i}2 int)"; done > {code} > * After the script completes, check the number of entities & list the actual > entities in the ATLAS_ENTITIES topic. > We can see one ENTITY_CREATE event for the Hive database for every table > created in Hive. For e.g.: > {code} > {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"e7ed4ad9-1fba-47ad-a089-1ff7e715c1ad","version":0,"typeName":"hive_db"},"typeName":"hive_db","values":{"name":"default","description":"Default > Hive > database","ownerType":{"value":"ROLE","ordinal":2},"qualifiedName":"primary.default","locationUri":"hdfs://localhost:9000/user/hive/warehouse","ownerName":"public","clusterName":"primary"},"traitNames":[],"traits":{}},"operationType":"ENTITY_CREATE","traits":[]} > {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"e7ed4ad9-1fba-47ad-a089-1ff7e715c1ad","version":0,"typeName":"hive_db"},"typeName":"hive_db","values":{"name":"default","description":"Default > Hive > database","ownerType":{"value":"ROLE","ordinal":2},"qualifiedName":"primary.default","locationUri":"hdfs://localhost:9000/user/hive/warehouse","ownerName":"public","clusterName":"primary"},"traitNames":[],"traits":{}},"operationType":"ENTITY_CREATE","traits":[]} > {code} > The expectation is that we don't have these extraneous events as there is no > change to the entity. -- This message was sent by Atlassian Jira (v8.3.4#803005)