[jira] [Updated] (ATLAS-3804) violates a uniqueness constraint [Referenceable.__u_qualifiedName__typeName]
[ https://issues.apache.org/jira/browse/ATLAS-3804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kevin updated ATLAS-3804: - Priority: Blocker (was: Major) > violates a uniqueness constraint [Referenceable.__u_qualifiedName__typeName] > > > Key: ATLAS-3804 > URL: https://issues.apache.org/jira/browse/ATLAS-3804 > Project: Atlas > Issue Type: Bug > Components: hive-integration >Affects Versions: 1.2.0 > Environment: cdh5.14.4 , atlas1.2.0 ,elasticsearch5.6.4,kafka2.1.1 >Reporter: kevin >Priority: Blocker > > {{create table t2 as select id, name from T1; }} > > If the T1 table is not used for 1 hours and the above statement is executed, > Atlas will have a unique constraint exception. > Immediately change table t2 to T3 for execution, Atlas normal parsing record > org.apache.atlas.repository.graphdb.AtlasSchemaViolationException: > org.janusgraph.core.SchemaViolationException: Adding this property for key > [Referenceable.__u_qualifiedName] and value [graph.gdemo3.name@primary] > violates a uniqueness constraint [Referenceable.__u_qualifiedName__typeName] > at > org.apache.atlas.repository.graphdb.janus.AtlasJanusElement.setProperty(AtlasJanusElement.java:114) > at > org.apache.atlas.repository.store.graph.v2.AtlasGraphUtilsV2.setProperty(AtlasGraphUtilsV2.java:227) > at > org.apache.atlas.repository.store.graph.v2.AtlasGraphUtilsV2.setEncodedProperty(AtlasGraphUtilsV2.java:195) > at > org.apache.atlas.repository.store.graph.v2.EntityGraphMapper.mapPrimitiveValue(EntityGraphMapper.java:751) > at > org.apache.atlas.repository.store.graph.v2.EntityGraphMapper.mapToVertexByTypeCategory(EntityGraphMapper.java:435) > at > org.apache.atlas.repository.store.graph.v2.EntityGraphMapper.mapAttribute(EntityGraphMapper.java:424) > at > org.apache.atlas.repository.store.graph.v2.EntityGraphMapper.mapAttributes(EntityGraphMapper.java:336) > at > org.apache.atlas.repository.store.graph.v2.EntityGraphMapper.mapAttributesAndClassifications(EntityGraphMapper.java:229) > at > org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:856) > at > org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:275) > at > org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$FastClassBySpringCGLIB$$6861dca9.invoke() > at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > at > org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) > at > org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:80) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) > at > org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671) > at > org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$EnhancerBySpringCGLIB$$e49957d9.createOrUpdate() > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.createOrUpdate(NotificationHookConsumer.java:701) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:582) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.doWork(NotificationHookConsumer.java:437) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.janusgraph.core.SchemaViolationException: Adding this property > for key [Referenceable.__u_qualifiedName] and value > [graph.gdemo3.name@primary] violates a uniqueness constraint > [Referenceable.__u_qualifiedName__typeName] > at > org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:817) > at > org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:745) > at > org.janusgraph.graphdb.vertices.AbstractVertex.property(AbstractVertex.java:152) > at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:72) > at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:33) > at > org.apache.atlas.repository.graphdb.janus.AtlasJanusElement.setProperty(AtlasJanusElement.java:112) > {{}} -- This message
Re: Review Request 72533: ATLAS-3802 : BasicSearch: filter, attribute with NEQ operator doesn't match attribute having null values
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72533/#review220843 --- Ship it! Ship It! - Madhan Neethiraj On May 21, 2020, 3:42 a.m., Pinal Shah wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72533/ > --- > > (Updated May 21, 2020, 3:42 a.m.) > > > Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath > Subramanian. > > > Bugs: ATLAS-3802 > https://issues.apache.org/jira/browse/ATLAS-3802 > > > Repository: atlas > > > Description > --- > > Issue: When filter with 'attribute' != 'value' is searched, it doesn't fetch > entities where attribute is not present. > > Workaround: In the current implementation attribute = null is ignored, So now > we can allow null when Operator is NEQ. > > > Diffs > - > > repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java > 4408cd449 > repository/src/test/java/org/apache/atlas/BasicTestSetup.java 7508f7855 > > repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java > 95f782abb > > repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java > 1951c3937 > > > Diff: https://reviews.apache.org/r/72533/diff/1/ > > > Testing > --- > > Added test cases and manually tested NEQ operator for below scenarios: > 1) Non String Attribute -> index query > 2) String Attributes -> graph query > 3) pipeSeperated Attributes (classificationNames,labels) -> NOT_CONTAINS > operator -> graph query > 4) businessMetadata attribute > > > Thanks, > > Pinal Shah > >
[jira] [Created] (ATLAS-3804) violates a uniqueness constraint [Referenceable.__u_qualifiedName__typeName]
kevin created ATLAS-3804: Summary: violates a uniqueness constraint [Referenceable.__u_qualifiedName__typeName] Key: ATLAS-3804 URL: https://issues.apache.org/jira/browse/ATLAS-3804 Project: Atlas Issue Type: Bug Components: hive-integration Affects Versions: 1.2.0 Environment: cdh5.14.4 , atlas1.2.0 ,elasticsearch5.6.4,kafka2.1.1 Reporter: kevin {{create table t2 as select id, name from T1; }} If the T1 table is not used for 1 hours and the above statement is executed, Atlas will have a unique constraint exception. Immediately change table t2 to T3 for execution, Atlas normal parsing record org.apache.atlas.repository.graphdb.AtlasSchemaViolationException: org.janusgraph.core.SchemaViolationException: Adding this property for key [Referenceable.__u_qualifiedName] and value [graph.gdemo3.name@primary] violates a uniqueness constraint [Referenceable.__u_qualifiedName__typeName] at org.apache.atlas.repository.graphdb.janus.AtlasJanusElement.setProperty(AtlasJanusElement.java:114) at org.apache.atlas.repository.store.graph.v2.AtlasGraphUtilsV2.setProperty(AtlasGraphUtilsV2.java:227) at org.apache.atlas.repository.store.graph.v2.AtlasGraphUtilsV2.setEncodedProperty(AtlasGraphUtilsV2.java:195) at org.apache.atlas.repository.store.graph.v2.EntityGraphMapper.mapPrimitiveValue(EntityGraphMapper.java:751) at org.apache.atlas.repository.store.graph.v2.EntityGraphMapper.mapToVertexByTypeCategory(EntityGraphMapper.java:435) at org.apache.atlas.repository.store.graph.v2.EntityGraphMapper.mapAttribute(EntityGraphMapper.java:424) at org.apache.atlas.repository.store.graph.v2.EntityGraphMapper.mapAttributes(EntityGraphMapper.java:336) at org.apache.atlas.repository.store.graph.v2.EntityGraphMapper.mapAttributesAndClassifications(EntityGraphMapper.java:229) at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:856) at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:275) at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$FastClassBySpringCGLIB$$6861dca9.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:80) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671) at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$EnhancerBySpringCGLIB$$e49957d9.createOrUpdate() at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.createOrUpdate(NotificationHookConsumer.java:701) at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:582) at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.doWork(NotificationHookConsumer.java:437) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.janusgraph.core.SchemaViolationException: Adding this property for key [Referenceable.__u_qualifiedName] and value [graph.gdemo3.name@primary] violates a uniqueness constraint [Referenceable.__u_qualifiedName__typeName] at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:817) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:745) at org.janusgraph.graphdb.vertices.AbstractVertex.property(AbstractVertex.java:152) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:72) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:33) at org.apache.atlas.repository.graphdb.janus.AtlasJanusElement.setProperty(AtlasJanusElement.java:112) {{}} -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72533: ATLAS-3802 : BasicSearch: filter, attribute with NEQ operator doesn't match attribute having null values
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72533/ --- (Updated May 21, 2020, 3:42 a.m.) Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Bugs: ATLAS-3802 https://issues.apache.org/jira/browse/ATLAS-3802 Repository: atlas Description --- Issue: When filter with 'attribute' != 'value' is searched, it doesn't fetch entities where attribute is not present. Workaround: In the current implementation attribute = null is ignored, So now we can allow null when Operator is NEQ. Diffs - repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 4408cd449 repository/src/test/java/org/apache/atlas/BasicTestSetup.java 7508f7855 repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java 95f782abb repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java 1951c3937 Diff: https://reviews.apache.org/r/72533/diff/1/ Testing --- Added test cases and manually tested NEQ operator for below scenarios: 1) Non String Attribute -> index query 2) String Attributes -> graph query 3) pipeSeperated Attributes (classificationNames,labels) -> NOT_CONTAINS operator -> graph query 4) businessMetadata attribute Thanks, Pinal Shah
[jira] [Commented] (ATLAS-3803) Update tag propagation to NONE for relationshipDefs in aws_s3_v2 and azure_adls_gen2 models
[ https://issues.apache.org/jira/browse/ATLAS-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17112661#comment-17112661 ] Madhan Neethiraj commented on ATLAS-3803: - +1 for the patch. Thanks [~sarath]! > Update tag propagation to NONE for relationshipDefs in aws_s3_v2 and > azure_adls_gen2 models > --- > > Key: ATLAS-3803 > URL: https://issues.apache.org/jira/browse/ATLAS-3803 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Fix For: 2.1.0 > > Attachments: ATLAS-3803.001.patch, ATLAS-3803.002.patch > > > The tag propagation value for AWS S3 model between "aws_s3_v2_container" and > "aws_s3_v2_contained" is ONE_TO_TWO. This change is to change the default > propagation direction to NONE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3803) Update tag propagation to NONE for relationshipDefs in aws_s3_v2 and azure_adls_gen2 models
[ https://issues.apache.org/jira/browse/ATLAS-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17112657#comment-17112657 ] Sarath Subramanian commented on ATLAS-3803: --- [^ATLAS-3803.002.patch] updates azure adls models realtionshipdefs (tag propagation to NONE) - *adls_gen2_account_containers* and *adls_gen2_parent_children* > Update tag propagation to NONE for relationshipDefs in aws_s3_v2 and > azure_adls_gen2 models > --- > > Key: ATLAS-3803 > URL: https://issues.apache.org/jira/browse/ATLAS-3803 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Fix For: 2.1.0 > > Attachments: ATLAS-3803.001.patch, ATLAS-3803.002.patch > > > The tag propagation value for AWS S3 model between "aws_s3_v2_container" and > "aws_s3_v2_contained" is ONE_TO_TWO. This change is to change the default > propagation direction to NONE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3803) Update tag propagation to NONE for relationshipDefs in aws_s3_v2 and azure_adls_gen2 models
[ https://issues.apache.org/jira/browse/ATLAS-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-3803: -- Attachment: ATLAS-3803.002.patch > Update tag propagation to NONE for relationshipDefs in aws_s3_v2 and > azure_adls_gen2 models > --- > > Key: ATLAS-3803 > URL: https://issues.apache.org/jira/browse/ATLAS-3803 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Fix For: 2.1.0 > > Attachments: ATLAS-3803.001.patch, ATLAS-3803.002.patch > > > The tag propagation value for AWS S3 model between "aws_s3_v2_container" and > "aws_s3_v2_contained" is ONE_TO_TWO. This change is to change the default > propagation direction to NONE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3803) Update tag propagation to NONE for relationshipDefs in aws_s3_v2 and azure_adls_gen2 models
[ https://issues.apache.org/jira/browse/ATLAS-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-3803: -- Summary: Update tag propagation to NONE for relationshipDefs in aws_s3_v2 and azure_adls_gen2 models (was: Update tag propagation to NONE between aws_s3_v2_container and aws_s3_v2_contained type) > Update tag propagation to NONE for relationshipDefs in aws_s3_v2 and > azure_adls_gen2 models > --- > > Key: ATLAS-3803 > URL: https://issues.apache.org/jira/browse/ATLAS-3803 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Fix For: 2.1.0 > > Attachments: ATLAS-3803.001.patch, ATLAS-3803.002.patch > > > The tag propagation value for AWS S3 model between "aws_s3_v2_container" and > "aws_s3_v2_contained" is ONE_TO_TWO. This change is to change the default > propagation direction to NONE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3803) Update tag propagation to NONE between aws_s3_v2_container and aws_s3_v2_contained type
[ https://issues.apache.org/jira/browse/ATLAS-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17112577#comment-17112577 ] Madhan Neethiraj commented on ATLAS-3803: - +1 for this update. This update is inline with other relationships in existing models, like: hive_table_db, hive_table_columns, hbase_table_namespace, hbase_table_column_families, hbase_column_family_columns. I suggest updating following relationship types as well: adls_gen2_account_containers, adls_gen2_parent_children. > Update tag propagation to NONE between aws_s3_v2_container and > aws_s3_v2_contained type > --- > > Key: ATLAS-3803 > URL: https://issues.apache.org/jira/browse/ATLAS-3803 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Fix For: 2.1.0 > > Attachments: ATLAS-3803.001.patch > > > The tag propagation value for AWS S3 model between "aws_s3_v2_container" and > "aws_s3_v2_contained" is ONE_TO_TWO. This change is to change the default > propagation direction to NONE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3803) Update tag propagation to NONE between aws_s3_v2_container and aws_s3_v2_contained type
[ https://issues.apache.org/jira/browse/ATLAS-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-3803: -- Attachment: ATLAS-3803.001.patch > Update tag propagation to NONE between aws_s3_v2_container and > aws_s3_v2_contained type > --- > > Key: ATLAS-3803 > URL: https://issues.apache.org/jira/browse/ATLAS-3803 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.0.0 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Fix For: 2.1.0 > > Attachments: ATLAS-3803.001.patch > > > The tag propagation value for AWS S3 model between "aws_s3_v2_container" and > "aws_s3_v2_contained" is ONE_TO_TWO. This change is to change the default > propagation direction to NONE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-3803) Update tag propagation to NONE between aws_s3_v2_container and aws_s3_v2_contained type
Sarath Subramanian created ATLAS-3803: - Summary: Update tag propagation to NONE between aws_s3_v2_container and aws_s3_v2_contained type Key: ATLAS-3803 URL: https://issues.apache.org/jira/browse/ATLAS-3803 Project: Atlas Issue Type: Improvement Components: atlas-core Affects Versions: 2.0.0 Reporter: Sarath Subramanian Assignee: Sarath Subramanian Fix For: 2.1.0 The tag propagation value for AWS S3 model between "aws_s3_v2_container" and "aws_s3_v2_contained" is ONE_TO_TWO. This change is to change the default propagation direction to NONE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3802) BasicSearch: filter, attribute with NEQ operator doesn't match attribute having null values
[ https://issues.apache.org/jira/browse/ATLAS-3802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pinal updated ATLAS-3802: - Description: When filter with 'attribute' != 'value' is searched, it doesn't fetch entities where attribute is not present. Eg. propagatedClassification != tag1 is searched, it fetches all entities not tagged to tag1 , but it will not match to entities which don't have *any* propagated tags was: When propagatedClassification != tag1 is searched, it fetches all entities not tagged to tag1 But it will not match to entities which don't have any propagated tags > BasicSearch: filter, attribute with NEQ operator doesn't match attribute > having null values > --- > > Key: ATLAS-3802 > URL: https://issues.apache.org/jira/browse/ATLAS-3802 > Project: Atlas > Issue Type: Bug >Reporter: Pinal >Assignee: Pinal >Priority: Major > > When filter with 'attribute' != 'value' is searched, it doesn't fetch > entities where attribute is not present. > Eg. propagatedClassification != tag1 is searched, it fetches all entities not > tagged to tag1 , but it will not match to entities which don't have *any* > propagated tags -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-3802) BasicSearch: filter, attribute with NEQ operator doesn't match attribute having null values
Pinal created ATLAS-3802: Summary: BasicSearch: filter, attribute with NEQ operator doesn't match attribute having null values Key: ATLAS-3802 URL: https://issues.apache.org/jira/browse/ATLAS-3802 Project: Atlas Issue Type: Bug Reporter: Pinal Assignee: Pinal When propagatedClassification != tag1 is searched, it fetches all entities not tagged to tag1 But it will not match to entities which don't have any propagated tags -- This message was sent by Atlassian Jira (v8.3.4#803005)