[jira] [Commented] (ATLAS-4749) What is the meaning of version in the JSON content of creating entities, and how to update version when updating entity content
[
https://issues.apache.org/jira/browse/ATLAS-4749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720007#comment-17720007
]
Sarath Subramanian commented on ATLAS-4749:
---
[~zhangkai] version is modeled as an internal field in atlas and not meant to
be an updated field. We suggest you add a new attribute to capture version.
> What is the meaning of version in the JSON content of creating entities, and
> how to update version when updating entity content
> ---
>
> Key: ATLAS-4749
> URL: https://issues.apache.org/jira/browse/ATLAS-4749
> Project: Atlas
> Issue Type: Task
>Affects Versions: 2.3.0
>Reporter: zhangkai
>Priority: Major
>
> Example:
> 1. Create Entity ("version"=20)
> 2. Query created entities
> 3. Update Entity ("version"=30)
> 4. Query the updated entity and find that version=20 has not changed
> Create an entity's JSON
> {
> "entity": {
> "typeName": "my_db_new",
> "attributes": {
> "description": "database",
> "clustername": "T2@TEST",
> "name": "all_dbs",
> "qualifiedName": "T2@TEST",
> "attr2": "test"
> },
> "version": 20
> }
> }
> Update entity's JSON
> {
> "entity": {
> "typeName": "my_db_new",
> "attributes": {
> "description": "database",
> "clustername": "T2@TEST",
> "name": "all_dbs_update",
> "qualifiedName": "T2@TEST",
> "attr2": "test"
> },
> "version": 30
> }
> }
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ATLAS-4748) 创建实体的JSON内容中版本是什么含义，更新实体内容时如何更新版本
[
https://issues.apache.org/jira/browse/ATLAS-4748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian resolved ATLAS-4748.
---
Resolution: Incomplete
> 创建实体的JSON内容中版本是什么含义，更新实体内容时如何更新版本
> -
>
> Key: ATLAS-4748
> URL: https://issues.apache.org/jira/browse/ATLAS-4748
> Project: Atlas
> Issue Type: Task
> Components: atlas-core
>Affects Versions: 2.3.0
>Reporter: zhangkai
>Priority: Major
>
> 创建实体的JSON内容中版本是什么含义，更新实体内容时如何更新版本
>
> 例：
> 1. 创建实体（“版本”=20）
> 2. 查询创建的实体
> 3. 更新实体（“版本”=30）
> 4. 查询更新的实体，发现 version=20 未更改
>
> 创建实体的 JSON
> {
> “实体”：{
> “类型名称”： “my_db_new”，
> “属性”：{
> “描述”： “数据库”，
> “集群名称”： “T2@TEST”，
> “名称”： “all_dbs”，
> “限定名称”： “T2@TEST”，
> “attr2”： “测试”
> },
> “版本”：20
> }
> }
>
>
> 更新实体的 JSON
> {
> “实体”：{
> “类型名称”： “my_db_new”，
> “属性”：{
> “描述”： “数据库”，
> “集群名称”： “T2@TEST”，
> “名称”： “all_dbs_update”，
> “限定名称”： “T2@TEST”，
> “attr2”： “测试”
> },
> “版本”：30
> }
> }
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ATLAS-4540) Upgrade JanusGraph to version 0.6.1
[ https://issues.apache.org/jira/browse/ATLAS-4540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17482043#comment-17482043 ] Sarath Subramanian commented on ATLAS-4540: --- Thanks for the patch [~madhan]. +1 > Upgrade JanusGraph to version 0.6.1 > --- > > Key: ATLAS-4540 > URL: https://issues.apache.org/jira/browse/ATLAS-4540 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj >Priority: Major > Attachments: ATLAS-4540.patch > > > Upgrade Atlas to use recent JanusGraph version, 0.6.1 released on Jan-18-2022. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (ATLAS-4536) The newline character in attribues will fail simple auth check
[
https://issues.apache.org/jira/browse/ATLAS-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4536:
--
Fix Version/s: 3.0.0
2.3.0
> The newline character in attribues will fail simple auth check
> --
>
> Key: ATLAS-4536
> URL: https://issues.apache.org/jira/browse/ATLAS-4536
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Garry Easop
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> When using Atlas simple authorization and the default json policy file,
> attributes can cause a 403 errors. This is because Atlas ends up calling
> isMatch() and if the string to be matched has a newline character isMatch
> will return false, which leads to the 403. The error in the log looks like:
> {code:java}
> 2021-11-17 22:03:30,328 ERROR - [pool-2-thread-4 -
> c347ce48-4f16-45eb-9453-6d49dde3eb9e:] ~ graph rollback due to exception
> (GraphTransactionInterceptor:167)
> org.apache.atlas.exception.AtlasBaseException: admin is not authorized to
> perform read entity: guid=da8c1532-1aa7-4734-bab3-1567f8565ed3
> at
> org.apache.atlas.authorize.AtlasAuthorizationUtils.verifyAccess(AtlasAuthorizationUtils.java:62)
> at
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.getById(AtlasEntityStoreV2.java:128)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (ATLAS-4536) The newline character in attribues will fail simple auth check
[
https://issues.apache.org/jira/browse/ATLAS-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian resolved ATLAS-4536.
---
Resolution: Fixed
> The newline character in attribues will fail simple auth check
> --
>
> Key: ATLAS-4536
> URL: https://issues.apache.org/jira/browse/ATLAS-4536
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Garry Easop
>Priority: Major
>
> When using Atlas simple authorization and the default json policy file,
> attributes can cause a 403 errors. This is because Atlas ends up calling
> isMatch() and if the string to be matched has a newline character isMatch
> will return false, which leads to the 403. The error in the log looks like:
> {code:java}
> 2021-11-17 22:03:30,328 ERROR - [pool-2-thread-4 -
> c347ce48-4f16-45eb-9453-6d49dde3eb9e:] ~ graph rollback due to exception
> (GraphTransactionInterceptor:167)
> org.apache.atlas.exception.AtlasBaseException: admin is not authorized to
> perform read entity: guid=da8c1532-1aa7-4734-bab3-1567f8565ed3
> at
> org.apache.atlas.authorize.AtlasAuthorizationUtils.verifyAccess(AtlasAuthorizationUtils.java:62)
> at
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.getById(AtlasEntityStoreV2.java:128)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (ATLAS-4522) Updating typedef with new supertype should be allowed only if attributes are unique compared to other existing supertypes
[
https://issues.apache.org/jira/browse/ATLAS-4522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4522:
--
Fix Version/s: 3.0.0
2.3.0
> Updating typedef with new supertype should be allowed only if attributes are
> unique compared to other existing supertypes
> -
>
> Key: ATLAS-4522
> URL: https://issues.apache.org/jira/browse/ATLAS-4522
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Radhika Kundam
>Assignee: Radhika Kundam
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> 1.Typedef1 with attributes \{a,b,c}
> 2. Typedef2 with attributes \{d} and superType\{Typedef1}
> 3. We created entities for these and everything is fine until here
> 4. Typedef3 with attributes\{a,b,c} => Attributes are same as Typedef1
> 5. Now we updated Typedef2 with one more supertype as below
> Typedef2 with superType\{Typedef1, Typedef3}
> 6. All the Typedef2 entities are having \{a,b,c} attribute values as null and
> because of that when user is searching entity of Typedef2 by it's name not
> returning the result.
> Root cause of the issue is that attributes in Typedef2 from SuperType-
> Typedef1 be overridden by SuperType-Typedef2 attributes as attribute names
> are same for Typedef1 & Typedef2
> Updating typedef with new super types should be allowed only if attributes of
> new supertype are unique and not present in any other existing supertypes.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (ATLAS-4485) Can HA support multi write/read?
[ https://issues.apache.org/jira/browse/ATLAS-4485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17446029#comment-17446029 ] Sarath Subramanian commented on ATLAS-4485: --- [~wzg547228197] , yes Atlas HA follows active/passive model where automatic failover happens when primary instance crashes. We don't currently support read replicas. > Can HA support multi write/read? > > > Key: ATLAS-4485 > URL: https://issues.apache.org/jira/browse/ATLAS-4485 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.2.0 >Reporter: wuzhiguo >Priority: Major > Labels: features, performance > > Currently HA is like a backup mode, the write/read operation can only go > through main instance, If main instance crashed because of sudden increase in > traffic, the backup instance will meet same problem. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (ATLAS-4381) Release Atlas 2.2.0
[ https://issues.apache.org/jira/browse/ATLAS-4381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian resolved ATLAS-4381. --- Resolution: Fixed > Release Atlas 2.2.0 > --- > > Key: ATLAS-4381 > URL: https://issues.apache.org/jira/browse/ATLAS-4381 > Project: Atlas > Issue Type: Task >Reporter: Sidharth Kumar Mishra >Assignee: Sidharth Kumar Mishra >Priority: Major > > parent task to capture release tasks for Apache Atlas 2.2.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-3972) Apache atlas 2.1.0
[ https://issues.apache.org/jira/browse/ATLAS-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian resolved ATLAS-3972. --- Resolution: Fixed > Apache atlas 2.1.0 > -- > > Key: ATLAS-3972 > URL: https://issues.apache.org/jira/browse/ATLAS-3972 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Frédéric Comte >Priority: Major > > I am using keycloak integration for authentication. Atlas is behind a reverse > proxy (HAProxy) that doing SSL termination. HAProxy send the header > X-Forwarded-Proto : https but Atlas does't not create an URL with https > scheme when constructing a redirect uri -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4408) Dynamic handling of failure in updating index
[ https://issues.apache.org/jira/browse/ATLAS-4408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4408: -- Affects Version/s: 3.0.0 2.2.0 > Dynamic handling of failure in updating index > - > > Key: ATLAS-4408 > URL: https://issues.apache.org/jira/browse/ATLAS-4408 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: 3.0.0, 2.2.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Attachments: IndexRecovery.png, IndexRecovery_FunctionalFlow.png > > > *Index failure resilience:* dynamic handling of failure in updating index > (i.e. HBase commit succeeds but index commit fails). > In case of secondary persistence failure scenario, there will be > inconsistency with indexes for all the transactions failed at Solr. And to > repair that, the existing option is re-indexing all the data which is time > consuming as it involves indexing the entire database. > To recover such inconsistencies we can use the *transaction write-ahead log > option*. By enabling write-ahead log(tx.log-tx), JanusGraph maintains all the > transaction log data which can be used to recover indices in case of > failures. With this approach, it’s extra overhead to maintain the log data > for all transactions but with this approach we can guarantee the system is > more resilient and proactive. So advantages of this approach can nullify the > overhead of maintaining log data. > Design details as below. > # Start new service - IndexRecoveryService at Atlas startup. > ## Continuously monitor for Solr(Index Client) health for every retryTime > millisecs > ### If Solr is healthy and recovery start time is available, > Start Transaction Recovery with available recovery start time(which is > noted when Solr became unhealthy) > Persist current recovery time as previous which can be used later by > passing as custom recovery time to start index recovery if required. > Reset current recovery start time > Continue with Solr health checkup. > ### If Solr is unhealthy and no recovery start time is available, > Shutdown the existing transaction recovery process. > Note down the time which should be the next recovery start time and > persist in graph. > Continue with Solr health checkup. > Configuration properties to be used for this feature. > 1.To enable or disable index recovery(By default index recovery will be > enabled on Atlas startup) > *atlas.graph.enable.index.recovery=true* > 2.To configure how frequently SOLR health check should be done > *atlas.graph.index.search.solr.status.retry.interval=* > 3.To start index recovery by custom recovery time as user provided > *atlas.graph.index.search.solr.recovery.start.time=1630086622* > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4408) Dynamic handling of failure in updating index
[ https://issues.apache.org/jira/browse/ATLAS-4408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4408: -- Labels: indexing (was: ) > Dynamic handling of failure in updating index > - > > Key: ATLAS-4408 > URL: https://issues.apache.org/jira/browse/ATLAS-4408 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: 3.0.0, 2.2.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Labels: indexing > Fix For: 3.0.0, 2.3.0 > > Attachments: IndexRecovery.png, IndexRecovery_FunctionalFlow.png > > > *Index failure resilience:* dynamic handling of failure in updating index > (i.e. HBase commit succeeds but index commit fails). > In case of secondary persistence failure scenario, there will be > inconsistency with indexes for all the transactions failed at Solr. And to > repair that, the existing option is re-indexing all the data which is time > consuming as it involves indexing the entire database. > To recover such inconsistencies we can use the *transaction write-ahead log > option*. By enabling write-ahead log(tx.log-tx), JanusGraph maintains all the > transaction log data which can be used to recover indices in case of > failures. With this approach, it’s extra overhead to maintain the log data > for all transactions but with this approach we can guarantee the system is > more resilient and proactive. So advantages of this approach can nullify the > overhead of maintaining log data. > Design details as below. > # Start new service - IndexRecoveryService at Atlas startup. > ## Continuously monitor for Solr(Index Client) health for every retryTime > millisecs > ### If Solr is healthy and recovery start time is available, > Start Transaction Recovery with available recovery start time(which is > noted when Solr became unhealthy) > Persist current recovery time as previous which can be used later by > passing as custom recovery time to start index recovery if required. > Reset current recovery start time > Continue with Solr health checkup. > ### If Solr is unhealthy and no recovery start time is available, > Shutdown the existing transaction recovery process. > Note down the time which should be the next recovery start time and > persist in graph. > Continue with Solr health checkup. > Configuration properties to be used for this feature. > 1.To enable or disable index recovery(By default index recovery will be > enabled on Atlas startup) > *atlas.graph.enable.index.recovery=true* > 2.To configure how frequently SOLR health check should be done > *atlas.graph.index.search.solr.status.retry.interval=* > 3.To start index recovery by custom recovery time as user provided > *atlas.graph.index.search.solr.recovery.start.time=1630086622* > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4408) Dynamic handling of failure in updating index
[ https://issues.apache.org/jira/browse/ATLAS-4408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4408: -- Fix Version/s: 2.3.0 3.0.0 > Dynamic handling of failure in updating index > - > > Key: ATLAS-4408 > URL: https://issues.apache.org/jira/browse/ATLAS-4408 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: 3.0.0, 2.2.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: IndexRecovery.png, IndexRecovery_FunctionalFlow.png > > > *Index failure resilience:* dynamic handling of failure in updating index > (i.e. HBase commit succeeds but index commit fails). > In case of secondary persistence failure scenario, there will be > inconsistency with indexes for all the transactions failed at Solr. And to > repair that, the existing option is re-indexing all the data which is time > consuming as it involves indexing the entire database. > To recover such inconsistencies we can use the *transaction write-ahead log > option*. By enabling write-ahead log(tx.log-tx), JanusGraph maintains all the > transaction log data which can be used to recover indices in case of > failures. With this approach, it’s extra overhead to maintain the log data > for all transactions but with this approach we can guarantee the system is > more resilient and proactive. So advantages of this approach can nullify the > overhead of maintaining log data. > Design details as below. > # Start new service - IndexRecoveryService at Atlas startup. > ## Continuously monitor for Solr(Index Client) health for every retryTime > millisecs > ### If Solr is healthy and recovery start time is available, > Start Transaction Recovery with available recovery start time(which is > noted when Solr became unhealthy) > Persist current recovery time as previous which can be used later by > passing as custom recovery time to start index recovery if required. > Reset current recovery start time > Continue with Solr health checkup. > ### If Solr is unhealthy and no recovery start time is available, > Shutdown the existing transaction recovery process. > Note down the time which should be the next recovery start time and > persist in graph. > Continue with Solr health checkup. > Configuration properties to be used for this feature. > 1.To enable or disable index recovery(By default index recovery will be > enabled on Atlas startup) > *atlas.graph.enable.index.recovery=true* > 2.To configure how frequently SOLR health check should be done > *atlas.graph.index.search.solr.status.retry.interval=* > 3.To start index recovery by custom recovery time as user provided > *atlas.graph.index.search.solr.recovery.start.time=1630086622* > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4246) Make Kafka Interface aware of Kafka Schema Registry
[ https://issues.apache.org/jira/browse/ATLAS-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17425734#comment-17425734 ] Sarath Subramanian commented on ATLAS-4246: --- [~aileeen], added some review comments. Please review/update. > Make Kafka Interface aware of Kafka Schema Registry > --- > > Key: ATLAS-4246 > URL: https://issues.apache.org/jira/browse/ATLAS-4246 > Project: Atlas > Issue Type: Improvement > Components: kafka-integration >Affects Versions: 2.1.0, 3.0.0 >Reporter: Aileen Toleikis >Assignee: Viktor Somogyi-Vass >Priority: Major > Labels: Kafka > Fix For: 3.0.0, 2.3.0 > > > Kafka Community is using Schema Registry more and more heavily but as Atlas > is currently unaware of this, this extension helps Atlas make use of the > Schemas. > > We have tested this extension and we have production environments where Atlas > will not be allowed without schema registry access. We have received feedback > that this extension would be sufficient to allow production use. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4440) Upgrade Atlas's Kafka dependency to 2.8
[ https://issues.apache.org/jira/browse/ATLAS-4440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17424224#comment-17424224 ] Sarath Subramanian commented on ATLAS-4440: --- [~viktorsomogyi], is this Kafka version backward compatible? Is there any compatibility issue when older Kafka client (hook) communicates with newer Kafka version or vice-versa? > Upgrade Atlas's Kafka dependency to 2.8 > --- > > Key: ATLAS-4440 > URL: https://issues.apache.org/jira/browse/ATLAS-4440 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Viktor Somogyi-Vass >Assignee: Viktor Somogyi-Vass >Priority: Major > Attachments: 0001-ATLAS-4440-Update-Kafka-dependency-to-2.8.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4440) Upgrade Atlas's Kafka dependency to 2.8
[ https://issues.apache.org/jira/browse/ATLAS-4440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17424223#comment-17424223 ] Sarath Subramanian commented on ATLAS-4440: --- Thanks for the Precommit run [~amestry]. +1 > Upgrade Atlas's Kafka dependency to 2.8 > --- > > Key: ATLAS-4440 > URL: https://issues.apache.org/jira/browse/ATLAS-4440 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Viktor Somogyi-Vass >Assignee: Viktor Somogyi-Vass >Priority: Major > Attachments: 0001-ATLAS-4440-Update-Kafka-dependency-to-2.8.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4351) Maven-jetty throwing warnings when running Integration tests for conflicting jars in classpath
[ https://issues.apache.org/jira/browse/ATLAS-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4351: -- Component/s: atlas-core > Maven-jetty throwing warnings when running Integration tests for conflicting > jars in classpath > --- > > Key: ATLAS-4351 > URL: https://issues.apache.org/jira/browse/ATLAS-4351 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 3.0.0, 2.2.0 >Reporter: chaitali borole >Assignee: chaitali borole >Priority: Major > Fix For: 3.0.0 > > > Below warnings are thrown when integration tests executed : > [INFO] jetty-9.4.31.v20200723; built: 2020-07-23T17:57:36.812Z; git: > 450ba27947e13e66baa8cd1ce7e85a4461cacc1d; jvm 1.8.0_291-b10 > [WARNING] javax.activation.ActivationDataFlavor scanned from multiple > locations: jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/jakarta.activation-api-1.2.1.jar!/javax/activation/ActivationDataFlavor.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/javax.activation-api-1.2.0.jar!/javax/activation/ActivationDataFlavor.class] > [WARNING] com.google.common.base.Stopwatch$1 scanned from multiple locations: > jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/guava-25.1-jre.jar!/com/google/common/base/Stopwatch$1.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/janusgraph-hbase-0.5.3.jar!/com/google/common/base/Stopwatch$1.class] > [WARNING] javax.servlet.AsyncContext scanned from multiple locations: jar: > [file:///home/jenkins/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncContext.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncContext.class] > [WARNING] javax.servlet.AsyncEvent scanned from multiple locations: jar: > [file:///home/jenkins/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncEvent.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncEvent.class] > [WARNING] javax.servlet.AsyncListener scanned from multiple locations: jar: > [file:///home/jenkins/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncListener.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncListener.class] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4351) Maven-jetty throwing warnings when running Integration tests for conflicting jars in classpath
[ https://issues.apache.org/jira/browse/ATLAS-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4351: -- Affects Version/s: 2.2.0 > Maven-jetty throwing warnings when running Integration tests for conflicting > jars in classpath > --- > > Key: ATLAS-4351 > URL: https://issues.apache.org/jira/browse/ATLAS-4351 > Project: Atlas > Issue Type: Improvement >Affects Versions: 3.0.0, 2.2.0 >Reporter: chaitali borole >Assignee: chaitali borole >Priority: Major > Fix For: 3.0.0 > > > Below warnings are thrown when integration tests executed : > [INFO] jetty-9.4.31.v20200723; built: 2020-07-23T17:57:36.812Z; git: > 450ba27947e13e66baa8cd1ce7e85a4461cacc1d; jvm 1.8.0_291-b10 > [WARNING] javax.activation.ActivationDataFlavor scanned from multiple > locations: jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/jakarta.activation-api-1.2.1.jar!/javax/activation/ActivationDataFlavor.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/javax.activation-api-1.2.0.jar!/javax/activation/ActivationDataFlavor.class] > [WARNING] com.google.common.base.Stopwatch$1 scanned from multiple locations: > jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/guava-25.1-jre.jar!/com/google/common/base/Stopwatch$1.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/janusgraph-hbase-0.5.3.jar!/com/google/common/base/Stopwatch$1.class] > [WARNING] javax.servlet.AsyncContext scanned from multiple locations: jar: > [file:///home/jenkins/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncContext.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncContext.class] > [WARNING] javax.servlet.AsyncEvent scanned from multiple locations: jar: > [file:///home/jenkins/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncEvent.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncEvent.class] > [WARNING] javax.servlet.AsyncListener scanned from multiple locations: jar: > [file:///home/jenkins/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncListener.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncListener.class] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4421) [Atlas: Hive Import] When import-hive is run with incorrect input the information is not conveyed to the user
[
https://issues.apache.org/jira/browse/ATLAS-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4421:
--
Component/s: atlas-core
> [Atlas: Hive Import] When import-hive is run with incorrect input the
> information is not conveyed to the user
> -
>
> Key: ATLAS-4421
> URL: https://issues.apache.org/jira/browse/ATLAS-4421
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.2.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Disha Talreja
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: ATLAS-4421.patch
>
>
> {code:java}
> [root@quasar-cxwzxp-3 bin]#
> /opt/cloudera/parcels/CDH/lib/atlas/hook-bin/import-hive.sh -f /tmp/file2.txt
> ...
> Log file for import is /var/log/atlas/import-hive.log
> ...
> Hive Meta Data imported successfully!!! {code}
> In the above example, file */tmp/file2.txt* does not exists.
> The log file is not created in the expected location
> /var/log/atlas/import-hive.log which is tracked by Jira**
> Here, this gives the user an impression that the import is success while
> nothing has actually happened.
> It would be good to convey that no data was imported in such cases
> This is true even when the database name/pattern provided to -d or table
> name/pattern provided to to -t are incorrect
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4421) [Atlas: Hive Import] When import-hive is run with incorrect input the information is not conveyed to the user
[
https://issues.apache.org/jira/browse/ATLAS-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4421:
--
Labels: hive (was: )
> [Atlas: Hive Import] When import-hive is run with incorrect input the
> information is not conveyed to the user
> -
>
> Key: ATLAS-4421
> URL: https://issues.apache.org/jira/browse/ATLAS-4421
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.2.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Disha Talreja
>Priority: Major
> Labels: hive
> Fix For: 3.0.0, 2.3.0
>
> Attachments: ATLAS-4421.patch
>
>
> {code:java}
> [root@quasar-cxwzxp-3 bin]#
> /opt/cloudera/parcels/CDH/lib/atlas/hook-bin/import-hive.sh -f /tmp/file2.txt
> ...
> Log file for import is /var/log/atlas/import-hive.log
> ...
> Hive Meta Data imported successfully!!! {code}
> In the above example, file */tmp/file2.txt* does not exists.
> The log file is not created in the expected location
> /var/log/atlas/import-hive.log which is tracked by Jira**
> Here, this gives the user an impression that the import is success while
> nothing has actually happened.
> It would be good to convey that no data was imported in such cases
> This is true even when the database name/pattern provided to -d or table
> name/pattern provided to to -t are incorrect
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4421) [Atlas: Hive Import] When import-hive is run with incorrect input the information is not conveyed to the user
[
https://issues.apache.org/jira/browse/ATLAS-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4421:
--
Fix Version/s: 2.3.0
3.0.0
> [Atlas: Hive Import] When import-hive is run with incorrect input the
> information is not conveyed to the user
> -
>
> Key: ATLAS-4421
> URL: https://issues.apache.org/jira/browse/ATLAS-4421
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Disha Talreja
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: ATLAS-4421.patch
>
>
> {code:java}
> [root@quasar-cxwzxp-3 bin]#
> /opt/cloudera/parcels/CDH/lib/atlas/hook-bin/import-hive.sh -f /tmp/file2.txt
> ...
> Log file for import is /var/log/atlas/import-hive.log
> ...
> Hive Meta Data imported successfully!!! {code}
> In the above example, file */tmp/file2.txt* does not exists.
> The log file is not created in the expected location
> /var/log/atlas/import-hive.log which is tracked by Jira**
> Here, this gives the user an impression that the import is success while
> nothing has actually happened.
> It would be good to convey that no data was imported in such cases
> This is true even when the database name/pattern provided to -d or table
> name/pattern provided to to -t are incorrect
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4421) [Atlas: Hive Import] When import-hive is run with incorrect input the information is not conveyed to the user
[
https://issues.apache.org/jira/browse/ATLAS-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4421:
--
Affects Version/s: 2.2.0
> [Atlas: Hive Import] When import-hive is run with incorrect input the
> information is not conveyed to the user
> -
>
> Key: ATLAS-4421
> URL: https://issues.apache.org/jira/browse/ATLAS-4421
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Disha Talreja
>Priority: Major
> Attachments: ATLAS-4421.patch
>
>
> {code:java}
> [root@quasar-cxwzxp-3 bin]#
> /opt/cloudera/parcels/CDH/lib/atlas/hook-bin/import-hive.sh -f /tmp/file2.txt
> ...
> Log file for import is /var/log/atlas/import-hive.log
> ...
> Hive Meta Data imported successfully!!! {code}
> In the above example, file */tmp/file2.txt* does not exists.
> The log file is not created in the expected location
> /var/log/atlas/import-hive.log which is tracked by Jira**
> Here, this gives the user an impression that the import is success while
> nothing has actually happened.
> It would be good to convey that no data was imported in such cases
> This is true even when the database name/pattern provided to -d or table
> name/pattern provided to to -t are incorrect
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ATLAS-4358) Mapping for some internal Atlas attributes ( like __patch.type , __timestamp, etc) does not exist in Elasticsearch
[
https://issues.apache.org/jira/browse/ATLAS-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17417946#comment-17417946
]
Sarath Subramanian commented on ATLAS-4358:
---
Thanks for the details [~mehtaanshul]. I have a question:
Internal patch attributes like "__createdBy, __modifiedBy, __timestamp,
__modificationTimestamp, __patch.id, __patch.description, __patch.type,
__patch.action, __patch.state" are created in GraphBackedSearchIndexer which is
Spring annotated component with Order(1)
The AtlasPatchService has an Order(3), so the internal indexes should be
created before Patch Manager is initialized right? Do you see a race scenario
where the init() happens during dependency injection of
AtlasPatchManager in AtlasPatchService?
> Mapping for some internal Atlas attributes ( like __patch.type , __timestamp,
> etc) does not exist in Elasticsearch
> --
>
> Key: ATLAS-4358
> URL: https://issues.apache.org/jira/browse/ATLAS-4358
> Project: Atlas
> Issue Type: Bug
>Reporter: Anshul Mehta
>Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> *Impact of the problem -*
> * Atlas pod taking much longer to become active and this time keep
> increasing as the assets increase. This basically means a downtime on every
> Atlas release.
> * Not able to filter via basic search on attributes like {{__timestamp}} ,
> {{__modificationTimestamp}} , {{createdBy}} and {{modifiedBy}} .
> *Issue -*
> So just before creating the mapping in the mixed index (ES index) Atlas
> creates something called {{propertyKey}} and this propertyKey is used to
> create the mapping. The code is written in a way that checks if propertyKey
> for the current property is null or not. If it is null it creates the
> propertyKey and then adds it to mixed index. If it is not Null it assumes
> that the property has already been added to the index and so skips adding it.
> Now in our case when Atlas checked the propertyKey it was not null (which
> should not have been the case) therefore Atlas skipped adding it to the mixed
> index and so these properties never got added to the mixed index. This
> basically meant propertyKey for these properties were getting created
> somewhere else. We looked into the entire codebase but could not find the use
> of makePropertyKey method ( which is used to create propertyKey) or any other
> similar method.
> Then I saw certain java patch vertices getting created even before these
> internal attributes are added to various indices. Though these patches were
> applied later once all internal attributes were added to all the indices.
> Now, these patch vertices have 9 attributes and we releaized these 9
> attributes are the only attributes missing from ES. So basically when patch
> vertices got created and these vertices with their attributes got added to
> cassandra via janusgraph, janusgraph automatically created propertyKey for
> all these attributes (the janusgraph's makePropertyKey method is not called
> during this process anywhere in the Atlas code). And because internal
> attributes were getting added to indices in another thread at the same time,
> when code checked for propertyKey, it was not null and so it did not add the
> property to the mixed index.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4431) Random NPE when retrieving tasks
[
https://issues.apache.org/jira/browse/ATLAS-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4431:
--
Fix Version/s: 2.3.0
3.0.0
> Random NPE when retrieving tasks
>
>
> Key: ATLAS-4431
> URL: https://issues.apache.org/jira/browse/ATLAS-4431
> Project: Atlas
> Issue Type: Bug
> Components: atlas-intg
>Affects Versions: 2.2.0
>Reporter: Disha Talreja
>Assignee: Disha Talreja
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: ATLAS-4431.patch
>
>
> System tests fail randomly because retrieving tasks using the "admin/tasks"
> endpoint sometimes returns HTTP 500.
> Atlas logs:
> {noformat}
> 2021-09-06 13:51:28,938 INFO - [etp985324122-24:] ~ Logged into Atlas as =
> hrt_qa, by proxyUser = null
> (AtlasAuthenticationFilter$KerberosFilterChainWrapper:724)
> 2021-09-06 13:51:28,939 INFO -
> [etp985324122-24:knox:POST/api/atlas/v2/entity/bulk/classification] ~ Request
> from authenticated user: knox,
> URL=/api/atlas/v2/entity/bulk/classification?doAs=hrt_qa
> (AtlasAuthenticationFilter$KerberosFilterChainWrapper:739)
> 2021-09-06 13:51:28,969 INFO - [etp985324122-24 -
> 78a12f6b-7472-4600-a35d-bfe660358712:knox:POST/api/atlas/v2/entity/bulk/classification]
> ~ updateClassificationText: tag_iuanl: tag_iuanl date Mon Sep 06 13:51:28
> UTC 2021 hive_principal_type USER boolean true string XXJid byte -58 double
> 9.475147275250526E37 arr_string fEOKO short -29141 arr_int 781216025 float
> -3.09E38 int 1369026004 long -2808157562510626816 (EntityGraphMapper:2732)
> 2021-09-06 13:51:29,173 INFO - [etp985324122-24 -
> 78a12f6b-7472-4600-a35d-bfe660358712:knox:POST/api/atlas/v2/entity/bulk/classification]
> ~
> {"type":"CLASSIFICATION_PROPAGATION_ADD","guid":"a4ed45a8-cc15-41dd-9340-ebaca95b07f1","createdBy":"hrt_qa","createdTime":1630936288947,"updatedTime":1630936288947,"parameters":{"relationshipGuid":null,"entityGuid":"5c89916f-9dbc-4be8-afd5-9e7659bbb745","classificationVertexId":"44486696"},"attemptCount":0,"status":"PENDING"}
> (TaskExecutor$TaskLogger:170)
> 2021-09-06 13:51:29,190 INFO - [atlas-task-0-etp985324122-24 -
> c6aa9190-8454-43a4-a862-1f61393994a6:] ~ updateClassificationText: tag_iuanl:
> tag_iuanl date Mon Sep 06 13:51:28 UTC 2021 hive_principal_type USER boolean
> true string XXJid byte -58 double 9.475147275250526E37 arr_string fEOKO short
> -29141 arr_int 781216025 float -3.09E38 int 1369026004 long
> -2808157562510626816 (EntityGraphMapper:2732)
> 2021-09-06 13:51:29,193 INFO - [atlas-task-0-etp985324122-24 -
> c6aa9190-8454-43a4-a862-1f61393994a6:] ~ updateClassificationText: tag_iuanl:
> tag_iuanl date Mon Sep 06 13:51:28 UTC 2021 hive_principal_type USER boolean
> true string XXJid byte -58 double 9.475147275250526E37 arr_string fEOKO short
> -29141 arr_int 781216025 float -3.09E38 int 1369026004 long
> -2808157562510626816 (EntityGraphMapper:2732)
> 2021-09-06 13:51:29,366 ERROR - [etp985324122-594:] ~ URL not supported in HA
> mode: /api/atlas/admin/tasks (ActiveServerFilter:121)
> 2021-09-06 13:51:29,366 INFO - [etp985324122-594:] ~ Logged into Atlas as =
> hrt_qa, by proxyUser = null
> (AtlasAuthenticationFilter$KerberosFilterChainWrapper:724)
> 2021-09-06 13:51:29,366 INFO -
> [etp985324122-594:knox:GET/api/atlas/admin/tasks] ~ Request from
> authenticated user: knox, URL=/api/atlas/admin/tasks?doAs=hrt_qa
> (AtlasAuthenticationFilter$KerberosFilterChainWrapper:739)
> 2021-09-06 13:51:29,375 ERROR - [etp985324122-594 -
> 73d22020-c612-4a00-8f47-b2ebd149e976:knox:GET/api/atlas/admin/tasks] ~ Error
> handling a request: f1318c48aa5440a9 (ExceptionMapperUtil:32)
> java.lang.NullPointerException
> at
> org.apache.atlas.tasks.TaskRegistry.toAtlasTask(TaskRegistry.java:188)
> at org.apache.atlas.tasks.TaskRegistry.getAll(TaskRegistry.java:155)
> at
> org.apache.atlas.tasks.TaskManagement.getAll(TaskManagement.java:120)
> at
> org.apache.atlas.web.resources.AdminResource.getTaskStatus(AdminResource.java:772)
> at jdk.internal.reflect.GeneratedMethodAccessor324.invoke(Unknown
> Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
>
[jira] [Updated] (ATLAS-4431) Random NPE when retrieving tasks
[
https://issues.apache.org/jira/browse/ATLAS-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4431:
--
Affects Version/s: 2.2.0
> Random NPE when retrieving tasks
>
>
> Key: ATLAS-4431
> URL: https://issues.apache.org/jira/browse/ATLAS-4431
> Project: Atlas
> Issue Type: Bug
> Components: atlas-intg
>Affects Versions: 2.2.0
>Reporter: Disha Talreja
>Assignee: Disha Talreja
>Priority: Major
> Attachments: ATLAS-4431.patch
>
>
> System tests fail randomly because retrieving tasks using the "admin/tasks"
> endpoint sometimes returns HTTP 500.
> Atlas logs:
> {noformat}
> 2021-09-06 13:51:28,938 INFO - [etp985324122-24:] ~ Logged into Atlas as =
> hrt_qa, by proxyUser = null
> (AtlasAuthenticationFilter$KerberosFilterChainWrapper:724)
> 2021-09-06 13:51:28,939 INFO -
> [etp985324122-24:knox:POST/api/atlas/v2/entity/bulk/classification] ~ Request
> from authenticated user: knox,
> URL=/api/atlas/v2/entity/bulk/classification?doAs=hrt_qa
> (AtlasAuthenticationFilter$KerberosFilterChainWrapper:739)
> 2021-09-06 13:51:28,969 INFO - [etp985324122-24 -
> 78a12f6b-7472-4600-a35d-bfe660358712:knox:POST/api/atlas/v2/entity/bulk/classification]
> ~ updateClassificationText: tag_iuanl: tag_iuanl date Mon Sep 06 13:51:28
> UTC 2021 hive_principal_type USER boolean true string XXJid byte -58 double
> 9.475147275250526E37 arr_string fEOKO short -29141 arr_int 781216025 float
> -3.09E38 int 1369026004 long -2808157562510626816 (EntityGraphMapper:2732)
> 2021-09-06 13:51:29,173 INFO - [etp985324122-24 -
> 78a12f6b-7472-4600-a35d-bfe660358712:knox:POST/api/atlas/v2/entity/bulk/classification]
> ~
> {"type":"CLASSIFICATION_PROPAGATION_ADD","guid":"a4ed45a8-cc15-41dd-9340-ebaca95b07f1","createdBy":"hrt_qa","createdTime":1630936288947,"updatedTime":1630936288947,"parameters":{"relationshipGuid":null,"entityGuid":"5c89916f-9dbc-4be8-afd5-9e7659bbb745","classificationVertexId":"44486696"},"attemptCount":0,"status":"PENDING"}
> (TaskExecutor$TaskLogger:170)
> 2021-09-06 13:51:29,190 INFO - [atlas-task-0-etp985324122-24 -
> c6aa9190-8454-43a4-a862-1f61393994a6:] ~ updateClassificationText: tag_iuanl:
> tag_iuanl date Mon Sep 06 13:51:28 UTC 2021 hive_principal_type USER boolean
> true string XXJid byte -58 double 9.475147275250526E37 arr_string fEOKO short
> -29141 arr_int 781216025 float -3.09E38 int 1369026004 long
> -2808157562510626816 (EntityGraphMapper:2732)
> 2021-09-06 13:51:29,193 INFO - [atlas-task-0-etp985324122-24 -
> c6aa9190-8454-43a4-a862-1f61393994a6:] ~ updateClassificationText: tag_iuanl:
> tag_iuanl date Mon Sep 06 13:51:28 UTC 2021 hive_principal_type USER boolean
> true string XXJid byte -58 double 9.475147275250526E37 arr_string fEOKO short
> -29141 arr_int 781216025 float -3.09E38 int 1369026004 long
> -2808157562510626816 (EntityGraphMapper:2732)
> 2021-09-06 13:51:29,366 ERROR - [etp985324122-594:] ~ URL not supported in HA
> mode: /api/atlas/admin/tasks (ActiveServerFilter:121)
> 2021-09-06 13:51:29,366 INFO - [etp985324122-594:] ~ Logged into Atlas as =
> hrt_qa, by proxyUser = null
> (AtlasAuthenticationFilter$KerberosFilterChainWrapper:724)
> 2021-09-06 13:51:29,366 INFO -
> [etp985324122-594:knox:GET/api/atlas/admin/tasks] ~ Request from
> authenticated user: knox, URL=/api/atlas/admin/tasks?doAs=hrt_qa
> (AtlasAuthenticationFilter$KerberosFilterChainWrapper:739)
> 2021-09-06 13:51:29,375 ERROR - [etp985324122-594 -
> 73d22020-c612-4a00-8f47-b2ebd149e976:knox:GET/api/atlas/admin/tasks] ~ Error
> handling a request: f1318c48aa5440a9 (ExceptionMapperUtil:32)
> java.lang.NullPointerException
> at
> org.apache.atlas.tasks.TaskRegistry.toAtlasTask(TaskRegistry.java:188)
> at org.apache.atlas.tasks.TaskRegistry.getAll(TaskRegistry.java:155)
> at
> org.apache.atlas.tasks.TaskManagement.getAll(TaskManagement.java:120)
> at
> org.apache.atlas.web.resources.AdminResource.getTaskStatus(AdminResource.java:772)
> at jdk.internal.reflect.GeneratedMethodAccessor324.invoke(Unknown
> Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
>
[jira] [Updated] (ATLAS-4369) import-hive.sh does not write logs to /var/log/atlas/import-hive.log file, instead write into ${atlas.log.file} under the directory "${atlas.log.dir}"
[
https://issues.apache.org/jira/browse/ATLAS-4369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4369:
--
Fix Version/s: 2.3.0
3.0.0
> import-hive.sh does not write logs to /var/log/atlas/import-hive.log file,
> instead write into ${atlas.log.file} under the directory "${atlas.log.dir}"
> ---
>
> Key: ATLAS-4369
> URL: https://issues.apache.org/jira/browse/ATLAS-4369
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Umesh Padashetty
>Assignee: Disha Talreja
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: ATLAS-4369.patch
>
>
> When the import scripts like
> * import-kafka.sh
> * import-hive.sh
> * import-hbase.sh
> are run, the corresponding logs are written into
> * /var/log/atlas/import-kafka.log
> * /var/log/atlas/import-hive.log
> * /var/log/atlas/import-hbase.log
> /var/log/atlas/import-kafka.log and /var/log/atlas/import-hbase.log logs are
> correctly written. But when import-hive.sh is ran,
> /var/log/atlas/import-hive.log is not created or written to
> {code:java}
> [root@ccycloud-1 atlas]# ls -ltr
> total 10988
> drwxr-xr-x 2 cstm_atlas cstm_atlas 4096 Jul 16 06:55 jstacks
> drwxr-xr-x 4 root root 4096 Jul 16 06:55 audit
> -rw-r--r-- 1 cstm_atlas cstm_atlas0 Jul 16 06:55 metric.log
> -rw-r--r-- 1 cstm_atlas cstm_atlas0 Jul 16 06:55 failed.log
> drwxr-xr-x 2 cstm_atlas cstm_atlas 4096 Jul 18 03:52 support-bundle
> -rw-r--r-- 1 root root 21573 Jul 22 09:56 import-kafka.log
> -rw-r--r-- 1 root root 159459 Jul 22 09:58 import-hbase.log
> -rw-r--r-- 1 cstm_atlas cstm_atlas19429 Jul 22 10:12
> gc-worker.log.0.current
> -rw-r--r-- 1 cstm_atlas cstm_atlas 179081 Jul 22 10:22 audit.log
> -rw-r--r-- 1 cstm_atlas cstm_atlas 10840903 Jul 22 10:23 application.log
> {code}
> whereas it creates a directory with the name ${atlas.log.dir} and a file with
> the name ${atlas.log.file} under it
> {code:java}
> [root@ccycloud-1 hook-bin]# ls -ltr
> total 44
> -rwxr-xr-x 1 root root 3892 Jul 14 13:09 import-kafka.sh
> -rwxr-xr-x 1 root root 4547 Jul 14 13:09 import-hive.sh
> -rwxr-xr-x 1 root root 4341 Jul 14 13:09 import-hbase.sh
> drwxr-sr-x 2 root root 4096 Jul 22 10:23 ${atlas.log.dir}
> -rw-r--r-- 1 root root 20094 Jul 22 10:23 derby.log {code}
> The import-hive.sh logs are written into ${atlas.log.file}
> {code:java}
> [root@ccycloud-1 ${atlas.log.dir}]# ls -ltra
> total 48
> drwxr-sr-x 3 root root 4096 Jul 22 10:23 ..
> drwxr-sr-x 2 root root 4096 Jul 22 10:23 .
> -rw-r--r-- 1 root root 40960 Jul 22 10:23 ${atlas.log.file} {code}
> The import-hive.sh logs should be written into /var/log/atlas/import-hive.log
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4369) import-hive.sh does not write logs to /var/log/atlas/import-hive.log file, instead write into ${atlas.log.file} under the directory "${atlas.log.dir}"
[
https://issues.apache.org/jira/browse/ATLAS-4369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4369:
--
Affects Version/s: 3.0.0
2.2.0
> import-hive.sh does not write logs to /var/log/atlas/import-hive.log file,
> instead write into ${atlas.log.file} under the directory "${atlas.log.dir}"
> ---
>
> Key: ATLAS-4369
> URL: https://issues.apache.org/jira/browse/ATLAS-4369
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Umesh Padashetty
>Assignee: Disha Talreja
>Priority: Major
> Attachments: ATLAS-4369.patch
>
>
> When the import scripts like
> * import-kafka.sh
> * import-hive.sh
> * import-hbase.sh
> are run, the corresponding logs are written into
> * /var/log/atlas/import-kafka.log
> * /var/log/atlas/import-hive.log
> * /var/log/atlas/import-hbase.log
> /var/log/atlas/import-kafka.log and /var/log/atlas/import-hbase.log logs are
> correctly written. But when import-hive.sh is ran,
> /var/log/atlas/import-hive.log is not created or written to
> {code:java}
> [root@ccycloud-1 atlas]# ls -ltr
> total 10988
> drwxr-xr-x 2 cstm_atlas cstm_atlas 4096 Jul 16 06:55 jstacks
> drwxr-xr-x 4 root root 4096 Jul 16 06:55 audit
> -rw-r--r-- 1 cstm_atlas cstm_atlas0 Jul 16 06:55 metric.log
> -rw-r--r-- 1 cstm_atlas cstm_atlas0 Jul 16 06:55 failed.log
> drwxr-xr-x 2 cstm_atlas cstm_atlas 4096 Jul 18 03:52 support-bundle
> -rw-r--r-- 1 root root 21573 Jul 22 09:56 import-kafka.log
> -rw-r--r-- 1 root root 159459 Jul 22 09:58 import-hbase.log
> -rw-r--r-- 1 cstm_atlas cstm_atlas19429 Jul 22 10:12
> gc-worker.log.0.current
> -rw-r--r-- 1 cstm_atlas cstm_atlas 179081 Jul 22 10:22 audit.log
> -rw-r--r-- 1 cstm_atlas cstm_atlas 10840903 Jul 22 10:23 application.log
> {code}
> whereas it creates a directory with the name ${atlas.log.dir} and a file with
> the name ${atlas.log.file} under it
> {code:java}
> [root@ccycloud-1 hook-bin]# ls -ltr
> total 44
> -rwxr-xr-x 1 root root 3892 Jul 14 13:09 import-kafka.sh
> -rwxr-xr-x 1 root root 4547 Jul 14 13:09 import-hive.sh
> -rwxr-xr-x 1 root root 4341 Jul 14 13:09 import-hbase.sh
> drwxr-sr-x 2 root root 4096 Jul 22 10:23 ${atlas.log.dir}
> -rw-r--r-- 1 root root 20094 Jul 22 10:23 derby.log {code}
> The import-hive.sh logs are written into ${atlas.log.file}
> {code:java}
> [root@ccycloud-1 ${atlas.log.dir}]# ls -ltra
> total 48
> drwxr-sr-x 3 root root 4096 Jul 22 10:23 ..
> drwxr-sr-x 2 root root 4096 Jul 22 10:23 .
> -rw-r--r-- 1 root root 40960 Jul 22 10:23 ${atlas.log.file} {code}
> The import-hive.sh logs should be written into /var/log/atlas/import-hive.log
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-4403) Improve find entity by unique attributes logic - when multiple unique attributes exist for entity type
Sarath Subramanian created ATLAS-4403: - Summary: Improve find entity by unique attributes logic - when multiple unique attributes exist for entity type Key: ATLAS-4403 URL: https://issues.apache.org/jira/browse/ATLAS-4403 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 2.2.0 Reporter: Sarath Subramanian Assignee: Sarath Subramanian Fix For: 2.3.0 When a entity type has multiple unique attributes defined: * unique_attr1 * unique_attr2 During creation of entity, we check if entity with unique attributes is already present and return/update them. The current logic check if entity is present - one unique attribute at a time, if present will return immediately. We should improve the lookup logic to look for entity with all unique attributes specified and not just return on the first unique attribute value. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4399) Restore deleted entity
[ https://issues.apache.org/jira/browse/ATLAS-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403260#comment-17403260 ] Sarath Subramanian commented on ATLAS-4399: --- To answer your questions: # You cannot create an entity with an initial DELETED status. # Currently reactivating deleted entity is not supported in Atlas. Hope this helps. > Restore deleted entity > -- > > Key: ATLAS-4399 > URL: https://issues.apache.org/jira/browse/ATLAS-4399 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Zhang Joseph >Priority: Major > Labels: Restore > > Hi All, > I have met this situation that when I delete one entity and entity status > changed from "ACTIVE" to "DELETED", then I want to re-active the entity with > same qualifiedName, but I got two entities with same qualifiedName, one with > status ACTIVE, another with status DELETED. instead of restoring the old one. > Here is my questions: > # Can I create one entity with initial status "DELETED"? > # Do we have approach to restore deleted entity? I have read some clues > about OMRS ways but with no sample, and not find any code in atlas source > code. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (ATLAS-4399) Restore deleted entity
[ https://issues.apache.org/jira/browse/ATLAS-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403250#comment-17403250 ] Sarath Subramanian edited comment on ATLAS-4399 at 8/23/21, 4:05 PM: - [~josephz], Atlas is a governance tool and it is important to store the metadata history of deleted entity as well. Currently there is no way to activate deleted entity in Atlas. If you reactive deleted entity, the metadata information of previous deleted entity is overriden and the history is lost. An entity in Atlas can have only one ACTIVE state and any number of DELETED entity states for the same qualifiedName. was (Author: sarath.ku...@gmail.com): [~josephz], Atlas is a governance tool and its key to store the metadata history of deleted entity as well. Currently there is no way to activate deleted entity in Atlas. If you reactive deleted entity, the metadata information of previous deleted entity is overriden and the history is lost. An entity in Atlas can have only one ACTIVE state and any number of DELETED entity states for the same qualifiedName. > Restore deleted entity > -- > > Key: ATLAS-4399 > URL: https://issues.apache.org/jira/browse/ATLAS-4399 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Zhang Joseph >Priority: Major > Labels: Restore > > Hi All, > I have met this situation that when I delete one entity and entity status > changed from "ACTIVE" to "DELETED", then I want to re-active the entity with > same qualifiedName, but I got two entities with same qualifiedName, one with > status ACTIVE, another with status DELETED. instead of restoring the old one. > Here is my questions: > # Can I create one entity with initial status "DELETED"? > # Do we have approach to restore deleted entity? I have read some clues > about OMRS ways but with no sample, and not find any code in atlas source > code. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4399) Restore deleted entity
[ https://issues.apache.org/jira/browse/ATLAS-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403250#comment-17403250 ] Sarath Subramanian commented on ATLAS-4399: --- [~josephz], Atlas is a governance tool and its key to store the metadata history of deleted entity as well. Currently there is no way to activate deleted entity in Atlas. If you reactive deleted entity, the metadata information of previous deleted entity is overriden and the history is lost. An entity in Atlas can have only one ACTIVE state and any number of DELETED entity states for the same qualifiedName. > Restore deleted entity > -- > > Key: ATLAS-4399 > URL: https://issues.apache.org/jira/browse/ATLAS-4399 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Zhang Joseph >Priority: Major > Labels: Restore > > Hi All, > I have met this situation that when I delete one entity and entity status > changed from "ACTIVE" to "DELETED", then I want to re-active the entity with > same qualifiedName, but I got two entities with same qualifiedName, one with > status ACTIVE, another with status DELETED. instead of restoring the old one. > Here is my questions: > # Can I create one entity with initial status "DELETED"? > # Do we have approach to restore deleted entity? I have read some clues > about OMRS ways but with no sample, and not find any code in atlas source > code. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4364) Update duplicate Java Patch Id of ProcessNamePatch
[ https://issues.apache.org/jira/browse/ATLAS-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4364: -- Fix Version/s: 2.2.0 3.0.0 > Update duplicate Java Patch Id of ProcessNamePatch > -- > > Key: ATLAS-4364 > URL: https://issues.apache.org/jira/browse/ATLAS-4364 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > Java Patch - ProcessNamePatch has patch_id which is already used for other > Java patch. > Need to update patch_id for ProcessNamePatch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4348) Atlas-Kafka Hook : When a producer publishes messages to multiple topics, the latest relationship is marked ACTIVE , rest are marked DELETED
[
https://issues.apache.org/jira/browse/ATLAS-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4348:
--
Fix Version/s: 2.2.0
3.0.0
> Atlas-Kafka Hook : When a producer publishes messages to multiple topics, the
> latest relationship is marked ACTIVE , rest are marked DELETED
>
>
> Key: ATLAS-4348
> URL: https://issues.apache.org/jira/browse/ATLAS-4348
> Project: Atlas
> Issue Type: Task
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Radhika Kundam
>Assignee: Radhika Kundam
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments:
> 0001-ATLAS-4348-Atlas-Kafka-Hook-appendRelationshipsOnPar.patch
>
>
> *Steps to reproduce the issue*:
> Created 2 topics : test1, test2
> Using console consumer produced messages to test1 and test2.
> Relationship between console-producer-lineage and test1 is ACTIVE now.
> Relationship between console-producer-lineage and test1 is set to DELETED
> when messages are published to test2.
> *Root cause of issue:*
> When messages published on topic, Atlas hook receives partial_update request
> with only one topic as output and by default partial update deletes
> relationships between previous topics by considering the only topic(which is
> in partial_update request) as the active.
> *Proposed Solution:*
> Introducing typedef option "appendRelationshipsOnPartialUpdate" which will
> hold list of relationship attribute names. While Atlas startup, it will
> resolve entity type definition and all the specified relationship attributes
> definitions will be updated with typedef property "isAppendOnPartialUpdate"
> only for that entity type.
> With this solution, we can keep all the previous relationship entries active
> along with latest relationship entry for the specific entity by providing
> typedef options as below.
> "typeDefOptions": { "appendRelationshipsOnPartialUpdate": "[\"inputs\",
> \"outputs\"]"}
>
> This solution can be used for any entity type to consider all the existing
> relationship entries as active along with new entries from partial_update.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4354) atlas import-hive.sh fails with java.lang.AbstractMethodError: Receiver class com.sun.jersey.api.uri.UriBuilderImpl does not define or inherit an implementation of the re
[ https://issues.apache.org/jira/browse/ATLAS-4354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4354: -- Fix Version/s: 2.2.0 3.0.0 > atlas import-hive.sh fails with java.lang.AbstractMethodError: Receiver class > com.sun.jersey.api.uri.UriBuilderImpl does not define or inherit an > implementation of the resolved method abstract uri > > > Key: ATLAS-4354 > URL: https://issues.apache.org/jira/browse/ATLAS-4354 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > While importing hive getting below error. > atlas import-hive.sh fails with java.lang.AbstractMethodError: Receiver class > com.sun.jersey.api.uri.UriBuilderImpl does not define or inherit an > implementation of the resolved method abstract uri > 2021-07-09 23:28:51,803|INFO|MainThread|machine.py:186 - > run()||GUID=78391325-5071-4406-99bd-99c1947a742f|at > javax.ws.rs.core.UriBuilder.fromUri(UriBuilder.java:119) > 2021-07-09 23:28:51,803|INFO|MainThread|machine.py:186 - > run()||GUID=78391325-5071-4406-99bd-99c1947a742f|at > org.apache.atlas.AtlasBaseClient.getAddressIfActive(AtlasBaseClient.java:633) > 2021-07-09 23:28:51,803|INFO|MainThread|machine.py:186 - > run()||GUID=78391325-5071-4406-99bd-99c1947a742f|at > org.apache.atlas.AtlasBaseClient.selectActiveServerAddress(AtlasBaseClient.java:617) > 2021-07-09 23:28:51,803|INFO|MainThread|machine.py:186 - > run()||GUID=78391325-5071-4406-99bd-99c1947a742f|at > org.apache.atlas.AtlasBaseClient.determineActiveServiceURL(AtlasBaseClient.java:323) > 2021-07-09 23:28:51,804|INFO|MainThread|machine.py:186 - > run()||GUID=78391325-5071-4406-99bd-99c1947a742f|at > org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:480) > 2021-07-09 23:28:51,804|INFO|MainThread|machine.py:186 - > run()||GUID=78391325-5071-4406-99bd-99c1947a742f|at > org.apache.atlas.AtlasBaseClient.initializeState(AtlasBaseClient.java:468) > 2021-07-09 23:28:51,804|INFO|MainThread|machine.py:186 - > run()||GUID=78391325-5071-4406-99bd-99c1947a742f|at > org.apache.atlas.AtlasBaseClient.(AtlasBaseClient.java:143) > 2021-07-09 23:28:51,804|INFO|MainThread|machine.py:186 - > run()||GUID=78391325-5071-4406-99bd-99c1947a742f|at > org.apache.atlas.AtlasClientV2.(AtlasClientV2.java:146) > 2021-07-09 23:28:51,804|INFO|MainThread|machine.py:186 - > run()||GUID=78391325-5071-4406-99bd-99c1947a742f|at > org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:158) > 2021-07-09 23:28:51,841|INFO|MainThread|machine.py:186 - > run()||GUID=78391325-5071-4406-99bd-99c1947a742f| -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4339) Atlas should support skip temporary tables using config property
[ https://issues.apache.org/jira/browse/ATLAS-4339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4339: -- Fix Version/s: 2.2.0 3.0.0 > Atlas should support skip temporary tables using config property > > > Key: ATLAS-4339 > URL: https://issues.apache.org/jira/browse/ATLAS-4339 > Project: Atlas > Issue Type: Task >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > Atlas should skip/allow temporary tables based on user configuration. > At present Atlas skips temporary tables by default. But it should be > configurable so that if user wants Atlas to collect temporary tables they > should be able to configure the option. > Introducing "atlas.hook.hive.skip.temp.tables" config property at Hive Hook > to handle this feature. With this change, by default atlas will skip > temporary tables as earlier. By providing false flag to skip.temp.tables > property, Atlas will collect temporary tables. > *atlas.hook.hive.skip.temp.tables=false* -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4339) Atlas should support skip temporary tables using config property
[ https://issues.apache.org/jira/browse/ATLAS-4339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4339: -- Labels: hive-hooks (was: ) > Atlas should support skip temporary tables using config property > > > Key: ATLAS-4339 > URL: https://issues.apache.org/jira/browse/ATLAS-4339 > Project: Atlas > Issue Type: Task > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Labels: hive-hooks > Fix For: 3.0.0, 2.2.0 > > > Atlas should skip/allow temporary tables based on user configuration. > At present Atlas skips temporary tables by default. But it should be > configurable so that if user wants Atlas to collect temporary tables they > should be able to configure the option. > Introducing "atlas.hook.hive.skip.temp.tables" config property at Hive Hook > to handle this feature. With this change, by default atlas will skip > temporary tables as earlier. By providing false flag to skip.temp.tables > property, Atlas will collect temporary tables. > *atlas.hook.hive.skip.temp.tables=false* -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4339) Atlas should support skip temporary tables using config property
[ https://issues.apache.org/jira/browse/ATLAS-4339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4339: -- Affects Version/s: 2.1.0 > Atlas should support skip temporary tables using config property > > > Key: ATLAS-4339 > URL: https://issues.apache.org/jira/browse/ATLAS-4339 > Project: Atlas > Issue Type: Task >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > > Atlas should skip/allow temporary tables based on user configuration. > At present Atlas skips temporary tables by default. But it should be > configurable so that if user wants Atlas to collect temporary tables they > should be able to configure the option. > Introducing "atlas.hook.hive.skip.temp.tables" config property at Hive Hook > to handle this feature. With this change, by default atlas will skip > temporary tables as earlier. By providing false flag to skip.temp.tables > property, Atlas will collect temporary tables. > *atlas.hook.hive.skip.temp.tables=false* -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4339) Atlas should support skip temporary tables using config property
[ https://issues.apache.org/jira/browse/ATLAS-4339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4339: -- Component/s: atlas-core > Atlas should support skip temporary tables using config property > > > Key: ATLAS-4339 > URL: https://issues.apache.org/jira/browse/ATLAS-4339 > Project: Atlas > Issue Type: Task > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > Atlas should skip/allow temporary tables based on user configuration. > At present Atlas skips temporary tables by default. But it should be > configurable so that if user wants Atlas to collect temporary tables they > should be able to configure the option. > Introducing "atlas.hook.hive.skip.temp.tables" config property at Hive Hook > to handle this feature. With this change, by default atlas will skip > temporary tables as earlier. By providing false flag to skip.temp.tables > property, Atlas will collect temporary tables. > *atlas.hook.hive.skip.temp.tables=false* -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4339) Atlas should support skip temporary tables using config property
[ https://issues.apache.org/jira/browse/ATLAS-4339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian resolved ATLAS-4339. --- Resolution: Fixed > Atlas should support skip temporary tables using config property > > > Key: ATLAS-4339 > URL: https://issues.apache.org/jira/browse/ATLAS-4339 > Project: Atlas > Issue Type: Task > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Labels: hive-hooks > Fix For: 3.0.0, 2.2.0 > > > Atlas should skip/allow temporary tables based on user configuration. > At present Atlas skips temporary tables by default. But it should be > configurable so that if user wants Atlas to collect temporary tables they > should be able to configure the option. > Introducing "atlas.hook.hive.skip.temp.tables" config property at Hive Hook > to handle this feature. With this change, by default atlas will skip > temporary tables as earlier. By providing false flag to skip.temp.tables > property, Atlas will collect temporary tables. > *atlas.hook.hive.skip.temp.tables=false* -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4340) Set Solr wait-searcher property to false by default to make Solr commits async
[ https://issues.apache.org/jira/browse/ATLAS-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4340: -- Attachment: ATLAS-4340-001.patch > Set Solr wait-searcher property to false by default to make Solr commits async > -- > > Key: ATLAS-4340 > URL: https://issues.apache.org/jira/browse/ATLAS-4340 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Labels: perfomance, solr > Fix For: 3.0.0, 2.2.0 > > Attachments: ATLAS-4340-001.patch > > > In Atlas when a transaction is committed, the entries are committed to HBase > (primary storage) and Solr (indexing storage). A transaction is rolled-back > if the primary storage commit fails, on the other hand when the secondary > commit fails (solr), the transaction is not-rolled back and logged as warning > and it is recommended to use reindex to repair the missing index documents. > This behavior is due to the fact that the primary storage is the source of > truth and indexes can be rebuild. > In Janusgraph, there is a property for Solr to make solr commits async. This > is set to *true* in Atlas making every commit to wait until the solr commit > is successful. This will have a negative impact on performance and is > recommended to be false by default. > Property: *index.[X].solr.wait-searcher* > |When mutating - wait for the index to reflect new mutations before > returning. This can have a negative impact on performance.| > > This Jira is about setting the default value for above property to FALSE and > can be overridden if need arises. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-4340) Set Solr wait-searcher property to false by default to make Solr commits async
Sarath Subramanian created ATLAS-4340: - Summary: Set Solr wait-searcher property to false by default to make Solr commits async Key: ATLAS-4340 URL: https://issues.apache.org/jira/browse/ATLAS-4340 Project: Atlas Issue Type: Improvement Components: atlas-core Affects Versions: 2.1.0 Reporter: Sarath Subramanian Assignee: Sarath Subramanian Fix For: 3.0.0, 2.2.0 In Atlas when a transaction is committed, the entries are committed to HBase (primary storage) and Solr (indexing storage). A transaction is rolled-back if the primary storage commit fails, on the other hand when the secondary commit fails (solr), the transaction is not-rolled back and logged as warning and it is recommended to use reindex to repair the missing index documents. This behavior is due to the fact that the primary storage is the source of truth and indexes can be rebuild. In Janusgraph, there is a property for Solr to make solr commits async. This is set to *true* in Atlas making every commit to wait until the solr commit is successful. This will have a negative impact on performance and is recommended to be false by default. Property: *index.[X].solr.wait-searcher* |When mutating - wait for the index to reflect new mutations before returning. This can have a negative impact on performance.| This Jira is about setting the default value for above property to FALSE and can be overridden if need arises. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4330) Add Kafka topics lag information on metrics and log
[
https://issues.apache.org/jira/browse/ATLAS-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4330:
--
Labels: kafka metrics topic-lag (was: )
> Add Kafka topics lag information on metrics and log
> ---
>
> Key: ATLAS-4330
> URL: https://issues.apache.org/jira/browse/ATLAS-4330
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Labels: kafka, metrics, topic-lag
> Fix For: 3.0.0, 2.2.0
>
> Attachments: image-2021-06-07-15-03-28-166.png
>
>
> In metrics page, in addition to other Notification details. Lag information
> of kafka consumer topics will be useful to check the current backed up
> messages in Atlas:
> !image-2021-06-07-15-03-28-166.png!
> It will also be useful to print the lag information periodically in atlas
> application.log on a frequency that can be made configurable (print lag every
> 1 hour, 5min or so).
> *atlas.notification.consumer.topic.report.frequency.seconds=60*
> {code:java}
> 2021-06-07 21:56:06,121 INFO - NotificationProcessingStats:
> topic=ATLAS_SPARK_HOOK-0, currentOffset=18, endOffset=18, lag=0
> 2021-06-07 21:56:09,105 INFO - NotificationProcessingStats:
> topic=ATLAS_HOOK-0, currentOffset=1118, endOffset=1118, lag=0
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4330) Add Kafka topics lag information on metrics and log
[
https://issues.apache.org/jira/browse/ATLAS-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4330:
--
Description:
In metrics page, in addition to other Notification details. Lag information of
kafka consumer topics will be useful to check the current backed up messages in
Atlas:
!image-2021-06-07-15-03-28-166.png!
It will also be useful to print the lag information periodically in atlas
application.log on a frequency that can be made configurable (print lag every 1
hour, 5min or so).
*atlas.notification.consumer.topic.report.frequency.seconds=60*
{code:java}
2021-06-07 21:56:06,121 INFO - NotificationProcessingStats:
topic=ATLAS_SPARK_HOOK-0, currentOffset=18, endOffset=18, lag=0
2021-06-07 21:56:09,105 INFO - NotificationProcessingStats: topic=ATLAS_HOOK-0,
currentOffset=1118, endOffset=1118, lag=0
{code}
was:
In metrics page, in addition to other Notification details. Lag information of
kafka consumer topics will be useful to check the current backup up messages in
Atlas:
!image-2021-06-07-15-03-28-166.png!
It will also be useful to print the lag information periodically in atlas
application.log on a frequency that can be made configurable (print lag every 1
hour, 5min or so).
*atlas.notification.consumer.topic.report.frequency.seconds=60*
{code:java}
2021-06-07 21:56:06,121 INFO - NotificationProcessingStats:
topic=ATLAS_SPARK_HOOK-0, currentOffset=18, endOffset=18, lag=0
2021-06-07 21:56:09,105 INFO - NotificationProcessingStats: topic=ATLAS_HOOK-0,
currentOffset=1118, endOffset=1118, lag=0
{code}
> Add Kafka topics lag information on metrics and log
> ---
>
> Key: ATLAS-4330
> URL: https://issues.apache.org/jira/browse/ATLAS-4330
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: image-2021-06-07-15-03-28-166.png
>
>
> In metrics page, in addition to other Notification details. Lag information
> of kafka consumer topics will be useful to check the current backed up
> messages in Atlas:
> !image-2021-06-07-15-03-28-166.png!
> It will also be useful to print the lag information periodically in atlas
> application.log on a frequency that can be made configurable (print lag every
> 1 hour, 5min or so).
> *atlas.notification.consumer.topic.report.frequency.seconds=60*
> {code:java}
> 2021-06-07 21:56:06,121 INFO - NotificationProcessingStats:
> topic=ATLAS_SPARK_HOOK-0, currentOffset=18, endOffset=18, lag=0
> 2021-06-07 21:56:09,105 INFO - NotificationProcessingStats:
> topic=ATLAS_HOOK-0, currentOffset=1118, endOffset=1118, lag=0
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4330) Add Kafka topics lag information on metrics and log
[
https://issues.apache.org/jira/browse/ATLAS-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4330:
--
Description:
In metrics page, in addition to other Notification details. Lag information of
kafka consumer topics will be useful to check the current backup up messages in
Atlas:
!image-2021-06-07-15-03-28-166.png!
It will also be useful to print the lag information periodically in atlas
application.log on a frequency that can be made configurable (print lag every 1
hour, 5min or so).
*atlas.notification.consumer.topic.report.frequency.seconds=60*
{code:java}
2021-06-07 21:56:06,121 INFO - NotificationProcessingStats:
topic=ATLAS_SPARK_HOOK-0, currentOffset=18, endOffset=18, lag=0
2021-06-07 21:56:09,105 INFO - NotificationProcessingStats: topic=ATLAS_HOOK-0,
currentOffset=1118, endOffset=1118, lag=0
{code}
was:
In metrics page, in addition to other Notification details. Lag information of
kafka consumer topics will be useful to check the current backup up messages in
Atlas:
!image-2021-06-07-15-03-28-166.png!
It will also be useful to print the lag information periodically in atlas
application.log on a frequency that can be made configurable (print lag every 1
hour, 5min or so).
*atlas.notification.consumer.topic.report.frequency.seconds=60*
{code:java}
2021-06-07 21:56:06,121 INFO - NotificationProcessingStats:
topic=ATLAS_SPARK_HOOK-0, currentOffset=18, endOffset=18, lag=0
2021-06-07 21:56:09,105 INFO - NotificationProcessingStats: topic=ATLAS_HOOK-0,
currentOffset=1118, endOffset=1118, lag=0
{code}
> Add Kafka topics lag information on metrics and log
> ---
>
> Key: ATLAS-4330
> URL: https://issues.apache.org/jira/browse/ATLAS-4330
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: image-2021-06-07-15-03-28-166.png
>
>
> In metrics page, in addition to other Notification details. Lag information
> of kafka consumer topics will be useful to check the current backup up
> messages in Atlas:
> !image-2021-06-07-15-03-28-166.png!
> It will also be useful to print the lag information periodically in atlas
> application.log on a frequency that can be made configurable (print lag every
> 1 hour, 5min or so).
> *atlas.notification.consumer.topic.report.frequency.seconds=60*
> {code:java}
> 2021-06-07 21:56:06,121 INFO - NotificationProcessingStats:
> topic=ATLAS_SPARK_HOOK-0, currentOffset=18, endOffset=18, lag=0
> 2021-06-07 21:56:09,105 INFO - NotificationProcessingStats:
> topic=ATLAS_HOOK-0, currentOffset=1118, endOffset=1118, lag=0
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4330) Add Kafka topics lag information on metrics and log
[
https://issues.apache.org/jira/browse/ATLAS-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4330:
--
Description:
In metrics page, in addition to other Notification details. Lag information of
kafka consumer topics will be useful to check the current backup up messages in
Atlas:
!image-2021-06-07-15-03-28-166.png!
It will also be useful to print the lag information periodically in atlas
application.log on a frequency that can be made configurable (print lag every 1
hour, 5min or so).
*atlas.notification.consumer.topic.report.frequency.seconds=60*
{code:java}
2021-06-07 21:56:06,121 INFO - NotificationProcessingStats:
topic=ATLAS_SPARK_HOOK-0, currentOffset=18, endOffset=18, lag=0
2021-06-07 21:56:09,105 INFO - NotificationProcessingStats: topic=ATLAS_HOOK-0,
currentOffset=1118, endOffset=1118, lag=0
{code}
was:
In metrics page, in addition to other Notification details. Lag information of
kafka consumer topics will be useful to check the current backup up messages in
Atlas:
!image-2021-06-07-15-03-28-166.png!
> Add Kafka topics lag information on metrics and log
> ---
>
> Key: ATLAS-4330
> URL: https://issues.apache.org/jira/browse/ATLAS-4330
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: image-2021-06-07-15-03-28-166.png
>
>
> In metrics page, in addition to other Notification details. Lag information
> of kafka consumer topics will be useful to check the current backup up
> messages in Atlas:
> !image-2021-06-07-15-03-28-166.png!
> It will also be useful to print the lag information periodically in atlas
> application.log on a frequency that can be made configurable (print lag every
> 1 hour, 5min or so).
>
> *atlas.notification.consumer.topic.report.frequency.seconds=60*
>
>
> {code:java}
> 2021-06-07 21:56:06,121 INFO - NotificationProcessingStats:
> topic=ATLAS_SPARK_HOOK-0, currentOffset=18, endOffset=18, lag=0
> 2021-06-07 21:56:09,105 INFO - NotificationProcessingStats:
> topic=ATLAS_HOOK-0, currentOffset=1118, endOffset=1118, lag=0
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-4330) Add Kafka topics lag information on metrics and log
Sarath Subramanian created ATLAS-4330: - Summary: Add Kafka topics lag information on metrics and log Key: ATLAS-4330 URL: https://issues.apache.org/jira/browse/ATLAS-4330 Project: Atlas Issue Type: Improvement Components: atlas-core Affects Versions: 2.1.0 Reporter: Sarath Subramanian Assignee: Sarath Subramanian Fix For: 3.0.0, 2.2.0 Attachments: image-2021-06-07-15-03-28-166.png In metrics page, in addition to other Notification details. Lag information of kafka consumer topics will be useful to check the current backup up messages in Atlas: !image-2021-06-07-15-03-28-166.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4329) Update Kafka version to 2.5
[ https://issues.apache.org/jira/browse/ATLAS-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358752#comment-17358752 ] Sarath Subramanian commented on ATLAS-4329: --- *Precommit:* [https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/620/] *Manual testing:* Build Atlas with embedded HBase and Solr profile and was able to start Atlas and validate basic functionality > Update Kafka version to 2.5 > > > Key: ATLAS-4329 > URL: https://issues.apache.org/jira/browse/ATLAS-4329 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Labels: kafka, pom > Fix For: 3.0.0, 2.2.0 > > Attachments: ATLAS-4329-Update-Kafka-version-to-2.5.patch > > > Atlas uses the following kafka versions for producer and consumer: > +*Current:*+ > * 2.0.0 > * 2.11 > +*New:*+ > * 2.5.0 > * 2.12 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4329) Update Kafka version to 2.5
[ https://issues.apache.org/jira/browse/ATLAS-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4329: -- Attachment: ATLAS-4329-Update-Kafka-version-to-2.5.patch > Update Kafka version to 2.5 > > > Key: ATLAS-4329 > URL: https://issues.apache.org/jira/browse/ATLAS-4329 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Labels: kafka, pom > Fix For: 3.0.0, 2.2.0 > > Attachments: ATLAS-4329-Update-Kafka-version-to-2.5.patch > > > Atlas uses the following kafka versions for producer and consumer: > +*Current:*+ > * 2.0.0 > * 2.11 > +*New:*+ > * 2.5.0 > * 2.12 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4329) Update Kafka version to 2.5
[ https://issues.apache.org/jira/browse/ATLAS-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4329: -- Labels: kafka pom (was: ) > Update Kafka version to 2.5 > > > Key: ATLAS-4329 > URL: https://issues.apache.org/jira/browse/ATLAS-4329 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Labels: kafka, pom > Fix For: 3.0.0, 2.2.0 > > > Atlas uses the following kafka versions for producer and consumer: > +*Current:*+ > * 2.0.0 > * 2.11 > +*New:*+ > * 2.5.0 > * 2.12 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-4329) Update Kafka version to 2.5
Sarath Subramanian created ATLAS-4329: - Summary: Update Kafka version to 2.5 Key: ATLAS-4329 URL: https://issues.apache.org/jira/browse/ATLAS-4329 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 2.1.0 Reporter: Sarath Subramanian Assignee: Sarath Subramanian Fix For: 3.0.0, 2.2.0 Atlas uses the following kafka versions for producer and consumer: +*Current:*+ * 2.0.0 * 2.11 +*New:*+ * 2.5.0 * 2.12 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4324) FS entity created for load data inpath is created as shell entity
[ https://issues.apache.org/jira/browse/ATLAS-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4324: -- Labels: hive-hooks (was: ) > FS entity created for load data inpath is created as shell entity > - > > Key: ATLAS-4324 > URL: https://issues.apache.org/jira/browse/ATLAS-4324 > Project: Atlas > Issue Type: Bug > Components: hive-integration >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Labels: hive-hooks > Fix For: 3.0.0, 2.2.0 > > Attachments: load_data_shell_entity.png > > > beeline > > create external table default.hive_table_cloud_load_data_in_path_123 > (student_roll int, student_name string, student_dob date) ROW FORMAT > DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE location > 'hdfs://ns1/tmp/hive_table_cloud_load_data_in_path_123' > > load data inpath 'hdfs://ns1/tmp/data123.txt' into table > hive_table_cloud_load_data_in_path_123; > > Creates 'hdfs://ns1/tmp/data123.txt' as shell entity. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4324) FS entity created for load data inpath is created as shell entity
[ https://issues.apache.org/jira/browse/ATLAS-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian resolved ATLAS-4324. --- Resolution: Fixed > FS entity created for load data inpath is created as shell entity > - > > Key: ATLAS-4324 > URL: https://issues.apache.org/jira/browse/ATLAS-4324 > Project: Atlas > Issue Type: Bug > Components: hive-integration >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Labels: hive-hooks > Fix For: 3.0.0, 2.2.0 > > Attachments: load_data_shell_entity.png > > > beeline > > create external table default.hive_table_cloud_load_data_in_path_123 > (student_roll int, student_name string, student_dob date) ROW FORMAT > DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE location > 'hdfs://ns1/tmp/hive_table_cloud_load_data_in_path_123' > > load data inpath 'hdfs://ns1/tmp/data123.txt' into table > hive_table_cloud_load_data_in_path_123; > > Creates 'hdfs://ns1/tmp/data123.txt' as shell entity. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4324) FS entity created for load data inpath is created as shell entity
[ https://issues.apache.org/jira/browse/ATLAS-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4324: -- Component/s: hive-integration > FS entity created for load data inpath is created as shell entity > - > > Key: ATLAS-4324 > URL: https://issues.apache.org/jira/browse/ATLAS-4324 > Project: Atlas > Issue Type: Bug > Components: hive-integration >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: load_data_shell_entity.png > > > beeline > > create external table default.hive_table_cloud_load_data_in_path_123 > (student_roll int, student_name string, student_dob date) ROW FORMAT > DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE location > 'hdfs://ns1/tmp/hive_table_cloud_load_data_in_path_123' > > load data inpath 'hdfs://ns1/tmp/data123.txt' into table > hive_table_cloud_load_data_in_path_123; > > Creates 'hdfs://ns1/tmp/data123.txt' as shell entity. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4324) FS entity created for load data inpath is created as shell entity
[ https://issues.apache.org/jira/browse/ATLAS-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4324: -- Fix Version/s: 2.2.0 3.0.0 > FS entity created for load data inpath is created as shell entity > - > > Key: ATLAS-4324 > URL: https://issues.apache.org/jira/browse/ATLAS-4324 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: load_data_shell_entity.png > > > beeline > > create external table default.hive_table_cloud_load_data_in_path_123 > (student_roll int, student_name string, student_dob date) ROW FORMAT > DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE location > 'hdfs://ns1/tmp/hive_table_cloud_load_data_in_path_123' > > load data inpath 'hdfs://ns1/tmp/data123.txt' into table > hive_table_cloud_load_data_in_path_123; > > Creates 'hdfs://ns1/tmp/data123.txt' as shell entity. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4324) FS entity created for load data inpath is created as shell entity
[ https://issues.apache.org/jira/browse/ATLAS-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4324: -- Affects Version/s: 2.1.0 > FS entity created for load data inpath is created as shell entity > - > > Key: ATLAS-4324 > URL: https://issues.apache.org/jira/browse/ATLAS-4324 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Attachments: load_data_shell_entity.png > > > beeline > > create external table default.hive_table_cloud_load_data_in_path_123 > (student_roll int, student_name string, student_dob date) ROW FORMAT > DELIMITED FIELDS TERMINATED BY ' ' STORED AS TEXTFILE location > 'hdfs://ns1/tmp/hive_table_cloud_load_data_in_path_123' > > load data inpath 'hdfs://ns1/tmp/data123.txt' into table > hive_table_cloud_load_data_in_path_123; > > Creates 'hdfs://ns1/tmp/data123.txt' as shell entity. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4301) Handle Test Case Failure on Pre-commit environment
[
https://issues.apache.org/jira/browse/ATLAS-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4301:
--
Labels: hive-hooks integration-tests (was: )
> Handle Test Case Failure on Pre-commit environment
> --
>
> Key: ATLAS-4301
> URL: https://issues.apache.org/jira/browse/ATLAS-4301
> Project: Atlas
> Issue Type: Test
> Components: hive-integration
>Affects Versions: 2.1.0
>Reporter: Mandar Ambawane
>Assignee: Mandar Ambawane
>Priority: Major
> Labels: hive-hooks, integration-tests
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-4301.patch
>
>
> Getting following error on Pre-commit build due to change in the file path
> hive-bridge/target/logs/application.log
> {code:java}
> Wrong FS:
> pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
> expected: file:///
> {code}
>
> Also,
> There is Enum "Type" in the "Entity"
> {code:java}
> Class: org.apache.hadoop.hive.ql.hooks.Entity
> Enum: Type{code}
> Enum "Type" has one constant "LOCAL_DIR"
> Due to recent changes, we need to provide support for this constant
> "LOCAL_DIR" in Testing Environment.
> Without which following issues occuring on Testing Environment:
> # While creating "hive_process" entity, The "outputs" attribute is not
> getting set (Which is of type "hdfs_path").
> # While setting the "qualifiedName" of "hive_process" entity, File path is
> not getting appended.
> This causing Failure of some Test cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4301) Handle Test Case Failure on Pre-commit environment
[
https://issues.apache.org/jira/browse/ATLAS-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4301:
--
Component/s: hive-integration
> Handle Test Case Failure on Pre-commit environment
> --
>
> Key: ATLAS-4301
> URL: https://issues.apache.org/jira/browse/ATLAS-4301
> Project: Atlas
> Issue Type: Test
> Components: hive-integration
>Affects Versions: 2.1.0
>Reporter: Mandar Ambawane
>Assignee: Mandar Ambawane
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-4301.patch
>
>
> Getting following error on Pre-commit build due to change in the file path
> hive-bridge/target/logs/application.log
> {code:java}
> Wrong FS:
> pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
> expected: file:///
> {code}
>
> Also,
> There is Enum "Type" in the "Entity"
> {code:java}
> Class: org.apache.hadoop.hive.ql.hooks.Entity
> Enum: Type{code}
> Enum "Type" has one constant "LOCAL_DIR"
> Due to recent changes, we need to provide support for this constant
> "LOCAL_DIR" in Testing Environment.
> Without which following issues occuring on Testing Environment:
> # While creating "hive_process" entity, The "outputs" attribute is not
> getting set (Which is of type "hdfs_path").
> # While setting the "qualifiedName" of "hive_process" entity, File path is
> not getting appended.
> This causing Failure of some Test cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4301) Handle Test Case Failure on Pre-commit environment
[
https://issues.apache.org/jira/browse/ATLAS-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4301:
--
Issue Type: Test (was: Improvement)
> Handle Test Case Failure on Pre-commit environment
> --
>
> Key: ATLAS-4301
> URL: https://issues.apache.org/jira/browse/ATLAS-4301
> Project: Atlas
> Issue Type: Test
>Affects Versions: 2.1.0
>Reporter: Mandar Ambawane
>Assignee: Mandar Ambawane
>Priority: Major
> Attachments: ATLAS-4301.patch
>
>
> Getting following error on Pre-commit build due to change in the file path
> hive-bridge/target/logs/application.log
> {code:java}
> Wrong FS:
> pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
> expected: file:///
> {code}
>
> Also,
> There is Enum "Type" in the "Entity"
> {code:java}
> Class: org.apache.hadoop.hive.ql.hooks.Entity
> Enum: Type{code}
> Enum "Type" has one constant "LOCAL_DIR"
> Due to recent changes, we need to provide support for this constant
> "LOCAL_DIR" in Testing Environment.
> Without which following issues occuring on Testing Environment:
> # While creating "hive_process" entity, The "outputs" attribute is not
> getting set (Which is of type "hdfs_path").
> # While setting the "qualifiedName" of "hive_process" entity, File path is
> not getting appended.
> This causing Failure of some Test cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4301) Handle Test Case Failure on Pre-commit environment
[
https://issues.apache.org/jira/browse/ATLAS-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4301:
--
Affects Version/s: 2.1.0
> Handle Test Case Failure on Pre-commit environment
> --
>
> Key: ATLAS-4301
> URL: https://issues.apache.org/jira/browse/ATLAS-4301
> Project: Atlas
> Issue Type: Improvement
>Affects Versions: 2.1.0
>Reporter: Mandar Ambawane
>Assignee: Mandar Ambawane
>Priority: Major
> Attachments: ATLAS-4301.patch
>
>
> Getting following error on Pre-commit build due to change in the file path
> hive-bridge/target/logs/application.log
> {code:java}
> Wrong FS:
> pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
> expected: file:///
> {code}
>
> Also,
> There is Enum "Type" in the "Entity"
> {code:java}
> Class: org.apache.hadoop.hive.ql.hooks.Entity
> Enum: Type{code}
> Enum "Type" has one constant "LOCAL_DIR"
> Due to recent changes, we need to provide support for this constant
> "LOCAL_DIR" in Testing Environment.
> Without which following issues occuring on Testing Environment:
> # While creating "hive_process" entity, The "outputs" attribute is not
> getting set (Which is of type "hdfs_path").
> # While setting the "qualifiedName" of "hive_process" entity, File path is
> not getting appended.
> This causing Failure of some Test cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4301) Handle Test Case Failure on Pre-commit environment
[
https://issues.apache.org/jira/browse/ATLAS-4301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4301:
--
Fix Version/s: 2.2.0
3.0.0
> Handle Test Case Failure on Pre-commit environment
> --
>
> Key: ATLAS-4301
> URL: https://issues.apache.org/jira/browse/ATLAS-4301
> Project: Atlas
> Issue Type: Test
>Affects Versions: 2.1.0
>Reporter: Mandar Ambawane
>Assignee: Mandar Ambawane
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-4301.patch
>
>
> Getting following error on Pre-commit build due to change in the file path
> hive-bridge/target/logs/application.log
> {code:java}
> Wrong FS:
> pfile:/home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/addons/hive-bridge/target/partition-data-{dir},
> expected: file:///
> {code}
>
> Also,
> There is Enum "Type" in the "Entity"
> {code:java}
> Class: org.apache.hadoop.hive.ql.hooks.Entity
> Enum: Type{code}
> Enum "Type" has one constant "LOCAL_DIR"
> Due to recent changes, we need to provide support for this constant
> "LOCAL_DIR" in Testing Environment.
> Without which following issues occuring on Testing Environment:
> # While creating "hive_process" entity, The "outputs" attribute is not
> getting set (Which is of type "hdfs_path").
> # While setting the "qualifiedName" of "hive_process" entity, File path is
> not getting appended.
> This causing Failure of some Test cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ATLAS-4310) NPE seen for CLASSIFICATION_PROPAGATION_DELETE Operation
[
https://issues.apache.org/jira/browse/ATLAS-4310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351942#comment-17351942
]
Sarath Subramanian commented on ATLAS-4310:
---
+1 thanks for the patch [~amestry]
> NPE seen for CLASSIFICATION_PROPAGATION_DELETE Operation
>
>
> Key: ATLAS-4310
> URL: https://issues.apache.org/jira/browse/ATLAS-4310
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: trunk
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
> Attachments: ATLAS-4310-Handled-NPE-for-DELETE-classification.patch
>
>
> *Steps to Duplicate*
> # Enabled admin tasks
> # Created an hdfs_path entity
> # In a loop for 330 times: (330 times because to generate 1000 audits)
> ## Updated entity ( updated path)
> ## Added tag1
> ## Removed tag1
> Expected results: Classification is removed.
> Actual results: Classification is removed. Logs indicate NPE:
> {code:java}
> at
> org.apache.atlas.repository.store.graph.v2.EntityGraphMapper.deleteClassificationPropagation(EntityGraphMapper.java:2595)
> at
> org.apache.atlas.repository.store.graph.v2.EntityGraphMapper.deleteClassificationPropagation(EntityGraphMapper.java:2595)
> at
> org.apache.atlas.repository.store.graph.v2.EntityGraphMapper$$FastClassBySpringCGLIB$$8e3f1c72.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:737)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:111)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:672)
> at
> org.apache.atlas.repository.store.graph.v2.EntityGraphMapper$$EnhancerBySpringCGLIB$$96822c39.deleteClassificationPropagation()
> at
> org.apache.atlas.repository.store.graph.v2.tasks.ClassificationPropagationTasks$Delete.run(ClassificationPropagationTasks.java:73)
> at
> org.apache.atlas.repository.store.graph.v2.tasks.ClassificationTask.perform(ClassificationTask.java:95)
> at org.apache.atlas.tasks.AbstractTask.run(AbstractTask.java:33) at
> org.apache.atlas.tasks.TaskExecutor$TaskConsumer.performTask(TaskExecutor.java:150)
> at
> org.apache.atlas.tasks.TaskExecutor$TaskConsumer.run(TaskExecutor.java:109)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834)Caused by:
> java.lang.NullPointerException at
> org.apache.atlas.repository.graph.GraphHelper.getTypeName(GraphHelper.java:867)
> at
> org.apache.atlas.repository.store.graph.v2.EntityGraphRetriever.toAtlasClassification(EntityGraphRetriever.java:334)
> at
> org.apache.atlas.repository.store.graph.v2.EntityGraphMapper.deleteClassificationPropagation(EntityGraphMapper.java:2572)
> ... 18 more2021-05-25 11:07:13,553 ERROR - [atlas-task-0-etp651100072-232 -
> ceaa7213-1d14-4006-8f84-d94e56f4e829:] ~ Task:
> c9f7c463-1c5d-4ae9-8232-506fd2c95a28: Error performing task!
> (ClassificationTask:99)org.apache.atlas.exception.AtlasBaseException:
> java.lang.NullPointerException at
> org.apache.atlas.repository.store.graph.v2.EntityGraphMapper.deleteClassificationPropagation(EntityGraphMapper.java:2595)
> at
> org.apache.atlas.repository.store.graph.v2.EntityGraphMapper$$FastClassBySpringCGLIB$$8e3f1c72.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:737)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:111)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:672)
> at
> org.apache.atlas.repository.store.graph.v2.EntityGraphMapper$$EnhancerBySpringCGLIB$$96822c39.deleteClassificationPropagation()
> at
>
[jira] [Updated] (ATLAS-4285) AtlasTasks: Multiple tag propagation tasks running concurrently, task is complete but propagation is not complete
[ https://issues.apache.org/jira/browse/ATLAS-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4285: -- Component/s: atlas-core > AtlasTasks: Multiple tag propagation tasks running concurrently, task is > complete but propagation is not complete > - > > Key: ATLAS-4285 > URL: https://issues.apache.org/jira/browse/ATLAS-4285 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: > ATLAS-4285-Multiple-propagations-with-intersecting-l.patch > > > Created a 500 level linear lineage . (table1 ---> table2 ---> table3 ---> > .. ---> table500) > Added tag1 to table1 > Added tag2 to table2 > Added tag3 to table3 > 3 tasks are created. > task2 got completed and tag2 is associated only to table2 and not propagated > till table500. > After sometime all tasks are completed , but propagation didn't happen -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4285) AtlasTasks: Multiple tag propagation tasks running concurrently, task is complete but propagation is not complete
[ https://issues.apache.org/jira/browse/ATLAS-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4285: -- Fix Version/s: 2.2.0 3.0.0 > AtlasTasks: Multiple tag propagation tasks running concurrently, task is > complete but propagation is not complete > - > > Key: ATLAS-4285 > URL: https://issues.apache.org/jira/browse/ATLAS-4285 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: > ATLAS-4285-Multiple-propagations-with-intersecting-l.patch > > > Created a 500 level linear lineage . (table1 ---> table2 ---> table3 ---> > .. ---> table500) > Added tag1 to table1 > Added tag2 to table2 > Added tag3 to table3 > 3 tasks are created. > task2 got completed and tag2 is associated only to table2 and not propagated > till table500. > After sometime all tasks are completed , but propagation didn't happen -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4285) AtlasTasks: Multiple tag propagation tasks running concurrently, task is complete but propagation is not complete
[ https://issues.apache.org/jira/browse/ATLAS-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4285: -- Labels: deferred-actions tagpropagation (was: ) > AtlasTasks: Multiple tag propagation tasks running concurrently, task is > complete but propagation is not complete > - > > Key: ATLAS-4285 > URL: https://issues.apache.org/jira/browse/ATLAS-4285 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Labels: deferred-actions, tagpropagation > Fix For: 3.0.0, 2.2.0 > > Attachments: > ATLAS-4285-Multiple-propagations-with-intersecting-l.patch > > > Created a 500 level linear lineage . (table1 ---> table2 ---> table3 ---> > .. ---> table500) > Added tag1 to table1 > Added tag2 to table2 > Added tag3 to table3 > 3 tasks are created. > task2 got completed and tag2 is associated only to table2 and not propagated > till table500. > After sometime all tasks are completed , but propagation didn't happen -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4285) AtlasTasks: Multiple tag propagation tasks running concurrently, task is complete but propagation is not complete
[ https://issues.apache.org/jira/browse/ATLAS-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4285: -- Affects Version/s: 2.1.0 > AtlasTasks: Multiple tag propagation tasks running concurrently, task is > complete but propagation is not complete > - > > Key: ATLAS-4285 > URL: https://issues.apache.org/jira/browse/ATLAS-4285 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Attachments: > ATLAS-4285-Multiple-propagations-with-intersecting-l.patch > > > Created a 500 level linear lineage . (table1 ---> table2 ---> table3 ---> > .. ---> table500) > Added tag1 to table1 > Added tag2 to table2 > Added tag3 to table3 > 3 tasks are created. > task2 got completed and tag2 is associated only to table2 and not propagated > till table500. > After sometime all tasks are completed , but propagation didn't happen -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4288) [Atlas: Glossary Term Bulk Import] Will all the data populated, while performing bulk import, PreferredToTerms relationship alone is not created
[
https://issues.apache.org/jira/browse/ATLAS-4288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4288:
--
Labels: bulk-glossary-import (was: )
> [Atlas: Glossary Term Bulk Import] Will all the data populated, while
> performing bulk import, PreferredToTerms relationship alone is not created
>
>
> Key: ATLAS-4288
> URL: https://issues.apache.org/jira/browse/ATLAS-4288
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Sidharth Kumar Mishra
>Priority: Major
> Labels: bulk-glossary-import
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-4288.patch, image-2021-05-17-16-43-31-487.png
>
>
> Consider the following input, here all the relations are established except
> the preferredToTerms (term_2)
> {code:java}
> GlossaryName, TermName, ShortDescription, LongDescription, Examples,
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor,
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso,
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> dharshmk_11,term_1,"short desc","long description", "Example", "G1", "Usage",
> "glossary:100%","glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2"
> dharshmk_11,term_2,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%""glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",
> dharshmk_11,term_3,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,
> dharshmk_11,term_4,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,,
> dharshmk_11,term_5,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%","glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2"
> dharshmk_11,term_6,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%""glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",
> dharshmk_11,term_7,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,
> dharshmk_11,term_8,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,,
> dharshmk_11,term_9,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%","glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2"
> dharshmk_11,term_10,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%""glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",
> dharshmk_11,term_11,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,
> dharshmk_11,term_12,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,,
> dharshmk_11,term_13,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%","glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2"
> {code}
>
> Before the above import happens, please do the initial import of the related
> terms with the following input
> {code:java}
> GlossaryName, TermName, ShortDescription, LongDescription, Examples,
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor,
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso,
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> glossaryBulkImport_1,termBulkImport_1
> glossaryBulkImport_1,termBulkImport_2
> glossaryBulkImport_1,termBulkImport_3
> glossaryBulkImport_1,termBulkImport_4
> glossaryBulkImport_1,termBulkImport_5
> glossaryBulkImport_2,termBulkImport_1
> glossaryBulkImport_2,termBulkImport_2
> glossaryBulkImport_2,termBulkImport_3
> glossaryBulkImport_2,termBulkImport_4
> glossaryBulkImport_2,termBulkImport_5
> glossaryBulkImport_3,termBulkImport_1
> glossaryBulkImport_3,termBulkImport_2
> glossaryBulkImport_3,termBulkImport_3
> glossaryBulkImport_3,termBulkImport_4
>
[jira] [Updated] (ATLAS-4288) [Atlas: Glossary Term Bulk Import] Will all the data populated, while performing bulk import, PreferredToTerms relationship alone is not created
[
https://issues.apache.org/jira/browse/ATLAS-4288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4288:
--
Affects Version/s: 2.1.0
> [Atlas: Glossary Term Bulk Import] Will all the data populated, while
> performing bulk import, PreferredToTerms relationship alone is not created
>
>
> Key: ATLAS-4288
> URL: https://issues.apache.org/jira/browse/ATLAS-4288
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Sidharth Kumar Mishra
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-4288.patch, image-2021-05-17-16-43-31-487.png
>
>
> Consider the following input, here all the relations are established except
> the preferredToTerms (term_2)
> {code:java}
> GlossaryName, TermName, ShortDescription, LongDescription, Examples,
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor,
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso,
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> dharshmk_11,term_1,"short desc","long description", "Example", "G1", "Usage",
> "glossary:100%","glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2"
> dharshmk_11,term_2,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%""glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",
> dharshmk_11,term_3,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,
> dharshmk_11,term_4,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,,
> dharshmk_11,term_5,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%","glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2"
> dharshmk_11,term_6,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%""glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",
> dharshmk_11,term_7,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,
> dharshmk_11,term_8,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,,
> dharshmk_11,term_9,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%","glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2"
> dharshmk_11,term_10,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%""glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",
> dharshmk_11,term_11,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,
> dharshmk_11,term_12,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,,
> dharshmk_11,term_13,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%","glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2"
> {code}
>
> Before the above import happens, please do the initial import of the related
> terms with the following input
> {code:java}
> GlossaryName, TermName, ShortDescription, LongDescription, Examples,
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor,
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso,
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> glossaryBulkImport_1,termBulkImport_1
> glossaryBulkImport_1,termBulkImport_2
> glossaryBulkImport_1,termBulkImport_3
> glossaryBulkImport_1,termBulkImport_4
> glossaryBulkImport_1,termBulkImport_5
> glossaryBulkImport_2,termBulkImport_1
> glossaryBulkImport_2,termBulkImport_2
> glossaryBulkImport_2,termBulkImport_3
> glossaryBulkImport_2,termBulkImport_4
> glossaryBulkImport_2,termBulkImport_5
> glossaryBulkImport_3,termBulkImport_1
> glossaryBulkImport_3,termBulkImport_2
> glossaryBulkImport_3,termBulkImport_3
> glossaryBulkImport_3,termBulkImport_4
> glossaryBulkImport_3,termBulkImport_5
>
[jira] [Updated] (ATLAS-4288) [Atlas: Glossary Term Bulk Import] Will all the data populated, while performing bulk import, PreferredToTerms relationship alone is not created
[
https://issues.apache.org/jira/browse/ATLAS-4288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4288:
--
Fix Version/s: 2.2.0
3.0.0
> [Atlas: Glossary Term Bulk Import] Will all the data populated, while
> performing bulk import, PreferredToTerms relationship alone is not created
>
>
> Key: ATLAS-4288
> URL: https://issues.apache.org/jira/browse/ATLAS-4288
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Sidharth Kumar Mishra
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-4288.patch, image-2021-05-17-16-43-31-487.png
>
>
> Consider the following input, here all the relations are established except
> the preferredToTerms (term_2)
> {code:java}
> GlossaryName, TermName, ShortDescription, LongDescription, Examples,
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor,
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso,
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> dharshmk_11,term_1,"short desc","long description", "Example", "G1", "Usage",
> "glossary:100%","glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2"
> dharshmk_11,term_2,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%""glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",
> dharshmk_11,term_3,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,
> dharshmk_11,term_4,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,,
> dharshmk_11,term_5,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%","glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2"
> dharshmk_11,term_6,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%""glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",
> dharshmk_11,term_7,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,
> dharshmk_11,term_8,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,,
> dharshmk_11,term_9,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%","glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2"
> dharshmk_11,term_10,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%""glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",
> dharshmk_11,term_11,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,
> dharshmk_11,term_12,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%",,"glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2",,,
> dharshmk_11,term_13,"short desc","long description", "Example", "G1",
> "Usage",
> "glossary:100%","glossaryBulkImport_1:termBulkImport_1|glossaryBulkImport_2:termBulkImport_2"
> {code}
>
> Before the above import happens, please do the initial import of the related
> terms with the following input
> {code:java}
> GlossaryName, TermName, ShortDescription, LongDescription, Examples,
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor,
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso,
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> glossaryBulkImport_1,termBulkImport_1
> glossaryBulkImport_1,termBulkImport_2
> glossaryBulkImport_1,termBulkImport_3
> glossaryBulkImport_1,termBulkImport_4
> glossaryBulkImport_1,termBulkImport_5
> glossaryBulkImport_2,termBulkImport_1
> glossaryBulkImport_2,termBulkImport_2
> glossaryBulkImport_2,termBulkImport_3
> glossaryBulkImport_2,termBulkImport_4
> glossaryBulkImport_2,termBulkImport_5
> glossaryBulkImport_3,termBulkImport_1
> glossaryBulkImport_3,termBulkImport_2
> glossaryBulkImport_3,termBulkImport_3
> glossaryBulkImport_3,termBulkImport_4
> glossaryBulkImport_3,termBulkImport_5
>
[jira] [Updated] (ATLAS-4274) [Atlas: Glossary] Non matching relation are created via bulk import
[
https://issues.apache.org/jira/browse/ATLAS-4274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4274:
--
Fix Version/s: 2.2.0
3.0.0
> [Atlas: Glossary] Non matching relation are created via bulk import
> ---
>
> Key: ATLAS-4274
> URL: https://issues.apache.org/jira/browse/ATLAS-4274
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Sidharth Kumar Mishra
>Priority: Major
> Labels: bulk-glossary-import, glossary
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-4274.patch, Screenshot 2021-05-04 at 3.31.00
> PM.png, Screenshot 2021-05-04 at 3.34.03 PM.png, Screenshot 2021-05-04 at
> 3.34.21 PM.png, Screenshot 2021-05-04 at 3.34.36 PM.png
>
>
> The related terms provided in the input does not match the relation created
> via import
> {code:java}
> GlossaryName, TermName, ShortDescription, LongDescription, Examples,
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor,
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso,
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> a_glossary_1,term_1,,,"a_glossary_1:term_2"
> a_glossary_1,term_2,"a_glossary_1:term_3",,
> a_glossary_1,term_3,,"a_glossary_1:term_1", {code}
> !Screenshot 2021-05-04 at 3.31.00 PM.png|width=1973,height=127!
> !Screenshot 2021-05-04 at 3.34.03 PM.png|width=1038,height=578!
> !Screenshot 2021-05-04 at 3.34.21 PM.png|width=1005,height=563!
> !Screenshot 2021-05-04 at 3.34.36 PM.png|width=541,height=303!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4274) [Atlas: Glossary] Non matching relation are created via bulk import
[
https://issues.apache.org/jira/browse/ATLAS-4274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4274:
--
Labels: bulk-glossary-import glossary (was: )
> [Atlas: Glossary] Non matching relation are created via bulk import
> ---
>
> Key: ATLAS-4274
> URL: https://issues.apache.org/jira/browse/ATLAS-4274
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Sidharth Kumar Mishra
>Priority: Major
> Labels: bulk-glossary-import, glossary
> Attachments: ATLAS-4274.patch, Screenshot 2021-05-04 at 3.31.00
> PM.png, Screenshot 2021-05-04 at 3.34.03 PM.png, Screenshot 2021-05-04 at
> 3.34.21 PM.png, Screenshot 2021-05-04 at 3.34.36 PM.png
>
>
> The related terms provided in the input does not match the relation created
> via import
> {code:java}
> GlossaryName, TermName, ShortDescription, LongDescription, Examples,
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor,
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso,
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> a_glossary_1,term_1,,,"a_glossary_1:term_2"
> a_glossary_1,term_2,"a_glossary_1:term_3",,
> a_glossary_1,term_3,,"a_glossary_1:term_1", {code}
> !Screenshot 2021-05-04 at 3.31.00 PM.png|width=1973,height=127!
> !Screenshot 2021-05-04 at 3.34.03 PM.png|width=1038,height=578!
> !Screenshot 2021-05-04 at 3.34.21 PM.png|width=1005,height=563!
> !Screenshot 2021-05-04 at 3.34.36 PM.png|width=541,height=303!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4274) [Atlas: Glossary] Non matching relation are created via bulk import
[
https://issues.apache.org/jira/browse/ATLAS-4274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4274:
--
Component/s: atlas-core
> [Atlas: Glossary] Non matching relation are created via bulk import
> ---
>
> Key: ATLAS-4274
> URL: https://issues.apache.org/jira/browse/ATLAS-4274
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Sidharth Kumar Mishra
>Priority: Major
> Attachments: ATLAS-4274.patch, Screenshot 2021-05-04 at 3.31.00
> PM.png, Screenshot 2021-05-04 at 3.34.03 PM.png, Screenshot 2021-05-04 at
> 3.34.21 PM.png, Screenshot 2021-05-04 at 3.34.36 PM.png
>
>
> The related terms provided in the input does not match the relation created
> via import
> {code:java}
> GlossaryName, TermName, ShortDescription, LongDescription, Examples,
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor,
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso,
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> a_glossary_1,term_1,,,"a_glossary_1:term_2"
> a_glossary_1,term_2,"a_glossary_1:term_3",,
> a_glossary_1,term_3,,"a_glossary_1:term_1", {code}
> !Screenshot 2021-05-04 at 3.31.00 PM.png|width=1973,height=127!
> !Screenshot 2021-05-04 at 3.34.03 PM.png|width=1038,height=578!
> !Screenshot 2021-05-04 at 3.34.21 PM.png|width=1005,height=563!
> !Screenshot 2021-05-04 at 3.34.36 PM.png|width=541,height=303!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4274) [Atlas: Glossary] Non matching relation are created via bulk import
[
https://issues.apache.org/jira/browse/ATLAS-4274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4274:
--
Affects Version/s: 2.1.0
> [Atlas: Glossary] Non matching relation are created via bulk import
> ---
>
> Key: ATLAS-4274
> URL: https://issues.apache.org/jira/browse/ATLAS-4274
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Sidharth Kumar Mishra
>Priority: Major
> Attachments: ATLAS-4274.patch, Screenshot 2021-05-04 at 3.31.00
> PM.png, Screenshot 2021-05-04 at 3.34.03 PM.png, Screenshot 2021-05-04 at
> 3.34.21 PM.png, Screenshot 2021-05-04 at 3.34.36 PM.png
>
>
> The related terms provided in the input does not match the relation created
> via import
> {code:java}
> GlossaryName, TermName, ShortDescription, LongDescription, Examples,
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor,
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso,
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> a_glossary_1,term_1,,,"a_glossary_1:term_2"
> a_glossary_1,term_2,"a_glossary_1:term_3",,
> a_glossary_1,term_3,,"a_glossary_1:term_1", {code}
> !Screenshot 2021-05-04 at 3.31.00 PM.png|width=1973,height=127!
> !Screenshot 2021-05-04 at 3.34.03 PM.png|width=1038,height=578!
> !Screenshot 2021-05-04 at 3.34.21 PM.png|width=1005,height=563!
> !Screenshot 2021-05-04 at 3.34.36 PM.png|width=541,height=303!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4284) Pruned tables are ignored
[ https://issues.apache.org/jira/browse/ATLAS-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4284: -- Affects Version/s: 2.1.0 > Pruned tables are ignored > - > > Key: ATLAS-4284 > URL: https://issues.apache.org/jira/browse/ATLAS-4284 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > > Settings in Atlas Server: > atlas.notification.consumer.preprocess.hive_table.prune.pattern=db2.* > No config changes for prune/update made in hive. > With the above settings , created db2.table1. The table is ignored instead of > getting pruned. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4284) Pruned tables are ignored
[ https://issues.apache.org/jira/browse/ATLAS-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4284: -- Labels: hive-hooks (was: ) > Pruned tables are ignored > - > > Key: ATLAS-4284 > URL: https://issues.apache.org/jira/browse/ATLAS-4284 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Labels: hive-hooks > Fix For: 3.0.0, 2.2.0 > > > Settings in Atlas Server: > atlas.notification.consumer.preprocess.hive_table.prune.pattern=db2.* > No config changes for prune/update made in hive. > With the above settings , created db2.table1. The table is ignored instead of > getting pruned. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4284) Pruned tables are ignored
[ https://issues.apache.org/jira/browse/ATLAS-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4284: -- Fix Version/s: 2.2.0 3.0.0 > Pruned tables are ignored > - > > Key: ATLAS-4284 > URL: https://issues.apache.org/jira/browse/ATLAS-4284 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > Settings in Atlas Server: > atlas.notification.consumer.preprocess.hive_table.prune.pattern=db2.* > No config changes for prune/update made in hive. > With the above settings , created db2.table1. The table is ignored instead of > getting pruned. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4284) Pruned tables are ignored
[ https://issues.apache.org/jira/browse/ATLAS-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4284: -- Component/s: atlas-core > Pruned tables are ignored > - > > Key: ATLAS-4284 > URL: https://issues.apache.org/jira/browse/ATLAS-4284 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > Settings in Atlas Server: > atlas.notification.consumer.preprocess.hive_table.prune.pattern=db2.* > No config changes for prune/update made in hive. > With the above settings , created db2.table1. The table is ignored instead of > getting pruned. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4164) [Atlas: Spooling] Tables created after spooling are created before the spooled tables when there is multiple frequent restart in kafka brokers
[
https://issues.apache.org/jira/browse/ATLAS-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4164:
--
Labels: spool (was: )
> [Atlas: Spooling] Tables created after spooling are created before the
> spooled tables when there is multiple frequent restart in kafka brokers
> --
>
> Key: ATLAS-4164
> URL: https://issues.apache.org/jira/browse/ATLAS-4164
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Ashutosh Mestry
>Priority: Major
> Labels: spool
> Fix For: 3.0.0
>
> Attachments: ATLAS-4164-Spooling-Status.patch
>
>
> Scenario:
> * Stop kafka broker
> * Create a few (20) tables save the prefix (abc_table_1, abc_table_2, ...
> abc_table_n)
> * Make sure the data is spooled
> * Start kafka and create a few more tables (xyz_table_1, xyz_table_2, ...
> xyz_table_n)
> * Wait for 5 mins for the tables to reflect in atlas
> In this case we expect all the abc_table_* to be created before xyz_table_1,
> meaning all the spooled tables are created before the tables that are created
> after spooling.
>
> Observation:
> createTime of some spooled tables is greater than the create time of the
> xyz_table_1
>
> Sample data:
> createTime for tables that are spooled:
> {code:java}
> [1613573518284, 1613573531470, 1613573531861, 1613573529446, 1613573543253,
> 1613573525390, 1613573525950, 1613573517796, 1613573518284, 1613573522629,
> 1613573513524, 1613573524856, 1613573518992, 1613573519477, 1613573519947,
> 1613573521737, 1613573514066, 1613573514555, 1613573515065,
> 1613573515605]{code}
> createTime for tables that are created after spooling:
> {code:java}
> [1613573540582, 1613573541300, 1613573551691, 1613573552628, 1613573553356,
> 1613573555478, 1613573556275, 1613573556940, 1613573557763, 1613573558659,
> 1613573560673, 1613573561363, 1613573562310, 1613573563096, 1613573564004,
> 1613573566533, 1613573567602, 1613573568439, 1613573569379, 1613573570202]
> {code}
> We expect all spooled tables to have createTime smaller than the tables
> created after spooling.
> But *1613573543253 (Spooled tabled create time) is greater than 1613573540582
> (table created after spooling)*
> which means, the table created after spooling is created before spooled table
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ATLAS-4164) [Atlas: Spooling] Tables created after spooling are created before the spooled tables when there is multiple frequent restart in kafka brokers
[
https://issues.apache.org/jira/browse/ATLAS-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian reassigned ATLAS-4164:
-
Assignee: Radhika Kundam (was: Ashutosh Mestry)
> [Atlas: Spooling] Tables created after spooling are created before the
> spooled tables when there is multiple frequent restart in kafka brokers
> --
>
> Key: ATLAS-4164
> URL: https://issues.apache.org/jira/browse/ATLAS-4164
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Radhika Kundam
>Priority: Major
> Labels: spool
> Fix For: 3.0.0
>
> Attachments: ATLAS-4164-Spooling-Status.patch
>
>
> Scenario:
> * Stop kafka broker
> * Create a few (20) tables save the prefix (abc_table_1, abc_table_2, ...
> abc_table_n)
> * Make sure the data is spooled
> * Start kafka and create a few more tables (xyz_table_1, xyz_table_2, ...
> xyz_table_n)
> * Wait for 5 mins for the tables to reflect in atlas
> In this case we expect all the abc_table_* to be created before xyz_table_1,
> meaning all the spooled tables are created before the tables that are created
> after spooling.
>
> Observation:
> createTime of some spooled tables is greater than the create time of the
> xyz_table_1
>
> Sample data:
> createTime for tables that are spooled:
> {code:java}
> [1613573518284, 1613573531470, 1613573531861, 1613573529446, 1613573543253,
> 1613573525390, 1613573525950, 1613573517796, 1613573518284, 1613573522629,
> 1613573513524, 1613573524856, 1613573518992, 1613573519477, 1613573519947,
> 1613573521737, 1613573514066, 1613573514555, 1613573515065,
> 1613573515605]{code}
> createTime for tables that are created after spooling:
> {code:java}
> [1613573540582, 1613573541300, 1613573551691, 1613573552628, 1613573553356,
> 1613573555478, 1613573556275, 1613573556940, 1613573557763, 1613573558659,
> 1613573560673, 1613573561363, 1613573562310, 1613573563096, 1613573564004,
> 1613573566533, 1613573567602, 1613573568439, 1613573569379, 1613573570202]
> {code}
> We expect all spooled tables to have createTime smaller than the tables
> created after spooling.
> But *1613573543253 (Spooled tabled create time) is greater than 1613573540582
> (table created after spooling)*
> which means, the table created after spooling is created before spooled table
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4164) [Atlas: Spooling] Tables created after spooling are created before the spooled tables when there is multiple frequent restart in kafka brokers
[
https://issues.apache.org/jira/browse/ATLAS-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4164:
--
Affects Version/s: (was: 3.0.0)
> [Atlas: Spooling] Tables created after spooling are created before the
> spooled tables when there is multiple frequent restart in kafka brokers
> --
>
> Key: ATLAS-4164
> URL: https://issues.apache.org/jira/browse/ATLAS-4164
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Ashutosh Mestry
>Priority: Major
> Attachments: ATLAS-4164-Spooling-Status.patch
>
>
> Scenario:
> * Stop kafka broker
> * Create a few (20) tables save the prefix (abc_table_1, abc_table_2, ...
> abc_table_n)
> * Make sure the data is spooled
> * Start kafka and create a few more tables (xyz_table_1, xyz_table_2, ...
> xyz_table_n)
> * Wait for 5 mins for the tables to reflect in atlas
> In this case we expect all the abc_table_* to be created before xyz_table_1,
> meaning all the spooled tables are created before the tables that are created
> after spooling.
>
> Observation:
> createTime of some spooled tables is greater than the create time of the
> xyz_table_1
>
> Sample data:
> createTime for tables that are spooled:
> {code:java}
> [1613573518284, 1613573531470, 1613573531861, 1613573529446, 1613573543253,
> 1613573525390, 1613573525950, 1613573517796, 1613573518284, 1613573522629,
> 1613573513524, 1613573524856, 1613573518992, 1613573519477, 1613573519947,
> 1613573521737, 1613573514066, 1613573514555, 1613573515065,
> 1613573515605]{code}
> createTime for tables that are created after spooling:
> {code:java}
> [1613573540582, 1613573541300, 1613573551691, 1613573552628, 1613573553356,
> 1613573555478, 1613573556275, 1613573556940, 1613573557763, 1613573558659,
> 1613573560673, 1613573561363, 1613573562310, 1613573563096, 1613573564004,
> 1613573566533, 1613573567602, 1613573568439, 1613573569379, 1613573570202]
> {code}
> We expect all spooled tables to have createTime smaller than the tables
> created after spooling.
> But *1613573543253 (Spooled tabled create time) is greater than 1613573540582
> (table created after spooling)*
> which means, the table created after spooling is created before spooled table
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4164) [Atlas: Spooling] Tables created after spooling are created before the spooled tables when there is multiple frequent restart in kafka brokers
[
https://issues.apache.org/jira/browse/ATLAS-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4164:
--
Affects Version/s: 3.0.0
2.1.0
> [Atlas: Spooling] Tables created after spooling are created before the
> spooled tables when there is multiple frequent restart in kafka brokers
> --
>
> Key: ATLAS-4164
> URL: https://issues.apache.org/jira/browse/ATLAS-4164
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.1.0, 3.0.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Ashutosh Mestry
>Priority: Major
> Attachments: ATLAS-4164-Spooling-Status.patch
>
>
> Scenario:
> * Stop kafka broker
> * Create a few (20) tables save the prefix (abc_table_1, abc_table_2, ...
> abc_table_n)
> * Make sure the data is spooled
> * Start kafka and create a few more tables (xyz_table_1, xyz_table_2, ...
> xyz_table_n)
> * Wait for 5 mins for the tables to reflect in atlas
> In this case we expect all the abc_table_* to be created before xyz_table_1,
> meaning all the spooled tables are created before the tables that are created
> after spooling.
>
> Observation:
> createTime of some spooled tables is greater than the create time of the
> xyz_table_1
>
> Sample data:
> createTime for tables that are spooled:
> {code:java}
> [1613573518284, 1613573531470, 1613573531861, 1613573529446, 1613573543253,
> 1613573525390, 1613573525950, 1613573517796, 1613573518284, 1613573522629,
> 1613573513524, 1613573524856, 1613573518992, 1613573519477, 1613573519947,
> 1613573521737, 1613573514066, 1613573514555, 1613573515065,
> 1613573515605]{code}
> createTime for tables that are created after spooling:
> {code:java}
> [1613573540582, 1613573541300, 1613573551691, 1613573552628, 1613573553356,
> 1613573555478, 1613573556275, 1613573556940, 1613573557763, 1613573558659,
> 1613573560673, 1613573561363, 1613573562310, 1613573563096, 1613573564004,
> 1613573566533, 1613573567602, 1613573568439, 1613573569379, 1613573570202]
> {code}
> We expect all spooled tables to have createTime smaller than the tables
> created after spooling.
> But *1613573543253 (Spooled tabled create time) is greater than 1613573540582
> (table created after spooling)*
> which means, the table created after spooling is created before spooled table
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4164) [Atlas: Spooling] Tables created after spooling are created before the spooled tables when there is multiple frequent restart in kafka brokers
[
https://issues.apache.org/jira/browse/ATLAS-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4164:
--
Fix Version/s: 3.0.0
> [Atlas: Spooling] Tables created after spooling are created before the
> spooled tables when there is multiple frequent restart in kafka brokers
> --
>
> Key: ATLAS-4164
> URL: https://issues.apache.org/jira/browse/ATLAS-4164
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Ashutosh Mestry
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: ATLAS-4164-Spooling-Status.patch
>
>
> Scenario:
> * Stop kafka broker
> * Create a few (20) tables save the prefix (abc_table_1, abc_table_2, ...
> abc_table_n)
> * Make sure the data is spooled
> * Start kafka and create a few more tables (xyz_table_1, xyz_table_2, ...
> xyz_table_n)
> * Wait for 5 mins for the tables to reflect in atlas
> In this case we expect all the abc_table_* to be created before xyz_table_1,
> meaning all the spooled tables are created before the tables that are created
> after spooling.
>
> Observation:
> createTime of some spooled tables is greater than the create time of the
> xyz_table_1
>
> Sample data:
> createTime for tables that are spooled:
> {code:java}
> [1613573518284, 1613573531470, 1613573531861, 1613573529446, 1613573543253,
> 1613573525390, 1613573525950, 1613573517796, 1613573518284, 1613573522629,
> 1613573513524, 1613573524856, 1613573518992, 1613573519477, 1613573519947,
> 1613573521737, 1613573514066, 1613573514555, 1613573515065,
> 1613573515605]{code}
> createTime for tables that are created after spooling:
> {code:java}
> [1613573540582, 1613573541300, 1613573551691, 1613573552628, 1613573553356,
> 1613573555478, 1613573556275, 1613573556940, 1613573557763, 1613573558659,
> 1613573560673, 1613573561363, 1613573562310, 1613573563096, 1613573564004,
> 1613573566533, 1613573567602, 1613573568439, 1613573569379, 1613573570202]
> {code}
> We expect all spooled tables to have createTime smaller than the tables
> created after spooling.
> But *1613573543253 (Spooled tabled create time) is greater than 1613573540582
> (table created after spooling)*
> which means, the table created after spooling is created before spooled table
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4183) web.xml requires external dtd resource
[ https://issues.apache.org/jira/browse/ATLAS-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4183: -- Fix Version/s: 2.2.0 3.0.0 > web.xml requires external dtd resource > -- > > Key: ATLAS-4183 > URL: https://issues.apache.org/jira/browse/ATLAS-4183 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > We have Servlet 2.3 deployment descriptor (web.xml), which follows J2EE 1.3 > DTDs schema. > Therefore it requires dtd resource hosted at > http://java.sun.com/dtd/web-app_2_3.dtd > In setups that are behind firewalls with limited access to external > resources, the Atlas server fails to come up without the external dtd > resource. > This can be fixed by hosting dtd resource locally or upgrading the deployment > descriptor. > We can upgrade it to Servlet 2.5 deployment descriptor which is Java EE 5 XML > schema and it does not require external dtd resource for validation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4183) web.xml requires external dtd resource
[ https://issues.apache.org/jira/browse/ATLAS-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4183: -- Affects Version/s: 3.0.0 2.1.0 > web.xml requires external dtd resource > -- > > Key: ATLAS-4183 > URL: https://issues.apache.org/jira/browse/ATLAS-4183 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0, 3.0.0 >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Major > > We have Servlet 2.3 deployment descriptor (web.xml), which follows J2EE 1.3 > DTDs schema. > Therefore it requires dtd resource hosted at > http://java.sun.com/dtd/web-app_2_3.dtd > In setups that are behind firewalls with limited access to external > resources, the Atlas server fails to come up without the external dtd > resource. > This can be fixed by hosting dtd resource locally or upgrading the deployment > descriptor. > We can upgrade it to Servlet 2.5 deployment descriptor which is Java EE 5 XML > schema and it does not require external dtd resource for validation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4183) web.xml requires external dtd resource
[ https://issues.apache.org/jira/browse/ATLAS-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4183: -- Affects Version/s: (was: 3.0.0) > web.xml requires external dtd resource > -- > > Key: ATLAS-4183 > URL: https://issues.apache.org/jira/browse/ATLAS-4183 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Major > > We have Servlet 2.3 deployment descriptor (web.xml), which follows J2EE 1.3 > DTDs schema. > Therefore it requires dtd resource hosted at > http://java.sun.com/dtd/web-app_2_3.dtd > In setups that are behind firewalls with limited access to external > resources, the Atlas server fails to come up without the external dtd > resource. > This can be fixed by hosting dtd resource locally or upgrading the deployment > descriptor. > We can upgrade it to Servlet 2.5 deployment descriptor which is Java EE 5 XML > schema and it does not require external dtd resource for validation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4278) Deferred Actions : Deleting a tag after disassociating it from a table which propagated fails
[ https://issues.apache.org/jira/browse/ATLAS-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4278: -- Fix Version/s: 2.2.0 3.0.0 > Deferred Actions : Deleting a tag after disassociating it from a table which > propagated fails > - > > Key: ATLAS-4278 > URL: https://issues.apache.org/jira/browse/ATLAS-4278 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Jayendra Parab >Assignee: Jayendra Parab >Priority: Major > Labels: deferred-actions > Fix For: 3.0.0, 2.2.0 > > > # Create a lineage : table1 ---> process1 > table2 > # Add tag tag1 to table1. > # Wait for task to complete, it propagates to table2 > # Disassociate tag from table1 > # Wait for task to complete , it is removed from all tables and process. > # Attempt to delete the tag. > # Tag deletion fails with `tag1` has references , though the tag is not > associated to any entity. > If the tag was added to entity without propagate flag set to False , and then > disassociated and deleted , it is deleted successfully. > This issue is seen only with deferred actions enabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4278) Deferred Actions : Deleting a tag after disassociating it from a table which propagated fails
[ https://issues.apache.org/jira/browse/ATLAS-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4278: -- Component/s: atlas-core > Deferred Actions : Deleting a tag after disassociating it from a table which > propagated fails > - > > Key: ATLAS-4278 > URL: https://issues.apache.org/jira/browse/ATLAS-4278 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Jayendra Parab >Assignee: Jayendra Parab >Priority: Major > > # Create a lineage : table1 ---> process1 > table2 > # Add tag tag1 to table1. > # Wait for task to complete, it propagates to table2 > # Disassociate tag from table1 > # Wait for task to complete , it is removed from all tables and process. > # Attempt to delete the tag. > # Tag deletion fails with `tag1` has references , though the tag is not > associated to any entity. > If the tag was added to entity without propagate flag set to False , and then > disassociated and deleted , it is deleted successfully. > This issue is seen only with deferred actions enabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4278) Deferred Actions : Deleting a tag after disassociating it from a table which propagated fails
[ https://issues.apache.org/jira/browse/ATLAS-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4278: -- Labels: deferred-actions (was: ) > Deferred Actions : Deleting a tag after disassociating it from a table which > propagated fails > - > > Key: ATLAS-4278 > URL: https://issues.apache.org/jira/browse/ATLAS-4278 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Jayendra Parab >Assignee: Jayendra Parab >Priority: Major > Labels: deferred-actions > > # Create a lineage : table1 ---> process1 > table2 > # Add tag tag1 to table1. > # Wait for task to complete, it propagates to table2 > # Disassociate tag from table1 > # Wait for task to complete , it is removed from all tables and process. > # Attempt to delete the tag. > # Tag deletion fails with `tag1` has references , though the tag is not > associated to any entity. > If the tag was added to entity without propagate flag set to False , and then > disassociated and deleted , it is deleted successfully. > This issue is seen only with deferred actions enabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4106) Adding Debug metrics to atlas.
[ https://issues.apache.org/jira/browse/ATLAS-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian resolved ATLAS-4106. --- Resolution: Fixed > Adding Debug metrics to atlas. > -- > > Key: ATLAS-4106 > URL: https://issues.apache.org/jira/browse/ATLAS-4106 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Mayank Jain >Assignee: Mayank Jain >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > This is a new feature added to atlas to create a ease for keeping track of > certain highly used api's in atlas , so at the time of evaluating the > performance of certain API we could have our data ready. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4106) Adding Debug metrics to atlas.
[ https://issues.apache.org/jira/browse/ATLAS-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4106: -- Component/s: atlas-core > Adding Debug metrics to atlas. > -- > > Key: ATLAS-4106 > URL: https://issues.apache.org/jira/browse/ATLAS-4106 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Mayank Jain >Assignee: Mayank Jain >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > This is a new feature added to atlas to create a ease for keeping track of > certain highly used api's in atlas , so at the time of evaluating the > performance of certain API we could have our data ready. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4106) Adding Debug metrics to atlas.
[ https://issues.apache.org/jira/browse/ATLAS-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4106: -- Affects Version/s: 2.1.0 > Adding Debug metrics to atlas. > -- > > Key: ATLAS-4106 > URL: https://issues.apache.org/jira/browse/ATLAS-4106 > Project: Atlas > Issue Type: New Feature >Affects Versions: 2.1.0 >Reporter: Mayank Jain >Assignee: Mayank Jain >Priority: Major > > This is a new feature added to atlas to create a ease for keeping track of > certain highly used api's in atlas , so at the time of evaluating the > performance of certain API we could have our data ready. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4106) Adding Debug metrics to atlas.
[ https://issues.apache.org/jira/browse/ATLAS-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4106: -- Fix Version/s: 2.2.0 3.0.0 > Adding Debug metrics to atlas. > -- > > Key: ATLAS-4106 > URL: https://issues.apache.org/jira/browse/ATLAS-4106 > Project: Atlas > Issue Type: New Feature >Affects Versions: 2.1.0 >Reporter: Mayank Jain >Assignee: Mayank Jain >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > This is a new feature added to atlas to create a ease for keeping track of > certain highly used api's in atlas , so at the time of evaluating the > performance of certain API we could have our data ready. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4269) Deferred Actions : When a tag is propagated from an entity via 2 processes , blocking 1 process removes tag propagated from another process
[ https://issues.apache.org/jira/browse/ATLAS-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4269: -- Attachment: default...@cm.png > Deferred Actions : When a tag is propagated from an entity via 2 processes , > blocking 1 process removes tag propagated from another process > --- > > Key: ATLAS-4269 > URL: https://issues.apache.org/jira/browse/ATLAS-4269 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Attachments: default...@cm.png > > > 1.Create a lineage as attached in the screenshot (i.e) tag propagated from a > source using 2 processes to a target > 2. Add a tag to t3 , now the tag is propagated to t4 via both process1 and > process2. > 3. Now block the propagation from t3 to process1. > 4. Expectation is that , tag will be propagated via process2 to t4. But the > propagated tag is removed from t4 > !default...@cm.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4269) Deferred Actions : When a tag is propagated from an entity via 2 processes , blocking 1 process removes tag propagated from another process
[ https://issues.apache.org/jira/browse/ATLAS-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4269: -- Labels: tagpropagation (was: ) > Deferred Actions : When a tag is propagated from an entity via 2 processes , > blocking 1 process removes tag propagated from another process > --- > > Key: ATLAS-4269 > URL: https://issues.apache.org/jira/browse/ATLAS-4269 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Labels: tagpropagation > Fix For: 3.0.0, 2.2.0 > > Attachments: default...@cm.png > > > 1.Create a lineage as attached in the screenshot (i.e) tag propagated from a > source using 2 processes to a target > 2. Add a tag to t3 , now the tag is propagated to t4 via both process1 and > process2. > 3. Now block the propagation from t3 to process1. > 4. Expectation is that , tag will be propagated via process2 to t4. But the > propagated tag is removed from t4 > !default...@cm.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-4269) Deferred Actions : When a tag is propagated from an entity via 2 processes , blocking 1 process removes tag propagated from another process
Sarath Subramanian created ATLAS-4269: - Summary: Deferred Actions : When a tag is propagated from an entity via 2 processes , blocking 1 process removes tag propagated from another process Key: ATLAS-4269 URL: https://issues.apache.org/jira/browse/ATLAS-4269 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 2.1.0 Reporter: Sarath Subramanian Assignee: Sarath Subramanian Attachments: default...@cm.png 1.Create a lineage as attached in the screenshot (i.e) tag propagated from a source using 2 processes to a target 2. Add a tag to t3 , now the tag is propagated to t4 via both process1 and process2. 3. Now block the propagation from t3 to process1. 4. Expectation is that , tag will be propagated via process2 to t4. But the propagated tag is removed from t4 !default...@cm.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4269) Deferred Actions : When a tag is propagated from an entity via 2 processes , blocking 1 process removes tag propagated from another process
[ https://issues.apache.org/jira/browse/ATLAS-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4269: -- Fix Version/s: 2.2.0 3.0.0 > Deferred Actions : When a tag is propagated from an entity via 2 processes , > blocking 1 process removes tag propagated from another process > --- > > Key: ATLAS-4269 > URL: https://issues.apache.org/jira/browse/ATLAS-4269 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: default...@cm.png > > > 1.Create a lineage as attached in the screenshot (i.e) tag propagated from a > source using 2 processes to a target > 2. Add a tag to t3 , now the tag is propagated to t4 via both process1 and > process2. > 3. Now block the propagation from t3 to process1. > 4. Expectation is that , tag will be propagated via process2 to t4. But the > propagated tag is removed from t4 > !default...@cm.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-4268) Deferred Actions : When a tag is already associated to a term and when the term is assigned to the entity , tag propagation doesn't happen
Sarath Subramanian created ATLAS-4268: - Summary: Deferred Actions : When a tag is already associated to a term and when the term is assigned to the entity , tag propagation doesn't happen Key: ATLAS-4268 URL: https://issues.apache.org/jira/browse/ATLAS-4268 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 2.1.0 Reporter: Sarath Subramanian Assignee: Sarath Subramanian Enabled deferred actions Create term term1. Add tag tag1 to term1. Associate term1 to entity1 Expected that tag1 will be propagated to entity1 from term1 , but didn't happen. Associated a new tag tag2 to term1 , now that tag2 is associated to entity1 Issue is seen only when deferred actions is enabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4261) Bulk Glossary Import Response and Failed Error Message Improvements
[ https://issues.apache.org/jira/browse/ATLAS-4261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4261: -- Fix Version/s: 2.2.0 3.0.0 > Bulk Glossary Import Response and Failed Error Message Improvements > > > Key: ATLAS-4261 > URL: https://issues.apache.org/jira/browse/ATLAS-4261 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Sidharth Kumar Mishra >Assignee: Sidharth Kumar Mishra >Priority: Major > Labels: glossary > Fix For: 3.0.0, 2.2.0 > > Attachments: ATLAS-4261.patch > > > 1. For bulk glossary import, if we have relations and there is some error > present then the failed error message contains duplicate messages. > 2. The success response doesn't contains information about the relations > 3. The failed messages at response contains extra '\n' > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4261) Bulk Glossary Import Response and Failed Error Message Improvements
[ https://issues.apache.org/jira/browse/ATLAS-4261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4261: -- Component/s: atlas-core > Bulk Glossary Import Response and Failed Error Message Improvements > > > Key: ATLAS-4261 > URL: https://issues.apache.org/jira/browse/ATLAS-4261 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Sidharth Kumar Mishra >Assignee: Sidharth Kumar Mishra >Priority: Major > Attachments: ATLAS-4261.patch > > > 1. For bulk glossary import, if we have relations and there is some error > present then the failed error message contains duplicate messages. > 2. The success response doesn't contains information about the relations > 3. The failed messages at response contains extra '\n' > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4261) Bulk Glossary Import Response and Failed Error Message Improvements
[ https://issues.apache.org/jira/browse/ATLAS-4261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17332967#comment-17332967 ] Sarath Subramanian commented on ATLAS-4261: --- Precommit - [https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/525/console] > Bulk Glossary Import Response and Failed Error Message Improvements > > > Key: ATLAS-4261 > URL: https://issues.apache.org/jira/browse/ATLAS-4261 > Project: Atlas > Issue Type: Bug >Reporter: Sidharth Kumar Mishra >Assignee: Sidharth Kumar Mishra >Priority: Major > Attachments: ATLAS-4261.patch > > > 1. For bulk glossary import, if we have relations and there is some error > present then the failed error message contains duplicate messages. > 2. The success response doesn't contains information about the relations > 3. The failed messages at response contains extra '\n' > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4261) Bulk Glossary Import Response and Failed Error Message Improvements
[ https://issues.apache.org/jira/browse/ATLAS-4261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4261: -- Affects Version/s: 2.1.0 > Bulk Glossary Import Response and Failed Error Message Improvements > > > Key: ATLAS-4261 > URL: https://issues.apache.org/jira/browse/ATLAS-4261 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Sidharth Kumar Mishra >Assignee: Sidharth Kumar Mishra >Priority: Major > Attachments: ATLAS-4261.patch > > > 1. For bulk glossary import, if we have relations and there is some error > present then the failed error message contains duplicate messages. > 2. The success response doesn't contains information about the relations > 3. The failed messages at response contains extra '\n' > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4261) Bulk Glossary Import Response and Failed Error Message Improvements
[ https://issues.apache.org/jira/browse/ATLAS-4261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4261: -- Labels: glossary (was: ) > Bulk Glossary Import Response and Failed Error Message Improvements > > > Key: ATLAS-4261 > URL: https://issues.apache.org/jira/browse/ATLAS-4261 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Sidharth Kumar Mishra >Assignee: Sidharth Kumar Mishra >Priority: Major > Labels: glossary > Attachments: ATLAS-4261.patch > > > 1. For bulk glossary import, if we have relations and there is some error > present then the failed error message contains duplicate messages. > 2. The success response doesn't contains information about the relations > 3. The failed messages at response contains extra '\n' > -- This message was sent by Atlassian Jira (v8.3.4#803005)
