[CVE-2020-13928 ] Apache Atlas Multiple XSS Vulnerability
Hello, Please find below details on CVE fixed in Apache Atlas releases *2.1.0* - CVE-2020-13928: Atlas was found vulnerable to a Cross-Site Scripting in Basic Search functionality. Severity: Critical Vendor:The Apache Software Foundation Versions affected:Apache Atlas versions 2.0.0 Users affected:Apache Atlas UI search functionality, Save Search Description: Apache Atlas Multiple XSS Vulnerability Fix detail: Apache Atlas was updated to sanitize the user input and while rendering Mitigation:Users should upgrade to 2.1.0 or later version of Apache Atlas Credit: Michał Orzechowski - Thanks, Keval
[jira] [Updated] (ATLAS-3941) NotificationHookConsumer: Reduce Retry Pause Interval
[ https://issues.apache.org/jira/browse/ATLAS-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Mestry updated ATLAS-3941: --- Attachment: ATLAS-3941-NotificationHookConsumer-Reduce-wait-time.patch > NotificationHookConsumer: Reduce Retry Pause Interval > - > > Key: ATLAS-3941 > URL: https://issues.apache.org/jira/browse/ATLAS-3941 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: trunk, 2.1.0 >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Fix For: trunk > > Attachments: > ATLAS-3941-NotificationHookConsumer-Reduce-wait-time.patch > > > *Background* > The retry logic introduced earlier, had a long wait time in case a concurrent > entity create was detected. This adversely affect ingest speed in the case > where there are a lot of errors in the data being ingested. > *Solution* > Reduce the wait time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3941) NotificationHookConsumer: Reduce Retry Pause Interval
[ https://issues.apache.org/jira/browse/ATLAS-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Mestry updated ATLAS-3941: --- Attachment: (was: ATLAS-3941-NotificationHookConsumer-Reduce-wait-time.patch) > NotificationHookConsumer: Reduce Retry Pause Interval > - > > Key: ATLAS-3941 > URL: https://issues.apache.org/jira/browse/ATLAS-3941 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: trunk, 2.1.0 >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Fix For: trunk > > Attachments: > ATLAS-3941-NotificationHookConsumer-Reduce-wait-time.patch > > > *Background* > The retry logic introduced earlier, had a long wait time in case a concurrent > entity create was detected. This adversely affect ingest speed in the case > where there are a lot of errors in the data being ingested. > *Solution* > Reduce the wait time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3941) NotificationHookConsumer: Reduce Retry Pause Interval
[ https://issues.apache.org/jira/browse/ATLAS-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17195606#comment-17195606 ] Sarath Subramanian commented on ATLAS-3941: --- Thanks for the patch. +1 > NotificationHookConsumer: Reduce Retry Pause Interval > - > > Key: ATLAS-3941 > URL: https://issues.apache.org/jira/browse/ATLAS-3941 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: trunk, 2.1.0 >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Fix For: trunk > > Attachments: > ATLAS-3941-NotificationHookConsumer-Reduce-wait-time.patch > > > *Background* > The retry logic introduced earlier, had a long wait time in case a concurrent > entity create was detected. This adversely affect ingest speed in the case > where there are a lot of errors in the data being ingested. > *Solution* > Reduce the wait time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3941) NotificationHookConsumer: Reduce Retry Pause Interval
[ https://issues.apache.org/jira/browse/ATLAS-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17195553#comment-17195553 ] Nixon Rodrigues commented on ATLAS-3941: +1 for the patch, Thanks [~amestry] > NotificationHookConsumer: Reduce Retry Pause Interval > - > > Key: ATLAS-3941 > URL: https://issues.apache.org/jira/browse/ATLAS-3941 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: trunk, 2.1.0 >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Fix For: trunk > > Attachments: > ATLAS-3941-NotificationHookConsumer-Reduce-wait-time.patch > > > *Background* > The retry logic introduced earlier, had a long wait time in case a concurrent > entity create was detected. This adversely affect ingest speed in the case > where there are a lot of errors in the data being ingested. > *Solution* > Reduce the wait time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3941) NotificationHookConsumer: Reduce Retry Pause Interval
[ https://issues.apache.org/jira/browse/ATLAS-3941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Mestry updated ATLAS-3941: --- Attachment: ATLAS-3941-NotificationHookConsumer-Reduce-wait-time.patch > NotificationHookConsumer: Reduce Retry Pause Interval > - > > Key: ATLAS-3941 > URL: https://issues.apache.org/jira/browse/ATLAS-3941 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: trunk, 2.1.0 >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Fix For: trunk > > Attachments: > ATLAS-3941-NotificationHookConsumer-Reduce-wait-time.patch > > > *Background* > The retry logic introduced earlier, had a long wait time in case a concurrent > entity create was detected. This adversely affect ingest speed in the case > where there are a lot of errors in the data being ingested. > *Solution* > Reduce the wait time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 72756: ATLAS-3918 : Regression : Renaming table has exceptions in Atlas application logs
> On Aug. 12, 2020, 1:45 a.m., Madhan Neethiraj wrote:
> > For some DDLs like 'create view', HMS doesn't have all details. So, it is
> > critical that we process DDLs in HiveServer2.
> >
> > Consider adding a new message-type ENTITY_RENAME, and for such messages
> > Atlas should ignore if source entity is not found.
I have updated patch with your suggested changes kindly review.
- chaitali
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72756/#review221551
---
On Sept. 14, 2020, 11:36 a.m., chaitali wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72756/
> ---
>
> (Updated Sept. 14, 2020, 11:36 a.m.)
>
>
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte,
> Nixon Rodrigues, and Sarath Subramanian.
>
>
> Bugs: ATLAS-3918
> https://issues.apache.org/jira/browse/ATLAS-3918
>
>
> Repository: atlas
>
>
> Description
> ---
>
> Issue : Messages are being sent for DDL operations one from HMS and
> HiveServer2 which causes duplicate messages hence it throws exception:
>
> message
> {"version":{"version":"1.0.0","versionParts":[1]},"msgCompressionKind":"NONE","msgSplitIdx":1,"msgSplitCount":1,"msgCreationTime":1594802485264,"message":{"type":"ENTITY_PARTIAL_UPDATE_V2","user":"hrt_qa","entityId":{"typeName":"hive_table","uniqueAttributes":{"qualifiedName":"default.t1@cm"}},"entity":{"entity":{"typeName":"hive_table","attributes":{"owner":"hrt_qa","tableType":"MANAGED_TABLE","temporary":false,"lastAccessTime":1594802469000,"aliases":["t1"],"createTime":1594802469000,"qualifiedName":"default.t2@cm","name":"t2","comment":null,"parameters":{"last_modified_time":"1594802481","rawDataSize":"0","numRows":"0","transient_lastDdlTime":"1594802481","bucketing_version":"2","last_modified_by":"hrt_qa","numFilesErasureCoded":"0","totalSize":"0","transactional_properties":"default","COLUMN_STATS_ACCURATE":"{\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"id\":\"true\"}}","numFiles":"0","transactional":"true"},"retention":0},"guid":"-21782793831367970","isIncomplete":fal
se,"provenanceType":0,"version":0,"proxy":false
>
> This patch fix it by providing a property "DDL_MESSAGES_ENABLE" in
> configuration when enabled the HiveServer2 will send ddl messages and if
> disabled then it will only send non ddl messages.
>
>
> Diffs
> -
>
>
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRename.java
> 6961fa7c2
>
> intg/src/main/java/org/apache/atlas/model/notification/HookNotification.java
> 5b5fa04e2
> intg/src/main/java/org/apache/atlas/utils/AtlasJson.java abeddf640
>
> webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
> f02c05fff
>
>
> Diff: https://reviews.apache.org/r/72756/diff/4/
>
>
> Testing
> ---
>
> Tested by creating and renaming the hive_table hence ddl messages are not
> sent from hiveserver2 after the property is set to false and so no exception
> is thrown:
> > create table t1(id int);
>
> > alter table t1 rename to t2;
>
>
> Thanks,
>
> chaitali
>
>
[jira] [Created] (ATLAS-3941) NotificationHookConsumer: Reduce Retry Pause Interval
Ashutosh Mestry created ATLAS-3941: -- Summary: NotificationHookConsumer: Reduce Retry Pause Interval Key: ATLAS-3941 URL: https://issues.apache.org/jira/browse/ATLAS-3941 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 2.1.0, trunk Reporter: Ashutosh Mestry Assignee: Ashutosh Mestry Fix For: trunk *Background* The retry logic introduced earlier, had a long wait time in case a concurrent entity create was detected. This adversely affect ingest speed in the case where there are a lot of errors in the data being ingested. *Solution* Reduce the wait time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-3906) “NoSuchMethodErrors” due to multiple versions of org.apache.curator:curator-client:jar
[
https://issues.apache.org/jira/browse/ATLAS-3906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17195501#comment-17195501
]
Madhan Neethiraj commented on ATLAS-3906:
-
[~Bing-ok] - thanks for the detailed description and the suggestions to address
the issue. Upgrading version of org.apache.hadoop:hadoop-common from 3.1.1 to
3.3.0 seems like a good option. Can you please submit a review with this fix?
> “NoSuchMethodErrors” due to multiple versions of
> org.apache.curator:curator-client:jar
> --
>
> Key: ATLAS-3906
> URL: https://issues.apache.org/jira/browse/ATLAS-3906
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Bing-ok
>Priority: Major
>
> h1. Issue description
> Hi, there are multiple versions of _*org.apache.curator:curator-client*_ in
> _*atlas/server-api*_. As shown in the following dependency tree, library
> _*org.apache.curator:curator-client:2.12.0*_ is transitively introduced by
> *_org.apache.hadoop:hadoop-auth:3.1.1_*, but has been managed to be version
> *_4.0.1_*.
> However, one method defined in shadowed version
> _*org.apache.curator:curator-client:2.12.0*_ is referenced by client project
> via *_org.apache.hadoop:hadoop-auth:3.1.1_*, but missing in the actually
> loaded version _*org.apache.curator:curator-client:4.0.1*_.
> For instance, the following missing method(defined in
> _*org.apache.curator:curator-client:2.12.0*_) is actually referenced by
> _*atlas/server-api*_, which will introduce a runtime error(i.e.,
> "NoSuchMethodError") into _*atlas/server-api*_.
> _*Missing method: org.codelibs.core.lang.StringUtil: java.lang.String
> newStringUnsafe(char[])*_ is invoked by _*atlas/server-api*_ via the
> following path:
> {noformat}
> paths--
> getEntity(java.lang.String)> atlas\server-api\target\classes
> java.lang.Object get(java.lang.Object)>
> Repositories\org\apache\curator\curator-client\4.0.1\curator-client-4.0.1.jar
> apply(java.lang.Object)>
> Repositories\org\apache\curator\curator-framework\4.0.1\curator-framework-4.0.1.jar
> apply(org.apache.curator.framework.api.CuratorListener)>
> Repositories\org\apache\curator\curator-framework\4.0.1\curator-framework-4.0.1.jar
> checkInterrupted(java.lang.Throwable)>{noformat}
> h1. Suggested fixing solutions
> 1. Upgrade dependency *_org.apache.hadoop:hadoop-common_* from _*3.1.1*_ to
> *_3.3.0_*. Because the newer version
> *_org.apache.hadoop:hadoop-common:3.3.0_* does not invoke the above missing
> method, such upgrading can solve the problem.
> 2. Declare a direct dependency _*org.apache.curator:curator-client:2.12.0*_
> in the pom file of _*atlas/server-api*_, to override
> _*org.apache.curator:curator-client*_'s managed version.
> 3. Use configuration attribute to unify the version of
> library _*org.apache.curator:curator-client*_ to be _*2.12.0*_ in
> _*atlas/server-api*_'s pom file.
> Please let me know which solution do you prefer? I can submit a PR to fix it.
> Thank you very much for your attention.
> Best regards,
> h1. Dependency tree
> {noformat}
> [INFO] org.apache.atlas:atlas-server-api:jar:3.0.0-SNAPSHOT
> [INFO] - org.apache.hadoop:hadoop-common:jar:3.1.1:compile
> [INFO] +- org.apache.hadoop:hadoop-auth:jar:3.1.1:compile
> [INFO] | - org.apache.curator:curator-framework:jar:4.0.1:compile (version
> managed from 2.12.0)
> [INFO] | - _*(org.apache.curator:curator-client:jar:4.0.1:compile - version
> managed from 2.12.0; omitted for duplicate)*_
> [INFO] - _*org.apache.curator:curator-client:jar:4.0.1:compile*_{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-3906) “NoSuchMethodErrors” due to multiple versions of org.apache.curator:curator-client:jar
[
https://issues.apache.org/jira/browse/ATLAS-3906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bing-ok updated ATLAS-3906:
---
Description:
h1. Issue description
Hi, there are multiple versions of _*org.apache.curator:curator-client*_ in
_*atlas/server-api*_. As shown in the following dependency tree, library
_*org.apache.curator:curator-client:2.12.0*_ is transitively introduced by
*_org.apache.hadoop:hadoop-auth:3.1.1_*, but has been managed to be version
*_4.0.1_*.
However, one method defined in shadowed version
_*org.apache.curator:curator-client:2.12.0*_ is referenced by client project
via *_org.apache.hadoop:hadoop-auth:3.1.1_*, but missing in the actually loaded
version _*org.apache.curator:curator-client:4.0.1*_.
For instance, the following missing method(defined in
_*org.apache.curator:curator-client:2.12.0*_) is actually referenced by
_*atlas/server-api*_, which will introduce a runtime error(i.e.,
"NoSuchMethodError") into _*atlas/server-api*_.
_*Missing method: org.codelibs.core.lang.StringUtil: java.lang.String
newStringUnsafe(char[])*_ is invoked by _*atlas/server-api*_ via the following
path:
{noformat}
paths--
atlas\server-api\target\classes
Repositories\org\apache\curator\curator-client\4.0.1\curator-client-4.0.1.jar
Repositories\org\apache\curator\curator-framework\4.0.1\curator-framework-4.0.1.jar
Repositories\org\apache\curator\curator-framework\4.0.1\curator-framework-4.0.1.jar
{noformat}
h1. Suggested fixing solutions
1. Upgrade dependency *_org.apache.hadoop:hadoop-common_* from _*3.1.1*_ to
*_3.3.0_*. Because the newer version *_org.apache.hadoop:hadoop-common:3.3.0_*
does not invoke the above missing method, such upgrading can solve the problem.
2. Declare a direct dependency _*org.apache.curator:curator-client:2.12.0*_ in
the pom file of _*atlas/server-api*_, to override
_*org.apache.curator:curator-client*_'s managed version.
3. Use configuration attribute to unify the version of
library _*org.apache.curator:curator-client*_ to be _*2.12.0*_ in
_*atlas/server-api*_'s pom file.
Please let me know which solution do you prefer? I can submit a PR to fix it.
Thank you very much for your attention.
Best regards,
h1. Dependency tree
{noformat}
[INFO] org.apache.atlas:atlas-server-api:jar:3.0.0-SNAPSHOT
[INFO] - org.apache.hadoop:hadoop-common:jar:3.1.1:compile
[INFO] +- org.apache.hadoop:hadoop-auth:jar:3.1.1:compile
[INFO] | - org.apache.curator:curator-framework:jar:4.0.1:compile (version
managed from 2.12.0)
[INFO] | - _*(org.apache.curator:curator-client:jar:4.0.1:compile - version
managed from 2.12.0; omitted for duplicate)*_
[INFO] - _*org.apache.curator:curator-client:jar:4.0.1:compile*_{noformat}
was:
h1. Issue description
Hi, there are multiple versions of _*org.apache.curator:curator-client*_ in
_*atlas/server-api*_. As shown in the following dependency tree, library
_*org.apache.curator:curator-client:2.12.0*_ is transitively introduced by
*_org.apache.hadoop:hadoop-auth:3.1.1_*, but has been managed to be version
*_4.0.1_*.
However, one method defined in shadowed version
_*org.apache.curator:curator-client:2.12.0*_ is referenced by client project
via *_org.apache.hadoop:hadoop-auth:3.1.1_*, but missing in the actually loaded
version _*org.apache.curator:curator-client:4.0.1*_.
For instance, the following missing method(defined in
_*org.apache.curator:curator-client:2.12.0*_) is actually referenced by
_*atlas/server-api*_, which will introduce a runtime error(i.e.,
"NoSuchMethodError") into _*atlas/server-api*_.
_*Missing method: org.codelibs.core.lang.StringUtil: java.lang.String
newStringUnsafe(char[])*_ is invoked by _*atlas/server-api*_ via the following
path:
{noformat}
paths--
atlas\server-api\target\classes
Repositories\org\apache\curator\curator-client\4.0.1\curator-client-4.0.1.jar
Repositories\org\apache\curator\curator-framework\4.0.1\curator-framework-4.0.1.jar
Repositories\org\apache\curator\curator-framework\4.0.1\curator-framework-4.0.1.jar
{noformat}
h1. Suggested fixing solutions
1. Upgrade dependency *_org.apache.hadoop:hadoop-common_* from _*3.1.1*_ to
*_3.3.0_*. Because the newer version *_org.apache.hadoop:hadoop-common:3.3.0_*
does not invoke the above missing method, such upgrading can solve the problem.
2. Declare a direct dependency _*org.apache.curator:curator-client:2.12.0*_ in
the pom file of _*atlas/server-api*_, to override
_*org.apache.curator:curator-client*_'s managed version.
3. Use configuration attribute to unify the version of
library _*org.apache.curator:curator-client*_ to be _*2.12.0*_ in
_*atlas/server-api*_'s pom file.
Please let me know which solution do you prefer? I can submit a PR to fix it.
Thank you very much for your attention.
Best regards,
h1. Dependency tree
{noformat}
[INFO] org.apache.atlas:atlas-server-api:jar:3.0.0-SNAPSHOT
[INFO] -
[jira] [Updated] (ATLAS-3906) “NoSuchMethodErrors” due to multiple versions of org.apache.curator:curator-client:jar
[
https://issues.apache.org/jira/browse/ATLAS-3906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bing-ok updated ATLAS-3906:
---
Description:
h1. Issue description
Hi, there are multiple versions of _*org.apache.curator:curator-client*_ in
_*atlas/server-api*_. As shown in the following dependency tree, library
_*org.apache.curator:curator-client:2.12.0*_ is transitively introduced by
*_org.apache.hadoop:hadoop-auth:3.1.1_*, but has been managed to be version
*_4.0.1_*.
However, one method defined in shadowed version
_*org.apache.curator:curator-client:2.12.0*_ is referenced by client project
via *_org.apache.hadoop:hadoop-auth:3.1.1_*, but missing in the actually loaded
version _*org.apache.curator:curator-client:4.0.1*_.
For instance, the following missing method(defined in
_*org.apache.curator:curator-client:2.12.0*_) is actually referenced by
_*atlas/server-api*_, which will introduce a runtime error(i.e.,
"NoSuchMethodError") into _*atlas/server-api*_.
_*Missing method: org.codelibs.core.lang.StringUtil: java.lang.String
newStringUnsafe(char[])*_ is invoked by _*atlas/server-api*_ via the following
path:
{noformat}
paths--
atlas\server-api\target\classes
Repositories\org\apache\curator\curator-client\4.0.1\curator-client-4.0.1.jar
Repositories\org\apache\curator\curator-framework\4.0.1\curator-framework-4.0.1.jar
Repositories\org\apache\curator\curator-framework\4.0.1\curator-framework-4.0.1.jar
{noformat}
h1. Suggested fixing solutions
1. Upgrade dependency *_org.apache.hadoop:hadoop-common_* from _*3.1.1*_ to
*_3.3.0_*. Because the newer version *_org.apache.hadoop:hadoop-common:3.3.0_*
does not invoke the above missing method, such upgrading can solve the problem.
2. Declare a direct dependency _*org.apache.curator:curator-client:2.12.0*_ in
the pom file of _*atlas/server-api*_, to override
_*org.apache.curator:curator-client*_'s managed version.
3. Use configuration attribute to unify the version of
library _*org.apache.curator:curator-client*_ to be _*2.12.0*_ in
_*atlas/server-api*_'s pom file.
Please let me know which solution do you prefer? I can submit a PR to fix it.
Thank you very much for your attention.
Best regards,
h1. Dependency tree
{noformat}
[INFO] org.apache.atlas:atlas-server-api:jar:3.0.0-SNAPSHOT
[INFO] - org.apache.hadoop:hadoop-common:jar:3.1.1:compile
[INFO] +- org.apache.hadoop:hadoop-auth:jar:3.1.1:compile
[INFO] | - org.apache.curator:curator-framework:jar:4.0.1:compile (version
managed from 2.12.0)
[INFO] | - _*(org.apache.curator:curator-client:jar:4.0.1:compile - version
managed from 2.12.0; omitted for duplicate)*_
[INFO] - _*org.apache.curator:curator-client:jar:4.0.1:compile*_{noformat}
was:
Hi, there are multiple versions of _*org.apache.curator:curator-client*_ in
_*atlas/server-api*_. As shown in the following dependency tree, according to
Maven's “nearest wins” strategy, only
_*org.apache.curator:curator-client:4.0.1*_ can be loaded,
_*org.apache.curator:curator-client:2.12.0*_ will be shadowed.
As _*org.apache.curator:curator-client:2.12.0*_ has not been loaded during the
building process, several methods are missing. However, the missing methods:
1. _*org.codelibs.core.lang.StringUtil: java.lang.String
newStringUnsafe(char[])*_
{noformat}
paths--
atlas\server-api\target\classes
Repositories\org\apache\curator\curator-client\4.0.1\curator-client-4.0.1.jar
Repositories\org\apache\curator\curator-framework\4.0.1\curator-framework-4.0.1.jar
Repositories\org\apache\curator\curator-framework\4.0.1\curator-framework-4.0.1.jar
{noformat}
The above missing methods are actually referenced by _*atlas/server-api*_,
which will cause “NoSuchMethodErrors” at runtime.
Suggested fixing solutions:
1. Upgrade dependency *_org.apache.hadoop:hadoop-common_* from _*3.1.1*_ to
*_3.3.0_*. Because one conflicting library version
_*org.apache.curator:curator-client:2.12.0*_ is transitively introduced by
*_org.apache.hadoop:hadoop-common:3.1.1_*. Upgrading dependency
*_org.apache.hadoop:hadoop-common_* from _*3.1.1*_ to *_3.3.0_* can solve this
dependency conflict.
2. Directly declare dependency _*org.apache.curator:curator-client:2.12.0*_.
3. Use configuration attribute to unify the version of
library _*org.apache.curator:curator-client*_ to be _*2.12.0*_ in
_*atlas/server-api*_'s pom file.
Please let me know which solution do you prefer? I can submit a PR to fix it.
Thank you very much for your attention.
Best regards,
Dependency tree
[INFO] org.apache.atlas:atlas-server-api:jar:3.0.0-SNAPSHOT
[INFO] \- org.apache.hadoop:hadoop-common:jar:3.1.1:compile
[INFO] +- org.apache.hadoop:hadoop-auth:jar:3.1.1:compile
[INFO] | \- org.apache.curator:curator-framework:jar:4.0.1:compile (version
managed from 2.12.0)
[INFO] | \- _*(org.apache.curator:curator-client:jar:4.0.1:compile - version
managed from 2.12.0; omitted for
Re: Review Request 72756: ATLAS-3918 : Regression : Renaming table has exceptions in Atlas application logs
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72756/
---
(Updated Sept. 14, 2020, 11:36 a.m.)
Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, Nixon
Rodrigues, and Sarath Subramanian.
Bugs: ATLAS-3918
https://issues.apache.org/jira/browse/ATLAS-3918
Repository: atlas
Description
---
Issue : Messages are being sent for DDL operations one from HMS and HiveServer2
which causes duplicate messages hence it throws exception:
message
{"version":{"version":"1.0.0","versionParts":[1]},"msgCompressionKind":"NONE","msgSplitIdx":1,"msgSplitCount":1,"msgCreationTime":1594802485264,"message":{"type":"ENTITY_PARTIAL_UPDATE_V2","user":"hrt_qa","entityId":{"typeName":"hive_table","uniqueAttributes":{"qualifiedName":"default.t1@cm"}},"entity":{"entity":{"typeName":"hive_table","attributes":{"owner":"hrt_qa","tableType":"MANAGED_TABLE","temporary":false,"lastAccessTime":1594802469000,"aliases":["t1"],"createTime":1594802469000,"qualifiedName":"default.t2@cm","name":"t2","comment":null,"parameters":{"last_modified_time":"1594802481","rawDataSize":"0","numRows":"0","transient_lastDdlTime":"1594802481","bucketing_version":"2","last_modified_by":"hrt_qa","numFilesErasureCoded":"0","totalSize":"0","transactional_properties":"default","COLUMN_STATS_ACCURATE":"{\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"id\":\"true\"}}","numFiles":"0","transactional":"true"},"retention":0},"guid":"-21782793831367970","isIncomplete":false
,"provenanceType":0,"version":0,"proxy":false
This patch fix it by providing a property "DDL_MESSAGES_ENABLE" in
configuration when enabled the HiveServer2 will send ddl messages and if
disabled then it will only send non ddl messages.
Diffs (updated)
-
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRename.java
6961fa7c2
intg/src/main/java/org/apache/atlas/model/notification/HookNotification.java
5b5fa04e2
intg/src/main/java/org/apache/atlas/utils/AtlasJson.java abeddf640
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
f02c05fff
Diff: https://reviews.apache.org/r/72756/diff/4/
Changes: https://reviews.apache.org/r/72756/diff/3-4/
Testing
---
Tested by creating and renaming the hive_table hence ddl messages are not sent
from hiveserver2 after the property is set to false and so no exception is
thrown:
> create table t1(id int);
> alter table t1 rename to t2;
Thanks,
chaitali
[jira] [Commented] (ATLAS-3939) Add build profile for berkeleydb-solr
[ https://issues.apache.org/jira/browse/ATLAS-3939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17195354#comment-17195354 ] chaitali borole commented on ATLAS-3939: +1 for the patch. > Add build profile for berkeleydb-solr > - > > Key: ATLAS-3939 > URL: https://issues.apache.org/jira/browse/ATLAS-3939 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: ATLAS-3939.patch > > > Atlas supports a number of build profiles, like > external-hbase-sol/embedded-hbase-solr/embedded-cassandra-solr/berkeley-elasticsearch, > for various combinations of backend stores an index stores. This should be > extended to support berkeley-solr as well; this profile can be useful for > dev/testing of Atlas. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [atlas] nixonrodrigues commented on pull request #110: ATLAS-3940 : Upgrade snakeyaml to a version without CVE-2017-18640
nixonrodrigues commented on pull request #110: URL: https://github.com/apache/atlas/pull/110#issuecomment-691860982 CI passed. https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/15/console +1 for PR, @crazylab , Thanks for PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (ATLAS-3940) Upgrade snakeyaml to a version without CVE-2017-18640
[ https://issues.apache.org/jira/browse/ATLAS-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nixon Rodrigues updated ATLAS-3940: --- Issue Type: Bug (was: Improvement) > Upgrade snakeyaml to a version without CVE-2017-18640 > -- > > Key: ATLAS-3940 > URL: https://issues.apache.org/jira/browse/ATLAS-3940 > Project: Atlas > Issue Type: Bug >Reporter: Nixon Rodrigues >Priority: Major > > Maven package cassandra-all has transitive dependency on > org.yaml:snakeyaml:1.11 which has > CVE-2017-18640:https://nvd.nist.gov/vuln/detail/CVE-2017-18640 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-3940) Upgrade snakeyaml to a version without CVE-2017-18640
Nixon Rodrigues created ATLAS-3940: -- Summary: Upgrade snakeyaml to a version without CVE-2017-18640 Key: ATLAS-3940 URL: https://issues.apache.org/jira/browse/ATLAS-3940 Project: Atlas Issue Type: Improvement Reporter: Nixon Rodrigues Maven package cassandra-all has transitive dependency on org.yaml:snakeyaml:1.11 which has CVE-2017-18640:https://nvd.nist.gov/vuln/detail/CVE-2017-18640 -- This message was sent by Atlassian Jira (v8.3.4#803005)
