Re: [DRAFT] [REPORT] Apache Atlas - September 2024
Thanks Madhan for the draft report. +1 > On Sep 11, 2024, at 2:51 PM, Madhan Neethiraj wrote: > > Atlas community, > > > > Please review the following draft report for this quarter. > > > > Thanks, > > Madhan > > > > > > > > > > ## Description: > > The mission of Apache Atlas is the creation and maintenance of software > related > > to a scalable and extensible set of core foundational governance services - > > enabling enterprises to effectively and efficiently meet their compliance > > requirements > > > > ## Project Status: > > Current project status: Ongoing, with low activity > > Issues for the board: none > > > > ## Membership Data: > > Apache Atlas was founded 2017-06-20 (7 years ago) > > There are currently 48 committers and 32 PMC members in this project. > > The Committer-to-PMC ratio is 3:2. > > > > Community changes, past quarter: > > - No new PMC members. Last addition was Ashutosh Mestry on 2019-04-16. > > - No new committers. Last addition was Umesh Padashetty on 2023-06-14. > > > > ## Project Activity: > > - fixes and improvements in import functionality > > - performance improvement - utility to analyze hook notifications > > - improvements in notification processing > > - UI fixes in handling of deferred tasks > > - updated versions of dependent libraries > > > > ## Community Health: > > - dev@atlas.apache.org had a 21% decrease in traffic in the past quarter (235 > emails compared to 296): > > - 10 commits in the past quarter, a decrease from 23 commits the previous > quarter > > - 4 code contributors in the past quarter, a decrease from 7 in the previous > quarter > > - 2 new contributors were added in the past quarter - Prashant Satam and > Chandra Kanth Peravelli > > > > ## Recent releases: > > - Apache Atlas 2.3.0 was released on 2022-12-06. > > - Apache Atlas 2.2.0 was released on 2021-08-17. > > - Apache Atlas 2.1.0 was released on 2020-07-15. > > > > >
Re: [DRAFT] [REPORT] Apache Atlas - June 2024
Thanks for the draft report Madhan. Looks good +1 Thanks, Sarath > On Jun 13, 2024, at 5:23 PM, Madhan Neethiraj wrote: > > Atlas community, > > Please review the following draft report for this quarter. > > Thanks, > Madhan > > > ## Description: > The mission of Apache Atlas is the creation and maintenance of software > related > to a scalable and extensible set of core foundational governance services - > enabling enterprises to effectively and efficiently meet their compliance > requirements > > ## Project Status: > Current project status: Ongoing, with low activity > Issues for the board: there are no issues requiring board attention > > ## Membership Data: > Apache Atlas was founded 2017-06-20 (7 years ago) > There are currently 48 committers and 32 PMC members in this project. > The Committer-to-PMC ratio is 3:2. > > Community changes, past quarter: > - No new PMC members. Last addition was Ashutosh Mestry on 2019-04-16. > - No new committers. Last addition was Umesh Padashetty on 2023-06-14. > > ## Project Activity: > - improvements in Export/Import functionality > - updates to support ignore patterns in all Atlas hooks > - added support for Chinese characters in Atlas entity attributes > - Docker setup update to address HBase failures > - updated versions of dependent libraries JanusGraph and Tinkerpop > > ## Community Health: > - dev@atlas.apache.org had a 166% increase in traffic in the past quarter (312 > emails compared to 117) > - 23 commits in the past quarter (64% increase) > - 7 code contributors in the past quarter (40% increase) > - 4 PRs opened on GitHub, past quarter (100% increase) > - 2 PRs closed on GitHub, past quarter (100% increase) > > ## Recent releases: > - Apache Atlas 2.3.0 was released on 2022-12-06 > - Apache Atlas 2.2.0 was released on 2021-08-17 > - Apache Atlas 2.1.0 was released on 2020-07-15 > > > >
Re: [DRAFT] [REPORT] Apache Atlas - March 2024
Thanks for the draft board report Madhan. Looks good. +1 > On Mar 14, 2024, at 2:27 AM, Nikhil Bonte wrote: > > Looks Good to me > Thanks Madhan for putting this together > > > Regards > Nikhil P. Bonte > >> On Wed, 13 Mar 2024 at 1:17 PM, Madhan Neethiraj wrote: >> >> Atlas community, >> >> Please review the following draft report for this quarter. >> >> Thanks, >> Madhan >> >> >> ## Description: >> The mission of Apache Atlas is the creation and maintenance of software >> related >> to a scalable and extensible set of core foundational governance services >> - >> enabling enterprises to effectively and efficiently meet their compliance >> requirements >> >> ## Project Status: >> Current project status: Ongoing, with low activity >> Issues for the board: here are no issues requiring board attention. >> >> ## Membership Data: >> Apache Atlas was founded 2017-06-20 (7 years ago) >> There are currently 48 committers and 32 PMC members in this project. >> The Committer-to-PMC ratio is 3:2. >> >> Community changes, past quarter: >> - No new PMC members. Last addition was Ashutosh Mestry on 2019-04-16. >> - No new committers. Last addition was Umesh Padashetty on 2023-06-14. >> >> ## Project Activity: >> - fixes in download search results functionality >> - UI updates to improve usability of tag-propagation enable/disable >> - fix in business metadata changes to update system attributes >> - updated versions of Spring Security, netty >> >> ## Community Health: >> - dev@atlas.apache.org had a 40% decrease in traffic in the past quarter >> (143 >> emails compared to 238) >> - u...@atlas.apache.org had a 100% increase in traffic in the past >> quarter (8 >> emails compared to 4) >> - 10 commits in the past quarter (-73% change) >> - 6 code contributors in the past quarter (-50% change) >> >> ## Recent releases: >> - Apache Atlas 2.3.0 was released on 2022-12-06 >> - Apache Atlas 2.2.0 was released on 2021-08-17 >> - Apache Atlas 2.1.0 was released on 2020-07-15 >> >> >> >> >>
Re: [DRAFT] [REPORT] Apache Atlas - December 2023
+1. Thanks for the draft Madhan Thanks, Sarath On Wed, Dec 13, 2023 at 9:25 AM Madhan Neethiraj wrote: > Atlas community, > > > > Please review the following draft report for this quarter. > > > > Thanks, > > Madhan > > > > ## Description: > > The mission of Apache Atlas is the creation and maintenance of software > related > > to a scalable and extensible set of core foundational governance services > - > > enabling enterprises to effectively and efficiently meet their compliance > > requirements > > > > ## Project Status: > > Current project status: Ongoing, with moderate activity > > Issues for the board: There are no issues requiring board attention. > > > > ## Membership Data: > > Apache Atlas was founded 2017-06-20 (6 years ago) > > There are currently 48 committers and 32 PMC members in this project. > > The Committer-to-PMC ratio is 3:2. > > > > Community changes, past quarter: > > - No new PMC members. Last addition was Ashutosh Mestry on 2019-04-16. > > - No new committers. Last addition was Umesh Padashetty on 2023-06-14. > > > > ## Project Activity: > > - enhancements to support aging of entity audits > > - added bridge for Couchbase > > - performance improvement in notification processing with optimization in > > fetching relationships > > - removed references to Python2 with Python3 > > - improvements in dynamic index recovery > > - fixes in Impala hook > > - published Atlas client Python library 0.0.15 > > - updated versions of Okio, Apache Ivy > > > > ## Community Health: > > - dev@atlas.apache.org had a 41% increase in traffic in the past quarter > (247 > > emails compared to 174) > > - 17 issues opened in JIRA, past quarter (-48% change) > > - 14 issues closed in JIRA, past quarter (-30% change) > > - 33 commits in the past quarter (3% increase) > > - 11 code contributors in the past quarter (22% increase) > > - 2 PRs opened on GitHub, past quarter (-71% change) > > - 4 PRs closed on GitHub, past quarter (no change) > > > > ## Recent releases: > > Apache Atlas 2.3.0 was released on 2022-12-06. > > Apache Atlas 2.2.0 was released on 2021-08-17. > > Apache Atlas 2.1.0 was released on 2020-07-15. > > > > > >
Re: Review Request 74587: ATLAS-4795: docker setup updated to support Apple M1
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74587/#review225735 --- Ship it! Ship It! - Sarath Subramanian On Sept. 8, 2023, 5:47 p.m., Madhan Neethiraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74587/ > --- > > (Updated Sept. 8, 2023, 5:47 p.m.) > > > Review request for atlas, Ashutosh Mestry, Nikhil Bonte, Nixon Rodrigues, > Radhika Kundam, Sarath Subramanian, Sharmadha Sainath, and Sidharth Mishra. > > > Bugs: ATLAS-4795 > https://issues.apache.org/jira/browse/ATLAS-4795 > > > Repository: atlas > > > Description > --- > > - updated docker setup scripts to use TARGETARCH > - added support for building and runing with JDK8 or JDK11 > > > Diffs > - > > dev-support/atlas-docker/.env 440be2a46 > dev-support/atlas-docker/Dockerfile 7bd79651f > dev-support/atlas-docker/Dockerfile.atlas 888779133 > dev-support/atlas-docker/Dockerfile.atlas-base e68daa69d > dev-support/atlas-docker/Dockerfile.atlas-build 4b381c1d9 > dev-support/atlas-docker/docker-compose.atlas-base.yml 349c306d6 > dev-support/atlas-docker/docker-compose.atlas-build.yml d2ec7bda5 > dev-support/atlas-docker/docker-compose.atlas.yml aeb706e59 > dev-support/atlas-docker/scripts/atlas-build.sh 592a055f7 > > > Diff: https://reviews.apache.org/r/74587/diff/1/ > > > Testing > --- > > - verified that Atlas builds and runs successfully via docker setup scripts > in MacBook Pro with Apple M1 > > > Thanks, > > Madhan Neethiraj > >
Re: Review Request 74586: ATLAS-4794: fixed initialization failure in HBase, Hive hooks
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74586/#review225733 --- Ship it! Ship It! - Sarath Subramanian On Sept. 8, 2023, 2:16 p.m., Madhan Neethiraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74586/ > --- > > (Updated Sept. 8, 2023, 2:16 p.m.) > > > Review request for atlas, Ashutosh Mestry, Nikhil Bonte, Nixon Rodrigues, > Radhika Kundam, Sarath Subramanian, Sharmadha Sainath, Sidharth Mishra, and > Umesh Padashetty. > > > Bugs: ATLAS-4794 > https://issues.apache.org/jira/browse/ATLAS-4794 > > > Repository: atlas > > > Description > --- > > - updated HBase hook packaging to include log4j-api and log4j-core libraries > - updated Hive hook packaging to include log4j-api and log4j-core libraries > > > Diffs > - > > addons/hbase-bridge/pom.xml d78abbb37 > addons/hive-bridge/pom.xml 3464b65f0 > > > Diff: https://reviews.apache.org/r/74586/diff/2/ > > > Testing > --- > > - verified that HBase and Hive hooks initialize and work successfully with > the fix > > > Thanks, > > Madhan Neethiraj > >
[jira] [Commented] (ATLAS-4749) What is the meaning of version in the JSON content of creating entities, and how to update version when updating entity content
[
https://issues.apache.org/jira/browse/ATLAS-4749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17720007#comment-17720007
]
Sarath Subramanian commented on ATLAS-4749:
---
[~zhangkai] version is modeled as an internal field in atlas and not meant to
be an updated field. We suggest you add a new attribute to capture version.
> What is the meaning of version in the JSON content of creating entities, and
> how to update version when updating entity content
> ---
>
> Key: ATLAS-4749
> URL: https://issues.apache.org/jira/browse/ATLAS-4749
> Project: Atlas
> Issue Type: Task
>Affects Versions: 2.3.0
>Reporter: zhangkai
>Priority: Major
>
> Example:
> 1. Create Entity ("version"=20)
> 2. Query created entities
> 3. Update Entity ("version"=30)
> 4. Query the updated entity and find that version=20 has not changed
> Create an entity's JSON
> {
> "entity": {
> "typeName": "my_db_new",
> "attributes": {
> "description": "database",
> "clustername": "T2@TEST",
> "name": "all_dbs",
> "qualifiedName": "T2@TEST",
> "attr2": "test"
> },
> "version": 20
> }
> }
> Update entity's JSON
> {
> "entity": {
> "typeName": "my_db_new",
> "attributes": {
> "description": "database",
> "clustername": "T2@TEST",
> "name": "all_dbs_update",
> "qualifiedName": "T2@TEST",
> "attr2": "test"
> },
> "version": 30
> }
> }
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ATLAS-4748) 创建实体的JSON内容中版本是什么含义，更新实体内容时如何更新版本
[
https://issues.apache.org/jira/browse/ATLAS-4748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian resolved ATLAS-4748.
---
Resolution: Incomplete
> 创建实体的JSON内容中版本是什么含义，更新实体内容时如何更新版本
> -
>
> Key: ATLAS-4748
> URL: https://issues.apache.org/jira/browse/ATLAS-4748
> Project: Atlas
> Issue Type: Task
> Components: atlas-core
>Affects Versions: 2.3.0
>Reporter: zhangkai
>Priority: Major
>
> 创建实体的JSON内容中版本是什么含义，更新实体内容时如何更新版本
>
> 例：
> 1. 创建实体（“版本”=20）
> 2. 查询创建的实体
> 3. 更新实体（“版本”=30）
> 4. 查询更新的实体，发现 version=20 未更改
>
> 创建实体的 JSON
> {
> “实体”：{
> “类型名称”： “my_db_new”，
> “属性”：{
> “描述”： “数据库”，
> “集群名称”： “T2@TEST”，
> “名称”： “all_dbs”，
> “限定名称”： “T2@TEST”，
> “attr2”： “测试”
> },
> “版本”：20
> }
> }
>
>
> 更新实体的 JSON
> {
> “实体”：{
> “类型名称”： “my_db_new”，
> “属性”：{
> “描述”： “数据库”，
> “集群名称”： “T2@TEST”，
> “名称”： “all_dbs_update”，
> “限定名称”： “T2@TEST”，
> “attr2”： “测试”
> },
> “版本”：30
> }
> }
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: Review Request 74375: ATLAS-4740: enable building Atlas using JDK versions higher than 8
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74375/#review225352 --- Ship it! Ship It! - Sarath Subramanian On March 30, 2023, 3:06 p.m., Madhan Neethiraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74375/ > --- > > (Updated March 30, 2023, 3:06 p.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, Pinal Shah, Sarath > Subramanian, and Sidharth Mishra. > > > Bugs: ATLAS-4740 > https://issues.apache.org/jira/browse/ATLAS-4740 > > > Repository: atlas > > > Description > --- > > added build properties to generate binaries for 1.8 > > > Diffs > - > > pom.xml 15aa57bc4 > > > Diff: https://reviews.apache.org/r/74375/diff/1/ > > > Testing > --- > > pre-commit build and test run: > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1335/ > > > Thanks, > > Madhan Neethiraj > >
Re: [DRAFT] [REPORT] Apache Atlas - March 2023
Thank you for drafting the board report Madhan. +1 Thanks, Sarath On Sun, Mar 12, 2023 at 7:16 PM Madhan Neethiraj wrote: > Atlas community, > > > > Please review the following draft report for this quarter. > > > > Thanks, > > Madhan > > > > > > ## Description: > > The mission of Apache Atlas is the creation and maintenance of software > related > > to a scalable and extensible set of core foundational governance services > - > > enabling enterprises to effectively and efficiently meet their compliance > > requirements > > > > ## Issues: > > There are no issues requiring board attention. > > > > ## Membership Data: > > Apache Atlas was founded 2017-06-20 (6 years ago) > > There are currently 47 committers and 32 PMC members in this project. > > The Committer-to-PMC ratio is roughly 3:2. > > > > Community changes, past quarter: > > - No new PMC members. Last addition was Ashutosh Mestry on 2019-04-16. > > - No new committers. Last addition was Mandar Ambawane on 2022-09-12. > > > > ## Project Activity: > > - released Python client 0.0.13 > > - updated versions of Spring-security, Netty, Swagger, Jettison, Snakeyaml, > > Tinkerpop > > - UI fixes in handling of deleted entities > > - fix to address failure in admin audit APIs due to missed index creation > > > > ## Community Health: > > - dev@atlas.apache.org had a 22% decrease in traffic in the past quarter > > (680 emails compared to 868) > > > > ## Most Recent releases: > > 2.3.0 was released on 2022-12-06 > > 2.2.0 was released on 2021-08-17 > > 2.1.0 was released on 2020-07-15 > > 0.8.4 was released on 2019-06-21 > >
Re: [VOTE] Release Apache Atlas 2.3.0 rc1
Sid, I followed the below steps to verify signature. I'm running on Mac OS Monterey. wget https://dist.apache.org/repos/dist/dev/atlas/KEYS > gpg --import KEYS > gpg --verify apache-atlas-2.3.0-sources.tar.gz.asc > apache-atlas-2.3.0-sources.tar.gz > Thanks, Sarath On Wed, Nov 30, 2022 at 11:50 PM Sidharth Mishra wrote: > Thank you Madhan for the release candidate. > > Validated the following and faced some issues as mentioned below: > > - Able to download source, signature, md5 and sha512 files and verify > the checksum hash > - I am not able to verify the signature of the source. Steps I followed: > curl -O https://downloads.apache.org/atlas/KEYS > gpg --import KEYS > gpg --list-sigs madhan # Gives the correct output > pub rsa4096 2020-06-10 [SC] [expires: 2024-06-09] > 1B6007E9CDEC4913DFB5031B630E02BA8823016D > uid [ unknown] Madhan Neethiraj < > mad...@apache.org> > sig 3630E02BA8823016D 2020-06-10 > Madhan Neethiraj >sub rsa4096 2020-06-10 [E] [expires: 2024-06-09] >sig 630E02BA8823016D 2020-06-10 Madhan > Neethiraj > >pub rsa2048 2014-11-10 [SC] [expired: 2018-11-10] > 4BDEE6708B5F5216CF0BA42754EA1B4FFA03B08A >uid [ expired] Madhan Neethiraj < > mad...@apache.org> >sig 354EA1B4FFA03B08A 2014-11-10 Madhan > Neethiraj >sig 9C0596B11E19B762 2014-11-10 [User ID not > found] >sig 2C1CD6311ED05C4A 2016-02-16 [User ID not > found] >sig 3 X EB4200BBD4393DE8 2016-04-30 [User ID not > found] > > gpg --verify apache-atlas-2.3.0-sources.tar.gz.asc > apache-atlas-2.3.0-sources.tar.gz > gpg: Signature made Mon Nov 28 13:14:22 2022 PST > gpg:using RSA key FED467D3B01179D4 > gpg: Can't check signature: No public key > ># Even tried this and no help as it didn't download any keys >gpg --keyserver > https://dist.apache.org/repos/dist/release/atlas/KEYS --recv-keys > FED467D3B01179D4 > > - Build failed for the source using embedded-hbase-solr profile - > > [ERROR] Failed to execute goal on project atlas-testtools: Could not > resolve dependencies for project > org.apache.atlas:atlas-testtools:jar:2.3.0: Failed to collect > dependencies at org.apache.solr:solr-test-framework:jar:8.6.3 -> > org.restlet.jee:org.restlet:jar:2.4.3: Failed to read artifact > descriptor for org.restlet.jee:org.restlet:jar:2.4.3: Could not > transfer artifact org.restlet.jee:org.restlet:pom:2.4.3 from/to > maven-restlet (https://maven.restlet.com): transfer failed for > > https://maven.restlet.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-2.4.3.pom > : > sun.security.validator.ValidatorException: PKIX path validation > failed: java.security.cert.CertPathValidatorException: validity check > failed: NotAfter: Sun Nov 13 09:05:56 PST 2022 -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with > the -e switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the errors and possible > solutions, please read the following articles: > [ERROR] [Help 1] > > http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException > [ERROR] > [ERROR] After correcting the problems, you can resume the build with > the command > [ERROR] mvn -rf :atlas-testtools > > By running the below command the above issue was resolved as it > ignored all ssl verifications (Similar issue - > > https://stackoverflow.com/questions/68199459/maven-build-failed-pkix-path-validation-failed-java-security-cert-certpathval > ): > ln -s /usr/local/bin/python3 /usr/local/bin/python # As by > default python3 gets installed when we upgrade to mac ventura and no > python symlink > mvn clean install > -Dmaven.wagon.http.ssl.ignore.validity.dates=true > -Dmaven.wagon.http.ssl.insecure=true > -Dmaven.wagon.http.ssl.allowall=true > mvn clean package -Pdist > -Dmaven.wagon.http.ssl.ignore.validity.dates=true > -Dmaven.wagon.http.ssl.insecure=true > -Dmaven.wagon.http.ssl.allowall=true > > - started Atlas and ran quickstart script to preload sample types and > entities > - validated relation search, basic and advanced search > > P.S. Recently I upgraded my mac to ventura and I'm not sure if these > issues are due to the upgrade. If someone else has a similar mac > version and it works for the
Re: [VOTE] Release Apache Atlas 2.3.0 rc1
Thank you Madhan for the release candidate. +1 for Apache Atlas 2.3.0 release candidate #1 validated the following: - Able to download source, signature, md5 and sha512 files and verified checksum hash - validated signature of source from release manager (Madhan Neethiraj < mad...@apache.org>) - Build the source successfully using embedded-hbase-solr profile - started Atlas and ran quickstart script to preload sample types and entities - validated relation search, basic and advanced search Thanks, Sarath On Mon, Nov 28, 2022 at 1:50 PM Madhan Neethiraj wrote: > Atlas team, > > > > Apache Atlas 2.3.0 release candidate #1 is now available for a vote within > dev community. Links to the release artifacts are given below. Please > review and vote. > > > > The vote will be open for at least 72 hours or until necessary votes are > reached. > > [ ] +1 approve > > [ ] +0 no opinion > > [ ] -1 disapprove (and reason why) > > > > Thanks, > > Madhan > > > > > > List of issues addressed in this release: > https://issues.apache.org/jira/issues/?jql=project%20%3D%20Atlas%20AND%20fixVersion%20%3D%202.3.0%20ORDER%20BY%20key%20DESC > > > > Git tag for the release: > https://github.com/apache/atlas/tree/release-2.3.0-rc1 > > Sources for the release: > https://dist.apache.org/repos/dist/dev/atlas/2.3.0-rc1/apache-atlas-2.3.0-sources.tar.gz > > > > Source release verification: > > PGP Signature: > https://dist.apache.org/repos/dist/dev/atlas/2.3.0-rc1/apache-atlas-2.3.0-sources.tar.gz.asc > > MD5 Hash: > https://dist.apache.org/repos/dist/dev/atlas/2.3.0-rc1/apache-atlas-2.3.0-sources.tar.gz.md5 > > SHA512 Hash: > https://dist.apache.org/repos/dist/dev/atlas/2.3.0-rc1/apache-atlas-2.3.0-sources.tar.gz.sha512 > > > > Keys to verify the signature of the release artifacts are available at: > https://dist.apache.org/repos/dist/dev/atlas/KEYS > >
Re: Review Request 74202: ATLAS-4713: docker setup updates to be in sync with component versions in project pom.xml
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74202/#review224881 --- Ship it! Ship It! - Sarath Subramanian On Nov. 15, 2022, 5:41 p.m., Madhan Neethiraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74202/ > --- > > (Updated Nov. 15, 2022, 5:41 p.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, Mandar Ambawane, > Nikhil Bonte, Nixon Rodrigues, Pinal Shah, Radhika Kundam, Sarath > Subramanian, and Sidharth Mishra. > > > Bugs: ATLAS-4713 > https://issues.apache.org/jira/browse/ATLAS-4713 > > > Repository: atlas > > > Description > --- > > - updated version of Kafka, Hive-hadoop > - fixed incorrect Atlas URL in hook configuration setup > > > Diffs > - > > dev-support/atlas-docker/.env 19175c2ed > dev-support/atlas-docker/scripts/atlas-hbase-application.properties > 8c5b9e088 > dev-support/atlas-docker/scripts/atlas-hive-application.properties > 8c5b9e088 > > > Diff: https://reviews.apache.org/r/74202/diff/1/ > > > Testing > --- > > - verified that Atlas docker setup builds successfully > - performed sanity testing of Atlas, Hive hook, HBase hook > > > Thanks, > > Madhan Neethiraj > >
Re: Review Request 74204: ATLAS-3820: fix import-kafka.sh failures
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74204/#review224880 --- Ship it! Ship It! - Sarath Subramanian On Nov. 15, 2022, 10:06 p.m., Madhan Neethiraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74204/ > --- > > (Updated Nov. 15, 2022, 10:06 p.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nikhil Bonte, > Nixon Rodrigues, Pinal Shah, Radhika Kundam, Ramesh Mani, Sarath Subramanian, > Sharmadha Sainath, Sidharth Mishra, and Umesh Padashetty. > > > Bugs: ATLAS-3820 > https://issues.apache.org/jira/browse/ATLAS-3820 > > > Repository: atlas > > > Description > --- > > - added missing libraries to Kafka bridge package > - updated docker setup to include Kafka bridge > > > Diffs > - > > addons/kafka-bridge/pom.xml 9ce22a757 > addons/kafka-bridge/src/bin/import-kafka.sh ada3b4ae2 > > addons/kafka-bridge/src/main/java/org/apache/atlas/kafka/bridge/KafkaBridge.java > 8be2fca3a > dev-support/atlas-docker/.dockerignore d4a3db26b > dev-support/atlas-docker/Dockerfile.atlas-kafka e06d62845 > dev-support/atlas-docker/scripts/atlas-build.sh e53ebbba9 > dev-support/atlas-docker/scripts/atlas-kafka-application.properties > PRE-CREATION > > > Diff: https://reviews.apache.org/r/74204/diff/1/ > > > Testing > --- > > - verified that import-kafka.sh sucessfully added Kafka topics to Atlas > > > Thanks, > > Madhan Neethiraj > >
Re: Review Request 74203: ATLAS-4241: fix import-hbase errors
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74203/#review224879 --- Ship it! Ship It! - Sarath Subramanian On Nov. 15, 2022, 9:53 p.m., Madhan Neethiraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74203/ > --- > > (Updated Nov. 15, 2022, 9:53 p.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nikhil Bonte, > Nixon Rodrigues, Pinal Shah, Radhika Kundam, Ramesh Mani, Sarath Subramanian, > and Sidharth Mishra. > > > Bugs: ATLAS-4241 > https://issues.apache.org/jira/browse/ATLAS-4241 > > > Repository: atlas > > > Description > --- > > - fixed classpath for import-hbase to exclude couple of libraries present in > HBase lib directory > - added following libraries in HBase plugin impl: commons-collections, > commons-logging, jackson-jaxrs-json-provider, jackson-jaxrs-base, > jersey-bundle > > > Diffs > - > > addons/hbase-bridge/pom.xml f51a7d6f1 > addons/hbase-bridge/src/bin/import-hbase.sh a343036fa > > > Diff: https://reviews.apache.org/r/74203/diff/1/ > > > Testing > --- > > verified the following using Docker setup > - HBase hook successfully reports changes to Atlas server > - import-hbase utility successfully imports namespaces and tables to Atlas > server > > > Thanks, > > Madhan Neethiraj > >
Re: Review Request 74201: ATLAS-4445: fix import-hive.sh failure due to NoClassDefFoundError: javax/ws/rs/core/Link$Builder
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74201/#review224878 --- Ship it! Ship It! - Sarath Subramanian On Nov. 15, 2022, 12:08 p.m., Madhan Neethiraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74201/ > --- > > (Updated Nov. 15, 2022, 12:08 p.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, keval bhatt, > Nikhil Bonte, Nixon Rodrigues, Pinal Shah, Radhika Kundam, Ramesh Mani, > Sarath Subramanian, and Sidharth Mishra. > > > Bugs: ATLAS-4445 > https://issues.apache.org/jira/browse/ATLAS-4445 > > > Repository: atlas > > > Description > --- > > - updated hive version from 3.1.0 to 3.1.2 > - removed packaging jsr311-api in Hive hook > > > Diffs > - > > addons/hive-bridge/pom.xml 856e5d95d > addons/hive-bridge/src/bin/import-hive.sh ebe6976f0 > pom.xml ca221fe29 > > > Diff: https://reviews.apache.org/r/74201/diff/1/ > > > Testing > --- > > - verified import-hive.sh in docker setup > - all existing tests pass successfully > - pre-commit tests run: > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1288/ > > > Thanks, > > Madhan Neethiraj > >
Re: Apache Atlas 2.3.0 release
Lots of great features and improvements! +1 for Atlas release 2.3.0 Thanks, Sarath On Wed, Oct 26, 2022 at 7:22 PM Madhan Neethiraj wrote: > Atlas community, > > > > It is more than a year since Apache Atlas 2.2.0 was released on Aug-2021. > During this time, the community has been busy in enhancing Apache Atlas > with several improvements and fixes. Here are few key items: > >- Search: introduced relationship search, added sorting capability to >quick search >- Resiliency: improvements in handling of index update failures >- Glossary: performance improvements in dealing with glossaries having >large number of terms and categories >- Glossary: performance improvements in bulk creation of glossary terms >- Observability: UI improvements to view metrics & stats, persistence >of metrics to enable historic data >- Dependent component version upgrades: Elasticsearch, Kafka, Storm, >JanusGraph, TinkerPop, Gson, Spring Framework >- UI: fixes and improvements in multiple areas like search, lineage, >business metadata >- Hive hook: multiple fixes/improvements >- Import Hive utility: multiple fixes/improvements >- Impala hook: performance improvements >- Log4j vulnerabilities: updated to version 2.17.2 >- Audits: support for authentication enabled Cassandra >- Performance: improved Atlas lineage API and UI performance >- enhanced notification messages to include hook version and message >source >- refactored Atlas server to removed Kafka server dependency >- addressed issues in applying type patches in HA environments >- ability to run Atlas and hooks in docker containers > > > > With these improvements in place, it is time now for the next maintenance > release of Apache Atlas! > > > > I propose to release Apache Atlas 2.3.0 by mid next month. Please review > and send your comments. > > > > Thanks, > > Madhan > > > > > > >
Re: Review Request 74157: ATLAS-4684: UI dependent components version update
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74157/#review224769 --- Ship it! Ship It! - Sarath Subramanian On Oct. 9, 2022, 2:32 a.m., Madhan Neethiraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74157/ > --- > > (Updated Oct. 9, 2022, 2:32 a.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, keval bhatt, > Mandar Ambawane, Nikhil Bonte, Nixon Rodrigues, Pinal Shah, Prasad Pawar, > Radhika Kundam, Sarath Subramanian, Sharmadha Sainath, and Sidharth Mishra. > > > Bugs: ATLAS-4684 > https://issues.apache.org/jira/browse/ATLAS-4684 > > > Repository: atlas > > > Description > --- > > This tracks following pull requests containing updates to UI component > versions: > > - https://github.com/apache/atlas/pull/162 > - https://github.com/apache/atlas/pull/164 > - https://github.com/apache/atlas/pull/165 > - https://github.com/apache/atlas/pull/166 > - https://github.com/apache/atlas/pull/167 > - https://github.com/apache/atlas/pull/168 > - https://github.com/apache/atlas/pull/179 > - https://github.com/apache/atlas/pull/180 > - https://github.com/apache/atlas/pull/186 > - https://github.com/apache/atlas/pull/188 > - https://github.com/apache/atlas/pull/189 > - https://github.com/apache/atlas/pull/190 > - https://github.com/apache/atlas/pull/191 > - https://github.com/apache/atlas/pull/194 > - https://github.com/apache/atlas/pull/195 > > > Diffs > - > > dashboardv2/package-lock.json e1d672d822860a6a7d9bda170a6f762c66dc0f91 > dashboardv2/package.json 30837fe5ebc2244b223654fe7d3281e95f2af4fa > dashboardv2/public/js/external_lib/atlas-lineage/package-lock.json > 69f5a4006918a8c29e7f1c66a2707fac9d79be49 > dashboardv3/package-lock.json 1ebd66fe9546faccb7771e94e01283136c882b16 > dashboardv3/package.json cc15a93348c28c8c9a5a5e7bb0e19e59852be4a0 > dashboardv3/public/js/external_lib/atlas-lineage/package-lock.json > 69f5a4006918a8c29e7f1c66a2707fac9d79be49 > > > Diff: https://reviews.apache.org/r/74157/diff/2/ > > > Testing > --- > > - pre-commit tests run: > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1246/ > > > Thanks, > > Madhan Neethiraj > >
Re: [DRAFT] [REPORT] Apache Atlas - September 2022
Thanks for putting this together Madhan. +1 Thanks, Sarath On Tue, Sep 13, 2022 at 8:15 PM Madhan Neethiraj wrote: > Atlas community, > > > > Please review the following draft report for this quarter. > > > > Thanks, > > Madhan > > > > ## Description: > > The mission of Apache Atlas is the creation and maintenance of software > related > > to a scalable and extensible set of core foundational governance services - > > enabling enterprises to effectively and efficiently meet their compliance > > requirements > > > > ## Issues: > > There are no issues requiring board attention. > > > > ## Membership Data: > > Apache Atlas was founded 2017-06-20 (5 years ago) There are currently 47 > > committers and 32 PMC members in this project. The Committer-to-PMC ratio > is > > roughly 3:2. > > > > Community changes, past quarter: > > - No new PMC members. Last addition was Ashutosh Mestry on 2019-04-16. > > - Mandar Ambawane was added as committer on 2022-09-12 > > - Prasad Pawar was added as committer on 2022-09-12 > > - 4 new Atlas contributors in last quarter: > > - Abhishek Pal, Kundan Kumar Jha, Patrik Marton, Beakal Lemeneh > > > > ## Project Activity: > > - fixes in search API handling of pagination > > - updates to support authenticated Cassandra cluster as audit repository > > - performance improvements in Atlas lineage API and UI > > - removed Kafka core dependency in Atlas webapp > > - improvements in import-hive utility to delete entities for deleted > objects > > in Hive > > - multiple UI fixes and improvements > > - dependent component version updates: Spring Framework 5.3.21, Spring > > Security 5.7.2, Netty Project 4.1.78.Final, POI 5.2.2, JUnit 4.13.2 > > - started a thread on Atlas 2.4.0 release. Looking to complete release by > > October 2022 > > > > > > ## Community Health: > > - dev@atlas.apache.org had a 22% decrease in traffic in the past quarter > (680 > > emails compared to 868) > > - u...@atlas.apache.org had a 37% decrease in traffic in the past quarter > (12 > > emails compared to 19) > > - 48 issues opened in JIRA, past quarter (-12% change) > > - 29 issues closed in JIRA, past quarter (38% increase) > > - 51 commits in the past quarter (-15% change) > > - 13 code contributors in the past quarter (8% increase) > > - 16 PRs opened on GitHub, past quarter (-40% change) > > - 7 PRs closed on GitHub, past quarter (no change) > > > > ## Most Recent releases: > > 2.2.0 was released on 2021-08-17 > > 2.1.0 was released on 2020-07-15 > > 0.8.4 was released on 2019-06-21 > >
Re: [DRAFT] [REPORT] Apache Atlas - July 2022
Thank you Madhan for putting this together. Looks good. +1 Thanks, Sarath On Sun, Jul 10, 2022 at 10:46 PM Madhan Neethiraj wrote: > Atlas community, > > Please review the following draft report for this quarter. > > Thanks, > Madhan > > ## Description: > The mission of Apache Atlas is the creation and maintenance of software > related > to a scalable and extensible set of core foundational governance services > - > enabling enterprises to effectively and efficiently meet their compliance > requirements > > ## Issues: > There are not issues requiring board attention. > > ## Membership Data: > Apache Atlas was founded 2017-06-20 (5 years ago) > There are currently 45 committers and 32 PMC members in this project. > The Committer-to-PMC ratio is roughly 3:2. > > Community changes, past quarter: > - No new PMC members. Last addition was Ashutosh Mestry on 2019-04-16. > - No new committers. Last addition was Radhika Kundam on 2022-02-04. > > ## Project Activity: > - resolved failures with Atlas HA with index-recovery enabled > - notification message processing improvements to enable server and hooks > to > be of different versions > - Kafka importer utility improvements > - Impala hook fix to handle large query string > - Glossary module performance improvements to handle glossaries with large > number of categories and terms > - import zip functionality improvements > - several UI enhancements/fixes > - dependent component version updates: Spring 5.3.20, jQuery-UI 1.13.0, > netty > 4.1.77, Spring security 5.7.1, gson 2.9.0 > > ## Community Health: > - dev@atlas.apache.org had a 22% decrease in traffic in the past quarter > (680 > emails compared to 868) > - u...@atlas.apache.org had a 37% decrease in traffic in the past quarter > (12 > emails compared to 19) > - 48 issues opened in JIRA, past quarter (-12% change) > - 24 issues closed in JIRA, past quarter (-20% change) > - 48 commits in the past quarter (-27% change) > - 11 code contributors in the past quarter (-15% change) > - 24 PRs opened on GitHub, past quarter (20% increase) > - 11 PRs closed on GitHub, past quarter (450% increase) > > ## Most Recent releases: > 2.2.0 was released on 2021-08-17 > 2.1.0 was released on 2020-07-15 > 0.8.4 was released on 2019-06-21 > > > >
Re: [DRAFT] [REPORT] Apache Atlas - March 2022
Thanks Madhan for putting the draft report. +1 Thanks, Sarath On Tue, Mar 8, 2022 at 2:38 PM Madhan Neethiraj wrote: > Atlas community, > > > > Please review the following draft report for this quarter. > > > > Thanks, > > Madhan > > > > > > > > ## Description: > > The mission of Apache Atlas is the creation and maintenance of software > > related to a scalable and extensible set of core foundational governance > > services - enabling enterprises to effectively and efficiently meet their > > compliance requirements > > > > ## Issues: > > There are no issues requiring board attention. > > > > ## Membership Data: > > Apache Atlas was founded 2017-06-20 (5 years ago). There are currently 45 > > committers and 32 PMC members in this project. The Committer-to-PMC ratio > is > > roughly 3:2. > > > > Community changes, past quarter: > > - No new PMC members. Last addition was Ashutosh Mestry on 2019-04-16. > > - Radhika Kundam was added as committer on 2022-02-04 > > > > ## Project Activity: > > - added build profile for Cassandra and Elastic Search stores > > - UI fixes to handle glossary with large number of terms > > - multiple fixes in search feature and UI > > - updated docker setup to use network name > > - updated monitoring to support Elastic Search index backend > > - updated Hive hook to support Ozone file scheme for external tables > > - fixed failure in Atlas startup in HA mode with index recovery enabled > > - dependent component version updates: JanusGraph 0.6.1, TinkerPop 3.5.2, > > Spring framework 5.3.15, log4j 2.17.1, testng 7.0, underscore.js 1.12.1 > > > > ## Community Health: > > - dev@atlas.apache.org had a 22% decrease in traffic in the past quarter > (680 > > emails compared to 868) > > - u...@atlas.apache.org had a 37% decrease in traffic in the past quarter > (12 > > emails compared to 19) > > - 61 issues opened in JIRA, past quarter (-27% change) > > - 29 issues closed in JIRA, past quarter (-51% change) > > - 56 commits in the past quarter (-53% change) > > - 13 code contributors in the past quarter (-27% change) > > - 9 PRs opened on GitHub, past quarter (12% increase) > > - 2 PRs closed on GitHub, past quarter (-50% change) > > > > ## Most Recent releases: > > 2.2.0 was released on 2021-08-17 > > 2.1.0 was released on 2020-07-15 > > 0.8.4 was released on 2019-06-21 > > > > > >
Re: Review Request 73826: ATLAS-4548: Set Empty attribute value for queryStr and queryId in case of null value when atlas.hook.hive.hive_process.populate.deprecated.attributes=true
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73826/#review224016 --- Ship it! Ship It! - Sarath Subramanian On Feb. 2, 2022, 11:08 a.m., Sidharth Mishra wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73826/ > --- > > (Updated Feb. 2, 2022, 11:08 a.m.) > > > Review request for atlas, Disha Talreja, Jayendra Parab, Pinal Shah, Radhika > Kundam, and Sarath Subramanian. > > > Bugs: ATLAS-4548 > https://issues.apache.org/jira/browse/ATLAS-4548 > > > Repository: atlas > > > Description > --- > > ATLAS-4548: Added Empty attribute value for queryStr and queryId in case of > null value > > > Diffs > - > > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java > 3e966ae66 > > > Diff: https://reviews.apache.org/r/73826/diff/1/ > > > Testing > --- > > Manually tested and PreCommit - > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1034/ > > > Thanks, > > Sidharth Mishra > >
[jira] [Commented] (ATLAS-4540) Upgrade JanusGraph to version 0.6.1
[ https://issues.apache.org/jira/browse/ATLAS-4540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482043#comment-17482043 ] Sarath Subramanian commented on ATLAS-4540: --- Thanks for the patch [~madhan]. +1 > Upgrade JanusGraph to version 0.6.1 > --- > > Key: ATLAS-4540 > URL: https://issues.apache.org/jira/browse/ATLAS-4540 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj >Priority: Major > Attachments: ATLAS-4540.patch > > > Upgrade Atlas to use recent JanusGraph version, 0.6.1 released on Jan-18-2022. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (ATLAS-4536) The newline character in attribues will fail simple auth check
[
https://issues.apache.org/jira/browse/ATLAS-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4536:
--
Fix Version/s: 3.0.0
2.3.0
> The newline character in attribues will fail simple auth check
> --
>
> Key: ATLAS-4536
> URL: https://issues.apache.org/jira/browse/ATLAS-4536
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Garry Easop
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> When using Atlas simple authorization and the default json policy file,
> attributes can cause a 403 errors. This is because Atlas ends up calling
> isMatch() and if the string to be matched has a newline character isMatch
> will return false, which leads to the 403. The error in the log looks like:
> {code:java}
> 2021-11-17 22:03:30,328 ERROR - [pool-2-thread-4 -
> c347ce48-4f16-45eb-9453-6d49dde3eb9e:] ~ graph rollback due to exception
> (GraphTransactionInterceptor:167)
> org.apache.atlas.exception.AtlasBaseException: admin is not authorized to
> perform read entity: guid=da8c1532-1aa7-4734-bab3-1567f8565ed3
> at
> org.apache.atlas.authorize.AtlasAuthorizationUtils.verifyAccess(AtlasAuthorizationUtils.java:62)
> at
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.getById(AtlasEntityStoreV2.java:128)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (ATLAS-4536) The newline character in attribues will fail simple auth check
[
https://issues.apache.org/jira/browse/ATLAS-4536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian resolved ATLAS-4536.
---
Resolution: Fixed
> The newline character in attribues will fail simple auth check
> --
>
> Key: ATLAS-4536
> URL: https://issues.apache.org/jira/browse/ATLAS-4536
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Garry Easop
>Priority: Major
>
> When using Atlas simple authorization and the default json policy file,
> attributes can cause a 403 errors. This is because Atlas ends up calling
> isMatch() and if the string to be matched has a newline character isMatch
> will return false, which leads to the 403. The error in the log looks like:
> {code:java}
> 2021-11-17 22:03:30,328 ERROR - [pool-2-thread-4 -
> c347ce48-4f16-45eb-9453-6d49dde3eb9e:] ~ graph rollback due to exception
> (GraphTransactionInterceptor:167)
> org.apache.atlas.exception.AtlasBaseException: admin is not authorized to
> perform read entity: guid=da8c1532-1aa7-4734-bab3-1567f8565ed3
> at
> org.apache.atlas.authorize.AtlasAuthorizationUtils.verifyAccess(AtlasAuthorizationUtils.java:62)
> at
> org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.getById(AtlasEntityStoreV2.java:128)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (ATLAS-4522) Updating typedef with new supertype should be allowed only if attributes are unique compared to other existing supertypes
[
https://issues.apache.org/jira/browse/ATLAS-4522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4522:
--
Fix Version/s: 3.0.0
2.3.0
> Updating typedef with new supertype should be allowed only if attributes are
> unique compared to other existing supertypes
> -
>
> Key: ATLAS-4522
> URL: https://issues.apache.org/jira/browse/ATLAS-4522
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Radhika Kundam
>Assignee: Radhika Kundam
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> 1.Typedef1 with attributes \{a,b,c}
> 2. Typedef2 with attributes \{d} and superType\{Typedef1}
> 3. We created entities for these and everything is fine until here
> 4. Typedef3 with attributes\{a,b,c} => Attributes are same as Typedef1
> 5. Now we updated Typedef2 with one more supertype as below
> Typedef2 with superType\{Typedef1, Typedef3}
> 6. All the Typedef2 entities are having \{a,b,c} attribute values as null and
> because of that when user is searching entity of Typedef2 by it's name not
> returning the result.
> Root cause of the issue is that attributes in Typedef2 from SuperType-
> Typedef1 be overridden by SuperType-Typedef2 attributes as attribute names
> are same for Typedef1 & Typedef2
> Updating typedef with new super types should be allowed only if attributes of
> new supertype are unique and not present in any other existing supertypes.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
Re: Review Request 73798: ATLAS-4529 : DSL Search : OrderBy does not work for ScriptEngineBasedExecutor (gremlin groovy)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73798/#review223958 --- Ship it! Ship It! - Sarath Subramanian On Jan. 12, 2022, 12:55 a.m., Pinal Shah wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73798/ > --- > > (Updated Jan. 12, 2022, 12:55 a.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, and Sarath > Subramanian. > > > Bugs: ATLAS-4529 > https://issues.apache.org/jira/browse/ATLAS-4529 > > > Repository: atlas > > > Description > --- > > After upgrading janusgraph version to 0.6.0 > When below property is changed to 'false', order by doesn't work > > atlas.dsl.executor.traversal=false > > > Diffs > - > > repository/src/main/java/org/apache/atlas/query/GremlinClause.java > 7014b4a4f > > repository/src/main/java/org/apache/atlas/util/AtlasGremlin3QueryProvider.java > 547a43dd7 > > repository/src/test/java/org/apache/atlas/query/GremlinQueryComposerTest.java > 52cb68caa > > > Diff: https://reviews.apache.org/r/73798/diff/1/ > > > Testing > --- > > > Thanks, > > Pinal Shah > >
Re: Review Request 73796: ATLAS-4528: updated log4j2 version to 2.17.1; removed references to org.apache.logging.NDC
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73796/#review223947 --- Ship it! Ship It! - Sarath Subramanian On Jan. 10, 2022, 7:58 a.m., Madhan Neethiraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73796/ > --- > > (Updated Jan. 10, 2022, 7:58 a.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, Pinal Shah, Sarath > Subramanian, and Sidharth Mishra. > > > Bugs: ATLAS-4528 > https://issues.apache.org/jira/browse/ATLAS-4528 > > > Repository: atlas > > > Description > --- > > - log4j2 version updated from 2.15.0 to 2.17.1 > - removed references to org.apache.logging.NDC, as this doesn't seem to be > used any more (was introduced in ATLAS-577) > > > Diffs > - > > pom.xml 8bb406474 > > webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationFilter.java > 6ad0da10c > > > Diff: https://reviews.apache.org/r/73796/diff/1/ > > > Testing > --- > > - precommit tests run at > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1012/ > > > Thanks, > > Madhan Neethiraj > >
Re: Review Request 73762: ATLAS-4508: Though a particular tag is blocked propagation , tag is still propagated
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73762/#review223877
---
Ship it!
Ship It!
- Sarath Subramanian
On Dec. 15, 2021, 3:27 p.m., Sidharth Mishra wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73762/
> ---
>
> (Updated Dec. 15, 2021, 3:27 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Disha Talreja, Jayendra Parab,
> Radhika Kundam, and Sarath Subramanian.
>
>
> Bugs: ATLAS-4508
> https://issues.apache.org/jira/browse/ATLAS-4508
>
>
> Repository: atlas
>
>
> Description
> ---
>
> ATLAS-4508: Made the deferred classification tasks sequential
>
>
> Diffs
> -
>
> repository/src/main/java/org/apache/atlas/tasks/TaskManagement.java
> 9a519bacc
>
>
> Diff: https://reviews.apache.org/r/73762/diff/1/
>
>
> Testing
> ---
>
> Manually Tested:
> 2021-12-14 23:14:08,187 INFO - [etp299413131-209 -
> 7c3d769f-ebca-46c0-aa5b-93b0edb93b22:] ~ Dispatching the number of tasks: 1
> (TaskManagement:178)
> 2021-12-14 23:14:08,188 INFO - [etp299413131-209 -
> 7c3d769f-ebca-46c0-aa5b-93b0edb93b22:] ~
> {"type":"CLASSIFICATION_PROPAGATION_ADD","guid":"9dd556a3-9b33-4ed9-83d6-dd8c8ae2a0c3","createdBy":"admin","createdTime":1639523648041,"updatedTime":1639523648041,"parameters":{"relationshipGuid":null,"entityGuid":"93ef7590-e438-4b26-8eb1-7c2a9840584c","classificationVertexId":"163848304"},"attemptCount":0,"status":"PENDING"}
> (TaskExecutor$TaskLogger:170)
> 2021-12-14 23:14:08,189 INFO - [etp299413131-209 -
> 7c3d769f-ebca-46c0-aa5b-93b0edb93b22:] ~ TaskManagement: Processing stats:
> total=4, sinceLastStatsReport=2 completedWithErrors=0, succeded=4
> (TaskExecutor$TaskLogger:166)
> -2021-12-14 23:14:08,366 INFO - [etp299413131-145 -
> 14af9f29-3cea-4022-a00e-04566a95bc5a:] ~ Dispatching the number of tasks: 1
> (TaskManagement:178)
> 2021-12-14 23:14:08,367 INFO - [etp299413131-145 -
> 14af9f29-3cea-4022-a00e-04566a95bc5a:] ~
> {"type":"CLASSIFICATION_PROPAGATION_RELATIONSHIP_UPDATE","guid":"ff13ed40-5c5e-419a-bb37-01a8831fcaaa","createdBy":"admin","createdTime":1639523648240,"updatedTime":1639523648240,"parameters":{"relationshipEdgeId":"a2p78-38i3k-4e1h-vqo","relationshipObject":"{\"typeName\":\"dataset_process_inputs\",\"guid\":\"e8d10244-7daa-4963-afaf-5774d8128276\",\"provenanceType\":0,\"end1\":{\"guid\":\"ac247f21-d9cd-4632-8934-59412615f175\",\"typeName\":\"hive_process\",\"uniqueAttributes\":{\"qualifiedName\":\"dbbb.tbl_590@manulife:1585415411000\"}},\"end2\":{\"guid\":\"93ef7590-e438-4b26-8eb1-7c2a9840584c\",\"typeName\":\"hive_table\",\"uniqueAttributes\":{\"qualifiedName\":\"dbbb.table_templ@manulife\"}},\"label\":\"__Process.inputs\",\"propagateTags\":\"TWO_TO_ONE\",\"status\":\"ACTIVE\",\"createdBy\":\"admin\",\"updatedBy\":\"admin\",\"createTime\":1638232823426,\"updateTime\":1638232823426,\"versio
n\":0,\"propagatedClassifications\":[{\"typeName\":\"cl10\",\"entityGuid\":\"93ef7590-e438-4b26-8eb1-7c2a9840584c\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"cl6\",\"entityGuid\":\"93ef7590-e438-4b26-8eb1-7c2a9840584c\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"cl3\",\"entityGuid\":\"93ef7590-e438-4b26-8eb1-7c2a9840584c\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"cl13\",\"entityGuid\":\"93ef7590-e438-4b26-8eb1-7c2a9840584c\",\"entityStat
Re: Review Request 73766: ATLAS-4507 : Atlas - Upgrade testng to 7.0.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73766/#review223870 --- Ship it! Ship It! - Sarath Subramanian On Dec. 17, 2021, 7:40 a.m., chaitali wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73766/ > --- > > (Updated Dec. 17, 2021, 7:40 a.m.) > > > Review request for atlas, Jayendra Parab, Pinal Shah, and Sarath Subramanian. > > > Bugs: ATLAS-4507 > https://issues.apache.org/jira/browse/ATLAS-4507 > > > Repository: atlas > > > Description > --- > > Currently Atlas is using 6.9.4 version for testng Upgrading testng to 7.0.0 > > > Diffs > - > > graphdb/common/pom.xml ae5046cbf > pom.xml 2cce90963 > > > Diff: https://reviews.apache.org/r/73766/diff/1/ > > > Testing > --- > > mvn clean install > quick_start.py > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1008/consoleFull > > > Thanks, > > chaitali > >
Re: Review Request 73762: ATLAS-4508: Though a particular tag is blocked propagation , tag is still propagated
> On Dec. 15, 2021, 9:43 p.m., Sarath Subramanian wrote:
> > Ship It!
can you add precommit link?
- Sarath
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73762/#review223864
---
On Dec. 15, 2021, 3:27 p.m., Sidharth Mishra wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73762/
> ---
>
> (Updated Dec. 15, 2021, 3:27 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Disha Talreja, Jayendra Parab,
> Radhika Kundam, and Sarath Subramanian.
>
>
> Bugs: ATLAS-4508
> https://issues.apache.org/jira/browse/ATLAS-4508
>
>
> Repository: atlas
>
>
> Description
> ---
>
> ATLAS-4508: Made the deferred classification tasks sequential
>
>
> Diffs
> -
>
> repository/src/main/java/org/apache/atlas/tasks/TaskManagement.java
> 9a519bacc
>
>
> Diff: https://reviews.apache.org/r/73762/diff/1/
>
>
> Testing
> ---
>
> Manually Tested:
> 2021-12-14 23:14:08,187 INFO - [etp299413131-209 -
> 7c3d769f-ebca-46c0-aa5b-93b0edb93b22:] ~ Dispatching the number of tasks: 1
> (TaskManagement:178)
> 2021-12-14 23:14:08,188 INFO - [etp299413131-209 -
> 7c3d769f-ebca-46c0-aa5b-93b0edb93b22:] ~
> {"type":"CLASSIFICATION_PROPAGATION_ADD","guid":"9dd556a3-9b33-4ed9-83d6-dd8c8ae2a0c3","createdBy":"admin","createdTime":1639523648041,"updatedTime":1639523648041,"parameters":{"relationshipGuid":null,"entityGuid":"93ef7590-e438-4b26-8eb1-7c2a9840584c","classificationVertexId":"163848304"},"attemptCount":0,"status":"PENDING"}
> (TaskExecutor$TaskLogger:170)
> 2021-12-14 23:14:08,189 INFO - [etp299413131-209 -
> 7c3d769f-ebca-46c0-aa5b-93b0edb93b22:] ~ TaskManagement: Processing stats:
> total=4, sinceLastStatsReport=2 completedWithErrors=0, succeded=4
> (TaskExecutor$TaskLogger:166)
> -2021-12-14 23:14:08,366 INFO - [etp299413131-145 -
> 14af9f29-3cea-4022-a00e-04566a95bc5a:] ~ Dispatching the number of tasks: 1
> (TaskManagement:178)
> 2021-12-14 23:14:08,367 INFO - [etp299413131-145 -
> 14af9f29-3cea-4022-a00e-04566a95bc5a:] ~
> {"type":"CLASSIFICATION_PROPAGATION_RELATIONSHIP_UPDATE","guid":"ff13ed40-5c5e-419a-bb37-01a8831fcaaa","createdBy":"admin","createdTime":1639523648240,"updatedTime":1639523648240,"parameters":{"relationshipEdgeId":"a2p78-38i3k-4e1h-vqo","relationshipObject":"{\"typeName\":\"dataset_process_inputs\",\"guid\":\"e8d10244-7daa-4963-afaf-5774d8128276\",\"provenanceType\":0,\"end1\":{\"guid\":\"ac247f21-d9cd-4632-8934-59412615f175\",\"typeName\":\"hive_process\",\"uniqueAttributes\":{\"qualifiedName\":\"dbbb.tbl_590@manulife:1585415411000\"}},\"end2\":{\"guid\":\"93ef7590-e438-4b26-8eb1-7c2a9840584c\",\"typeName\":\"hive_table\",\"uniqueAttributes\":{\"qualifiedName\":\"dbbb.table_templ@manulife\"}},\"label\":\"__Process.inputs\",\"propagateTags\":\"TWO_TO_ONE\",\"status\":\"ACTIVE\",\"createdBy\":\"admin\",\"updatedBy\":\"admin\",\"createTime\":1638232823426,\"updateTime\":1638232823426,\"versio
n\":0,\"propagatedClassifications\":[{\"typeName\":\"cl10\",\"entityGuid\":\"93ef7590-e438-4b26-8eb1-7c2a9840584c\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"cl6\",\"entityGuid\":\"93ef7590-e438-4b26-8eb1-7c2a9840584c\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"cl3\",\"entityGuid\":\"93ef7590-e438-4b26-8eb1-7c2a9840584c\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"cl13\",\"entityGuid
Re: Review Request 73762: ATLAS-4508: Though a particular tag is blocked propagation , tag is still propagated
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73762/#review223864
---
Ship it!
Ship It!
- Sarath Subramanian
On Dec. 15, 2021, 3:27 p.m., Sidharth Mishra wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73762/
> ---
>
> (Updated Dec. 15, 2021, 3:27 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Disha Talreja, Jayendra Parab,
> Radhika Kundam, and Sarath Subramanian.
>
>
> Bugs: ATLAS-4508
> https://issues.apache.org/jira/browse/ATLAS-4508
>
>
> Repository: atlas
>
>
> Description
> ---
>
> ATLAS-4508: Made the deferred classification tasks sequential
>
>
> Diffs
> -
>
> repository/src/main/java/org/apache/atlas/tasks/TaskManagement.java
> 9a519bacc
>
>
> Diff: https://reviews.apache.org/r/73762/diff/1/
>
>
> Testing
> ---
>
> Manually Tested:
> 2021-12-14 23:14:08,187 INFO - [etp299413131-209 -
> 7c3d769f-ebca-46c0-aa5b-93b0edb93b22:] ~ Dispatching the number of tasks: 1
> (TaskManagement:178)
> 2021-12-14 23:14:08,188 INFO - [etp299413131-209 -
> 7c3d769f-ebca-46c0-aa5b-93b0edb93b22:] ~
> {"type":"CLASSIFICATION_PROPAGATION_ADD","guid":"9dd556a3-9b33-4ed9-83d6-dd8c8ae2a0c3","createdBy":"admin","createdTime":1639523648041,"updatedTime":1639523648041,"parameters":{"relationshipGuid":null,"entityGuid":"93ef7590-e438-4b26-8eb1-7c2a9840584c","classificationVertexId":"163848304"},"attemptCount":0,"status":"PENDING"}
> (TaskExecutor$TaskLogger:170)
> 2021-12-14 23:14:08,189 INFO - [etp299413131-209 -
> 7c3d769f-ebca-46c0-aa5b-93b0edb93b22:] ~ TaskManagement: Processing stats:
> total=4, sinceLastStatsReport=2 completedWithErrors=0, succeded=4
> (TaskExecutor$TaskLogger:166)
> -2021-12-14 23:14:08,366 INFO - [etp299413131-145 -
> 14af9f29-3cea-4022-a00e-04566a95bc5a:] ~ Dispatching the number of tasks: 1
> (TaskManagement:178)
> 2021-12-14 23:14:08,367 INFO - [etp299413131-145 -
> 14af9f29-3cea-4022-a00e-04566a95bc5a:] ~
> {"type":"CLASSIFICATION_PROPAGATION_RELATIONSHIP_UPDATE","guid":"ff13ed40-5c5e-419a-bb37-01a8831fcaaa","createdBy":"admin","createdTime":1639523648240,"updatedTime":1639523648240,"parameters":{"relationshipEdgeId":"a2p78-38i3k-4e1h-vqo","relationshipObject":"{\"typeName\":\"dataset_process_inputs\",\"guid\":\"e8d10244-7daa-4963-afaf-5774d8128276\",\"provenanceType\":0,\"end1\":{\"guid\":\"ac247f21-d9cd-4632-8934-59412615f175\",\"typeName\":\"hive_process\",\"uniqueAttributes\":{\"qualifiedName\":\"dbbb.tbl_590@manulife:1585415411000\"}},\"end2\":{\"guid\":\"93ef7590-e438-4b26-8eb1-7c2a9840584c\",\"typeName\":\"hive_table\",\"uniqueAttributes\":{\"qualifiedName\":\"dbbb.table_templ@manulife\"}},\"label\":\"__Process.inputs\",\"propagateTags\":\"TWO_TO_ONE\",\"status\":\"ACTIVE\",\"createdBy\":\"admin\",\"updatedBy\":\"admin\",\"createTime\":1638232823426,\"updateTime\":1638232823426,\"versio
n\":0,\"propagatedClassifications\":[{\"typeName\":\"cl10\",\"entityGuid\":\"93ef7590-e438-4b26-8eb1-7c2a9840584c\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"cl6\",\"entityGuid\":\"93ef7590-e438-4b26-8eb1-7c2a9840584c\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"cl3\",\"entityGuid\":\"93ef7590-e438-4b26-8eb1-7c2a9840584c\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"cl13\",\"entityGuid\":\"93ef7590-e438-4b26-8eb1-7c2a9840584c\",\"entityStat
Re: Review Request 73747: ATLAS-4502: Add cassandra and elasticsearch search profile for Atlas
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73747/#review223825 --- Ship it! Ship It! - Sarath Subramanian On Dec. 8, 2021, 4:26 p.m., Sidharth Mishra wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73747/ > --- > > (Updated Dec. 8, 2021, 4:26 p.m.) > > > Review request for atlas, Ashutosh Mestry, Disha Talreja, Radhika Kundam, and > Sarath Subramanian. > > > Bugs: ATLAS-4502 > https://issues.apache.org/jira/browse/ATLAS-4502 > > > Repository: atlas > > > Description > --- > > ATLAS-4502: Add cassandra and elasticsearch search profile for Atlas > > > Diffs > - > > distro/pom.xml 950a79c85 > pom.xml 4c92a09ad > > repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java > fb890fc5d > webapp/pom.xml 858ab9aaf > > > Diff: https://reviews.apache.org/r/73747/diff/2/ > > > Testing > --- > > Manually tested locally with local Cassandra ans embedded elastic search > > > Thanks, > > Sidharth Mishra > >
Re: Review Request 73747: ATLAS-4502: Add cassandra and elasticsearch search profile for Atlas
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73747/#review223817
---
Fix it, then Ship it!
pom.xml
Line 586 (original), 586 (patched)
<https://reviews.apache.org/r/73747/#comment312878>
consider using variable here ${guava.version}
- Sarath Subramanian
On Dec. 7, 2021, 12:21 p.m., Sidharth Mishra wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73747/
> ---
>
> (Updated Dec. 7, 2021, 12:21 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Disha Talreja, Radhika Kundam, and
> Sarath Subramanian.
>
>
> Bugs: ATLAS-4502
> https://issues.apache.org/jira/browse/ATLAS-4502
>
>
> Repository: atlas
>
>
> Description
> ---
>
> ATLAS-4502: Add cassandra and elasticsearch search profile for Atlas
>
>
> Diffs
> -
>
> distro/pom.xml 950a79c85
> pom.xml 4c92a09ad
> webapp/pom.xml 858ab9aaf
>
>
> Diff: https://reviews.apache.org/r/73747/diff/1/
>
>
> Testing
> ---
>
> Manually tested locally with local Cassandra ans embedded elastic search
>
>
> Thanks,
>
> Sidharth Mishra
>
>
Re: [DRAFT] [REPORT] Apache Atlas - Dec-2021
Thanks Madhan for the draft report. +1 Thanks, Sarath On Wed, Dec 8, 2021 at 1:20 AM Madhan Neethiraj wrote: > Atlas community, > > Please review the following draft report for this quarter. > > Thanks, > Madhan > > ## Description: > The mission of Apache Atlas is the creation and maintenance of software > related > to a scalable and extensible set of core foundational governance services > - > enabling enterprises to effectively and efficiently meet their compliance > requirements > > ## Issues: > There are no issues requiring board attention. > > ## Membership Data: > Apache Atlas was founded 2017-06-20 (4 years ago) > There are currently 44 committers and 32 PMC members in this project. > The Committer-to-PMC ratio is roughly 3:2. > > Community changes, past quarter: > - No new PMC members. Last addition was Ashutosh Mestry on 2019-04-16. > - No new committers. Last addition was Sidharth Mishra on 2021-04-19. > > ## Project Activity: > - enhanced hook messages to capture hook messages and message source > - updated quick search to support sorting > - added metrics persistence to retrieve historical data > - UI: updated term details page to add additional attributes > - UI: chart representation of metrics data > - UI: implemented session timeout > - performance improvement in glossary bulk import > - fixes in index health monitor > - Dependent component version updates: JanusGraph 0.6.0, Apache Strom > 2.3.0, Elastic search 6.8.17, Apache Kafka 2.8.1, netty 4.1.69, Junit > 4.13.1, jackson 2.11.0, commons-io 2.11.0, Spring Security 5.4.4 > > ## Community Health: > - dev@atlas.apache.org had a 10% decrease in traffic in the past quarter > (798 > emails compared to 881) > - u...@atlas.apache.org had a 45% decrease in traffic in the past quarter > (10 > emails compared to 18) > - 87 issues opened in JIRA, past quarter (3% increase) > - 56 issues closed in JIRA, past quarter (-43% change) > - 111 commits in the past quarter (-5% change) > - 15 code contributors in the past quarter (-34% change) > - 7 PRs opened on GitHub, past quarter (no change) > - 3 PRs closed on GitHub, past quarter (-50% change) > > ## Most Recent releases: > 2.2.0 was released on 2021-08-17 > 2.1.0 was released on 2020-07-15 > 0.8.4 was released on 2019-06-21 > > >
Re: Review Request 73740: ATLAS-4491 :- Atlas - Upgrade Spring framework to 5.3.10+/5.2.17+
> On Dec. 6, 2021, 10:52 a.m., Sarath Subramanian wrote: > > Ship It! do we have a clean pre-commit? - Sarath --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73740/#review223808 --- On Dec. 6, 2021, 1:58 a.m., Vinayak Marraiya wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73740/ > --- > > (Updated Dec. 6, 2021, 1:58 a.m.) > > > Review request for atlas, Ashutosh Mestry, chaitali, Jayendra Parab, Mandar > Ambawane, Pinal Shah, and Sarath Subramanian. > > > Bugs: ATLAS-4491 > https://issues.apache.org/jira/browse/ATLAS-4491 > > > Repository: atlas > > > Description > --- > > Atlas upgraded to 5.3.10 or higher > > > Diffs > - > > pom.xml 4b3408acc > > > Diff: https://reviews.apache.org/r/73740/diff/2/ > > > Testing > --- > > 1)mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > Atlas server runnning > 2)Quick started was passed successfully > > > Precommit - > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/982/console > > > Thanks, > > Vinayak Marraiya > >
Re: Review Request 73740: ATLAS-4491 :- Atlas - Upgrade Spring framework to 5.3.10+/5.2.17+
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73740/#review223808 --- Ship it! Ship It! - Sarath Subramanian On Dec. 6, 2021, 1:58 a.m., Vinayak Marraiya wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73740/ > --- > > (Updated Dec. 6, 2021, 1:58 a.m.) > > > Review request for atlas, Ashutosh Mestry, chaitali, Jayendra Parab, Mandar > Ambawane, Pinal Shah, and Sarath Subramanian. > > > Bugs: ATLAS-4491 > https://issues.apache.org/jira/browse/ATLAS-4491 > > > Repository: atlas > > > Description > --- > > Atlas upgraded to 5.3.10 or higher > > > Diffs > - > > pom.xml 4b3408acc > > > Diff: https://reviews.apache.org/r/73740/diff/2/ > > > Testing > --- > > 1)mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > Atlas server runnning > 2)Quick started was passed successfully > > > Precommit - > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/982/console > > > Thanks, > > Vinayak Marraiya > >
Re: Review Request 73637: ATLAS-4370: Persist Metrics for user to retrieve Metrics info at a past timestamp
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73637/#review223793 --- Ship it! Ship It! - Sarath Subramanian On Nov. 15, 2021, 8:06 a.m., Disha Talreja wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73637/ > --- > > (Updated Nov. 15, 2021, 8:06 a.m.) > > > Review request for atlas, Ashutosh Mestry, Radhika Kundam, Sarath > Subramanian, and Sidharth Mishra. > > > Bugs: ATLAS-4370 > https://issues.apache.org/jira/browse/ATLAS-4370 > > > Repository: atlas > > > Description > --- > > Rebased patch #73483 (https://reviews.apache.org/r/73483/) against the newest > master branch. > > > Diffs > - > > addons/models/-Area0/0010-base_model.json 769d8859f > intg/src/main/java/org/apache/atlas/AtlasConfiguration.java b63fab7cf > intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 7d092616d > > intg/src/main/java/org/apache/atlas/model/metrics/AtlasMetricsMapToChart.java > PRE-CREATION > intg/src/main/java/org/apache/atlas/model/metrics/AtlasMetricsStat.java > PRE-CREATION > intg/src/main/java/org/apache/atlas/utils/AtlasEntityUtil.java 1e78e2550 > > repository/src/main/java/org/apache/atlas/repository/ogm/metrics/AtlasMetricsStatDTO.java > PRE-CREATION > repository/src/main/java/org/apache/atlas/services/MetricsService.java > ceb252822 > repository/src/test/java/org/apache/atlas/TestModules.java 8dda20825 > repository/src/test/java/org/apache/atlas/services/MetricsServiceTest.java > 040592184 > test-tools/src/main/resources/solr/core-template/solrconfig.xml 3341e7199 > webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java > 0580f7f33 > > > Diff: https://reviews.apache.org/r/73637/diff/7/ > > > Testing > --- > > 1) Tested locally. > 2) Patched a cluster. > > PC build: > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/880/ > > > Thanks, > > Disha Talreja > >
Re: Review Request 73704: ATLAS-4475: Import Hive V2 is throwing wrong log message when running with set of tables
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73704/#review223752 --- Ship it! Ship It! - Sarath Subramanian On Nov. 16, 2021, 5:06 p.m., Sidharth Mishra wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73704/ > --- > > (Updated Nov. 16, 2021, 5:06 p.m.) > > > Review request for atlas, Ashutosh Mestry, Disha Talreja, Radhika Kundam, and > Sarath Subramanian. > > > Bugs: ATLAS-4475 > https://issues.apache.org/jira/browse/ATLAS-4475 > > > Repository: atlas > > > Description > --- > > ATLAS-4475: Fixed log messages for Import Hive V2 when running with set of > tables > > > Diffs > - > > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeV2.java > 0627c0e09 > > > Diff: https://reviews.apache.org/r/73704/diff/2/ > > > Testing > --- > > Manually tested. PreCommit run - > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/974/ > > > Thanks, > > Sidharth Mishra > >
[jira] [Commented] (ATLAS-4485) Can HA support multi write/read?
[ https://issues.apache.org/jira/browse/ATLAS-4485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17446029#comment-17446029 ] Sarath Subramanian commented on ATLAS-4485: --- [~wzg547228197] , yes Atlas HA follows active/passive model where automatic failover happens when primary instance crashes. We don't currently support read replicas. > Can HA support multi write/read? > > > Key: ATLAS-4485 > URL: https://issues.apache.org/jira/browse/ATLAS-4485 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.2.0 >Reporter: wuzhiguo >Priority: Major > Labels: features, performance > > Currently HA is like a backup mode, the write/read operation can only go > through main instance, If main instance crashed because of sudden increase in > traffic, the backup instance will meet same problem. -- This message was sent by Atlassian Jira (v8.20.1#820001)
Re: Review Request 73711: ATLAS-4481: Atlas API on update classifications is failing with NPE when we pass body without propagate flag
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73711/#review223742 --- Ship it! Ship It! - Sarath Subramanian On Nov. 16, 2021, 1:32 p.m., Sidharth Mishra wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73711/ > --- > > (Updated Nov. 16, 2021, 1:32 p.m.) > > > Review request for atlas, Ashutosh Mestry, Disha Talreja, Radhika Kundam, and > Sarath Subramanian. > > > Bugs: ATLAS-4481 > https://issues.apache.org/jira/browse/ATLAS-4481 > > > Repository: atlas > > > Description > --- > > ATLAS-4481: Fixed NPE at Atlas API on update classifications without > propagate flag > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java > 6d8305a32 > > > Diff: https://reviews.apache.org/r/73711/diff/1/ > > > Testing > --- > > Manually tested. Precommit - > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/971/ > > > Thanks, > > Sidharth Mishra > >
Re: Review Request 73700: ATLAS-4473 GlossaryTerms Bulk Create Performance Improvement
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73700/#review223730 --- Fix it, then Ship it! repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java Lines 1011 (patched) <https://reviews.apache.org/r/73700/#comment312832> extract as static constant - "r:AtlasGlossaryTermAnchor" - Sarath Subramanian On Nov. 9, 2021, 11:23 p.m., Mandar Ambawane wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73700/ > --- > > (Updated Nov. 9, 2021, 11:23 p.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, Pinal Shah, and > Sarath Subramanian. > > > Bugs: ATLAS-4473 > https://issues.apache.org/jira/browse/ATLAS-4473 > > > Repository: atlas > > > Description > --- > > While creating GlossaryTerms we need information about Glossary and > GlossaryCategory (if present) > > As per current design, > When every Term is created, respective Glossary and Category objects are > fetched from db and they are populated with unwanted information. > > In case of bulk creation, > Even if multiple Terms have same Glossary and Category associated with them, > every time these objects are fetched from db and they are populated. > > This patch stores the optimum information of Glossaries and Categoies in the > cache against their respective Guids. > This helps in performance improvement of Bulk Term Creation. > > > Diffs > - > > repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java > 9c8459874 > repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java > d92daeea5 > > > Diff: https://reviews.apache.org/r/73700/diff/1/ > > > Testing > --- > > Earlier, it was taking more than a day to complete the creation process of > the Terms. > > Now with the Performacne optimization, it is taking 5 to 6 minutes to > complete. > > > Thanks, > > Mandar Ambawane > >
Re: Review Request 73637: ATLAS-4370: Persist Metrics for user to retrieve Metrics info at a past timestamp
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73637/#review223729
---
Fix it, then Ship it!
addons/models/-Area0/0010-base_model.json
Lines 416 (patched)
<https://reviews.apache.org/r/73637/#comment312829>
"__AtlasMetricsStat" type is duplicated. Please review.
webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java
Lines 428 (patched)
<https://reviews.apache.org/r/73637/#comment312830>
why add 4 debug log for entry/exit methods? Add a single entry and exit for
main method:
At start:
LOG.debug("==> AdminResource.scheduleSaveAndDeleteMetrics()");
At End:
LOG.debug("<== AdminResource.scheduleSaveAndDeleteMetrics()");
webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java
Lines 453 (patched)
<https://reviews.apache.org/r/73637/#comment312831>
consider moving private methods after public
- Sarath Subramanian
On Nov. 9, 2021, 11:54 a.m., Disha Talreja wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73637/
> ---
>
> (Updated Nov. 9, 2021, 11:54 a.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Radhika Kundam, Sarath
> Subramanian, and Sidharth Mishra.
>
>
> Bugs: ATLAS-4370
> https://issues.apache.org/jira/browse/ATLAS-4370
>
>
> Repository: atlas
>
>
> Description
> ---
>
> Rebased patch #73483 (https://reviews.apache.org/r/73483/) against the newest
> master branch.
>
>
> Diffs
> -
>
> addons/models/-Area0/0010-base_model.json 769d8859f
> intg/src/main/java/org/apache/atlas/AtlasConfiguration.java b63fab7cf
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 7d092616d
>
> intg/src/main/java/org/apache/atlas/model/metrics/AtlasMetricsMapToChart.java
> PRE-CREATION
> intg/src/main/java/org/apache/atlas/model/metrics/AtlasMetricsStat.java
> PRE-CREATION
> intg/src/main/java/org/apache/atlas/utils/AtlasEntityUtil.java 1e78e2550
>
> repository/src/main/java/org/apache/atlas/repository/ogm/metrics/AtlasMetricsStatDTO.java
> PRE-CREATION
> repository/src/main/java/org/apache/atlas/services/MetricsService.java
> ceb252822
> repository/src/test/java/org/apache/atlas/TestModules.java 8dda20825
> repository/src/test/java/org/apache/atlas/services/MetricsServiceTest.java
> 040592184
> test-tools/src/main/resources/solr/core-template/solrconfig.xml 3341e7199
> webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java
> 0580f7f33
>
>
> Diff: https://reviews.apache.org/r/73637/diff/6/
>
>
> Testing
> ---
>
> 1) Tested locally.
> 2) Patched a cluster.
>
> PC build:
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/880/
>
>
> Thanks,
>
> Disha Talreja
>
>
Re: Review Request 73691: ATLAS-4431: Random NPE when retrieving tasks (“part2 - additional null check”)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73691/#review223715 --- Ship it! Ship It! - Sarath Subramanian On Nov. 8, 2021, 3:33 p.m., Disha Talreja wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73691/ > --- > > (Updated Nov. 8, 2021, 3:33 p.m.) > > > Review request for atlas, Ashutosh Mestry, Radhika Kundam, Sarath > Subramanian, and Sidharth Mishra. > > > Bugs: ATLAS-4431 > https://issues.apache.org/jira/browse/ATLAS-4431 > > > Repository: atlas > > > Description > --- > > System tests fail randomly because retrieving tasks using the "admin/tasks" > endpoint sometimes returns NPE. This patch includes a few additional null > checks to avoid NPE. > > > Diffs > - > > repository/src/main/java/org/apache/atlas/tasks/TaskRegistry.java 32e0ad91b > > > Diff: https://reviews.apache.org/r/73691/diff/4/ > > > Testing > --- > > Local Build Successful. > > > Thanks, > > Disha Talreja > >
Re: Review Request 73586: ATLAS-4424: Enhance the Import hive utility to create export zip files and run bulk import
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73586/#review223712 --- Fix it, then Ship it! addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeV2.java Lines 459 (patched) <https://reviews.apache.org/r/73586/#comment312820> this method - loadRelatedGuidsFromAtlas() populates the 'qualifiedNameGuidMap' cache. Consider renaming to: populateQualifiedNameGuidMap() for more relavence. - Sarath Subramanian On Oct. 29, 2021, 12:40 p.m., Sidharth Mishra wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73586/ > --- > > (Updated Oct. 29, 2021, 12:40 p.m.) > > > Review request for atlas, Ashutosh Mestry, Disha Talreja, Radhika Kundam, and > Sarath Subramanian. > > > Bugs: ATLAS-4424 > https://issues.apache.org/jira/browse/ATLAS-4424 > > > Repository: atlas > > > Description > --- > > ATLAS-4424: Enhanced the Import hive utility to create export zip files and > run bulk import > > > Diffs > - > > addons/hive-bridge/src/bin/import-hive.sh c3539377f > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java > c361ac69a > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeV2.java > PRE-CREATION > intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java > 2c1870444 > > intg/src/main/java/org/apache/atlas/model/migration/MigrationImportStatus.java > 3430fdae8 > > repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationStatusService.java > a22c68790 > > repository/src/main/java/org/apache/atlas/repository/migration/ZipFileMigrationImporter.java > bfb114814 > > repository/src/test/java/org/apache/atlas/repository/impexp/DataMigrationStatusServiceTest.java > f1dc990e8 > webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java > 135b94b36 > > > Diff: https://reviews.apache.org/r/73586/diff/7/ > > > Testing > --- > > Manually tested the new zip file creation and bulk import to atlas. PreCommit > - https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/941/ > > > Thanks, > > Sidharth Mishra > >
Re: Review Request 73669: ATLAS-4462: Atlas tests are failing and PreCommit tests are also failing
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73669/#review223692 --- Ship it! Ship It! - Sarath Subramanian On Oct. 28, 2021, 5:02 p.m., Sidharth Mishra wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73669/ > --- > > (Updated Oct. 28, 2021, 5:02 p.m.) > > > Review request for atlas, Ashutosh Mestry, Disha Talreja, Radhika Kundam, and > Sarath Subramanian. > > > Bugs: ATLAS-4462 > https://issues.apache.org/jira/browse/ATLAS-4462 > > > Repository: atlas > > > Description > --- > > ATLAS-4462: Updated the pom to stop jetty before start, Added daemon flag to > true to stop jetty at end of post integration, Added retry logic for Kafka > start having port bind issue, Fixed Infinite loop for Index Recovery Service > > > Diffs > - > > addons/falcon-bridge/pom.xml de22fa5a0 > addons/hbase-bridge/pom.xml 50fb9e838 > addons/hive-bridge/pom.xml ea2de4d3b > addons/impala-bridge/pom.xml a1bd5920b > addons/kafka-bridge/pom.xml 7fe97eb39 > addons/sqoop-bridge/pom.xml 9d5ac52d9 > addons/storm-bridge/pom.xml 71525438b > notification/src/main/java/org/apache/atlas/kafka/EmbeddedKafkaServer.java > 19717fb2b > notification/src/main/java/org/apache/atlas/util/CommandHandlerUtility.java > PRE-CREATION > pom.xml 61f132320 > webapp/pom.xml d90f68601 > > > Diff: https://reviews.apache.org/r/73669/diff/5/ > > > Testing > --- > > Manually tested the Infinite loop issue at Index Recovery Service, Kafka > retry logic for kafka start. Along with this I ran the local tests which > passed. > > > Thanks, > > Sidharth Mishra > >
Re: Review Request 73670: ATLAS-4351 Maven-jetty throwing warnings when running Integration tests for conflicting jars in classpath
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73670/#review223689 --- Ship it! Ship It! - Sarath Subramanian On Oct. 28, 2021, 2:14 p.m., Sidharth Mishra wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73670/ > --- > > (Updated Oct. 28, 2021, 2:14 p.m.) > > > Review request for atlas, Ashutosh Mestry, Disha Talreja, Radhika Kundam, and > Sarath Subramanian. > > > Bugs: ATLAS-4351 > https://issues.apache.org/jira/browse/ATLAS-4351 > > > Repository: atlas > > > Description > --- > > Reverting: ATLAS-4351 > Maven-jetty throwing warnings when running Integration tests for conflicting > jars in classpath > > > Diffs > - > > addons/falcon-bridge/pom.xml 9d56fd8d3 > addons/hbase-bridge/pom.xml 151a62a77 > addons/hbase-testing-util/pom.xml 8ca962adc > addons/hive-bridge/pom.xml f62140918 > addons/impala-bridge/pom.xml 87650e1da > addons/kafka-bridge/pom.xml 2ac19fd20 > addons/sqoop-bridge/pom.xml 03861df84 > addons/storm-bridge-shim/pom.xml e1e89ceda > addons/storm-bridge/pom.xml 6bfe15b5b > common/pom.xml ebf8ea5ae > graphdb/janus-hbase2/pom.xml 9ad64062f > graphdb/janus/pom.xml f38c4e9a3 > pom.xml 84630e76b > test-tools/pom.xml 59c7a9dbf > webapp/pom.xml 443559c8d > > > Diff: https://reviews.apache.org/r/73670/diff/1/ > > > Testing > --- > > As there are Precommit job failures I am revrting the changes. Later we can > fix the warning issue separately. > > > Thanks, > > Sidharth Mishra > >
Re: Review Request 73671: ATLAS-4463: Infinite loop at Index Health Monitor (IndexRecoveryService)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73671/#review223688 --- Ship it! Ship It! - Sarath Subramanian On Oct. 28, 2021, 2:32 p.m., Sidharth Mishra wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73671/ > --- > > (Updated Oct. 28, 2021, 2:32 p.m.) > > > Review request for atlas, Ashutosh Mestry, Disha Talreja, Radhika Kundam, and > Sarath Subramanian. > > > Bugs: ATLAS-4463 > https://issues.apache.org/jira/browse/ATLAS-4463 > > > Repository: atlas > > > Description > --- > > ATLAS-4463: Fixed Infinite loop at Index Health Monitor (IndexRecoveryService) > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/repository/graph/IndexRecoveryService.java > 2f1161067 > > > Diff: https://reviews.apache.org/r/73671/diff/2/ > > > Testing > --- > > Run local UT/ITs and Manually ran jetty run and some IT to see the loop is > exiting at the end > > > Thanks, > > Sidharth Mishra > >
Re: Review Request 73669: ATLAS-4462: Atlas tests are failing and PreCommit tests are also failing
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73669/#review223687
---
Fix it, then Ship it!
notification/src/main/java/org/apache/atlas/util/CommandHandlerUtility.java
Lines 33 (patched)
<https://reviews.apache.org/r/73669/#comment312794>
+1
'attemptCount' is not a valid member when you are making this a generic
method in CommandHandlerUtility
use boolean - forceKill - true/false
notification/src/main/java/org/apache/atlas/util/CommandHandlerUtility.java
Lines 38 (patched)
<https://reviews.apache.org/r/73669/#comment312795>
getPIDOfPortUse => findProcessIdUsingPort
notification/src/main/java/org/apache/atlas/util/CommandHandlerUtility.java
Lines 40 (patched)
<https://reviews.apache.org/r/73669/#comment312793>
'reader' is never used? consider removing
pom.xml
Line 1963 (original), 1963 (patched)
<https://reviews.apache.org/r/73669/#comment312796>
consider using variable here:
${surefire.version}
- Sarath Subramanian
On Oct. 28, 2021, 2:19 p.m., Sidharth Mishra wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73669/
> ---
>
> (Updated Oct. 28, 2021, 2:19 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Disha Talreja, Radhika Kundam, and
> Sarath Subramanian.
>
>
> Bugs: ATLAS-4462
> https://issues.apache.org/jira/browse/ATLAS-4462
>
>
> Repository: atlas
>
>
> Description
> ---
>
> ATLAS-4462: Updated the pom to stop jetty before start, Added daemon flag to
> true to stop jetty at end of post integration, Added retry logic for Kafka
> start having port bind issue, Fixed Infinite loop for Index Recovery Service
>
>
> Diffs
> -
>
> addons/falcon-bridge/pom.xml 9d56fd8d3
> addons/hbase-bridge/pom.xml 151a62a77
> addons/hive-bridge/pom.xml f62140918
> addons/impala-bridge/pom.xml 87650e1da
> addons/kafka-bridge/pom.xml 2ac19fd20
> addons/sqoop-bridge/pom.xml 03861df84
> addons/storm-bridge/pom.xml 6bfe15b5b
> notification/src/main/java/org/apache/atlas/kafka/EmbeddedKafkaServer.java
> 19717fb2b
> notification/src/main/java/org/apache/atlas/util/CommandHandlerUtility.java
> PRE-CREATION
> pom.xml 84630e76b
> webapp/pom.xml 443559c8d
>
>
> Diff: https://reviews.apache.org/r/73669/diff/2/
>
>
> Testing
> ---
>
> Manually tested the Infinite loop issue at Index Recovery Service, Kafka
> retry logic for kafka start. Along with this I ran the local tests which
> passed.
>
>
> Thanks,
>
> Sidharth Mishra
>
>
Re: Review Request 73654: ATLAS-4454 : Upgrade JanusGraph version to 0.6.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73654/#review223633 --- Ship it! Ship It! - Sarath Subramanian On Oct. 19, 2021, 4:27 a.m., Pinal Shah wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73654/ > --- > > (Updated Oct. 19, 2021, 4:27 a.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, and Sarath > Subramanian. > > > Bugs: ATLAS-4454 > https://issues.apache.org/jira/browse/ATLAS-4454 > > > Repository: atlas > > > Description > --- > > Upgrading janusgraph version - 0.6.0 > Upgrading tinkerpop version - 3.5.1 > > > Diffs > - > > graphdb/janus-hbase2/pom.xml 9ad64062f > > graphdb/janus-hbase2/src/main/java/org/janusgraph/diskstorage/hbase2/HBaseKeyColumnValueStore.java > 9aa552d4a > > graphdb/janus-hbase2/src/main/java/org/janusgraph/diskstorage/hbase2/HBaseStoreManager.java > 40bcd1ad3 > graphdb/janus/pom.xml f38c4e9a3 > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphDatabase.java > 0d47e38fd > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/AtlasGraphSONReader.java > 2d5bd8ae7 > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/query/NativeJanusGraphQuery.java > 16fc4451b > graphdb/janus/src/main/java/org/janusgraph/diskstorage/solr/Solr6Index.java > 81ae202f1 > > graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/BaseUtils.java > 82ae67514 > pom.xml 84630e76b > repository/src/test/java/org/apache/atlas/query/TraversalComposerTest.java > dd56ed141 > test-tools/src/main/resources/solr/core-template/solrconfig.xml e7956cb36 > tools/atlas-index-repair/pom.xml a723bfa0c > > > Diff: https://reviews.apache.org/r/73654/diff/3/ > > > Testing > --- > > mvn clean package -Pdist,embedded-hbase-solr- passes > Quick start - passes > > Precommit running - > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/900 > > > Temporary changes for solrconfig.xml to pass PC > > > Thanks, > > Pinal Shah > >
Re: Review Request 73586: ATLAS-4424: Enhance the Import hive utility to create export zip files and run bulk import
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73586/#review223618
---
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
Line 131 (original), 150 (patched)
<https://reviews.apache.org/r/73586/#comment312721>
instead of throwing ParseException, should we set exitCode =
EXIT_CODE_INVALID_ARG?
Also add a lOG.ERROR about the details.
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
Lines 190 (patched)
<https://reviews.apache.org/r/73586/#comment312722>
consider combining catch statements 190 and 194, since the contents are
same:
catch (MissingArgumentException | ParseException e) {
...
}
addons/hive-bridge/src/bin/import-hive.sh
Line 160 (original), 168 (patched)
<https://reviews.apache.org/r/73586/#comment312745>
nit: this overrides previous change. Consider reverting and keep the
original echo msg.
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeV2.java
Lines 163 (patched)
<https://reviews.apache.org/r/73586/#comment312744>
line 163-169 (4 methods) deals with error checking and initializing.
Consider encapsulating to an init() method to hide the specifics.
intg/src/main/java/org/apache/atlas/model/migration/MigrationImportStatus.java
Line 38 (original), 33 (patched)
<https://reviews.apache.org/r/73586/#comment312746>
instead of getting rid of "name" and using "fileHash". Can we capture both
- name and fileHash?
will this change in POJO, affect cross-version compat (backward-compatible)?
- Sarath Subramanian
On Oct. 17, 2021, 11:18 p.m., Sidharth Mishra wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73586/
> ---
>
> (Updated Oct. 17, 2021, 11:18 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Disha Talreja, Radhika Kundam, and
> Sarath Subramanian.
>
>
> Bugs: ATLAS-4424
> https://issues.apache.org/jira/browse/ATLAS-4424
>
>
> Repository: atlas
>
>
> Description
> ---
>
> ATLAS-4424: Enhanced the Import hive utility to create export zip files and
> run bulk import
>
>
> Diffs
> -
>
> addons/hive-bridge/src/bin/import-hive.sh c3539377f
>
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
> c361ac69a
>
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeV2.java
> PRE-CREATION
> intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java
> 2c1870444
>
> intg/src/main/java/org/apache/atlas/model/migration/MigrationImportStatus.java
> 3430fdae8
>
> repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationStatusService.java
> a22c68790
>
> repository/src/main/java/org/apache/atlas/repository/migration/ZipFileMigrationImporter.java
> bfb114814
>
> repository/src/test/java/org/apache/atlas/repository/impexp/DataMigrationStatusServiceTest.java
> f1dc990e8
> webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java
> 135b94b36
>
>
> Diff: https://reviews.apache.org/r/73586/diff/6/
>
>
> Testing
> ---
>
> Manually tested the new zip file creation and bulk import to atlas
>
>
> Thanks,
>
> Sidharth Mishra
>
>
[jira] [Resolved] (ATLAS-4381) Release Atlas 2.2.0
[ https://issues.apache.org/jira/browse/ATLAS-4381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian resolved ATLAS-4381. --- Resolution: Fixed > Release Atlas 2.2.0 > --- > > Key: ATLAS-4381 > URL: https://issues.apache.org/jira/browse/ATLAS-4381 > Project: Atlas > Issue Type: Task >Reporter: Sidharth Kumar Mishra >Assignee: Sidharth Kumar Mishra >Priority: Major > > parent task to capture release tasks for Apache Atlas 2.2.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-3972) Apache atlas 2.1.0
[ https://issues.apache.org/jira/browse/ATLAS-3972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian resolved ATLAS-3972. --- Resolution: Fixed > Apache atlas 2.1.0 > -- > > Key: ATLAS-3972 > URL: https://issues.apache.org/jira/browse/ATLAS-3972 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Frédéric Comte >Priority: Major > > I am using keycloak integration for authentication. Atlas is behind a reverse > proxy (HAProxy) that doing SSL termination. HAProxy send the header > X-Forwarded-Proto : https but Atlas does't not create an URL with https > scheme when constructing a redirect uri -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 73595: ATLAS-4428 : Enhance Atlas hook messages to capture hook version & message source
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73595/#review223610 --- Ship it! Ship It! - Sarath Subramanian On Oct. 12, 2021, 11:20 a.m., chaitali wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73595/ > --- > > (Updated Oct. 12, 2021, 11:20 a.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, Pinal Shah, and > Sarath Subramanian. > > > Bugs: ATLAS-4428 > https://issues.apache.org/jira/browse/ATLAS-4428 > > > Repository: atlas > > > Description > --- > > Atlas hook messages doesn't specify the source information of the messages > coming i.e. which hook source - HiveServer2, HiveMetastore, Impala, Spark, > HBase, Kafka. > > Each hook message already captures, the msgSourceIP, msgCreatedBy and > msgCreationTime info. In addition to these, we should capture the msgSource > information as well. > > When messages come from different hook versions, Atlas server should be aware > of the version of different hook messages to process the message based on the > hook version. > > > Diffs > - > > > addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java > 8c09d3394 > > addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java > 0ab06f2c5 > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java > 3cc7b3bc6 > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveMetastoreHookImpl.java > 6a492c214 > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/ImpalaLineageHook.java > 10ae08f6d > > addons/sqoop-bridge/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java > 4785960a7 > > addons/storm-bridge/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java > 779c5cb20 > common/src/main/java/org/apache/atlas/repository/Constants.java aea0c1311 > > intg/src/main/java/org/apache/atlas/model/notification/AtlasNotificationBaseMessage.java > ff45d5713 > > intg/src/main/java/org/apache/atlas/model/notification/AtlasNotificationMessage.java > 5869910cb > intg/src/main/java/org/apache/atlas/model/notification/MessageSource.java > PRE-CREATION > intg/src/main/resources/atlas-buildinfo.properties PRE-CREATION > notification/src/main/java/org/apache/atlas/hook/AtlasHook.java 9162ac144 > > notification/src/main/java/org/apache/atlas/notification/AbstractNotification.java > c45a1da95 > > notification/src/main/java/org/apache/atlas/notification/NotificationInterface.java > 3d8d9cc0a > > notification/src/main/java/org/apache/atlas/notification/spool/AtlasFileSpool.java > 0c92c300e > notification/src/test/java/org/apache/atlas/hook/AtlasHookTest.java > 1ae7c278c > > notification/src/test/java/org/apache/atlas/notification/AbstractNotificationTest.java > 8078a6ceb > > notification/src/test/java/org/apache/atlas/notification/AtlasNotificationMessageTest.java > 91a195d4b > > notification/src/test/java/org/apache/atlas/notification/entity/EntityNotificationDeserializerTest.java > 13eafb6b1 > > notification/src/test/java/org/apache/atlas/notification/hook/HookNotificationDeserializerTest.java > d04817024 > > > Diff: https://reviews.apache.org/r/73595/diff/9/ > > > Testing > --- > > mvn clean install > mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > testcases running successfully > > > Thanks, > > chaitali > >
Re: Review Request 73641: ATLAS-4447: Update Composite indexes status
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73641/#review223602 --- Ship it! Ship It! - Sarath Subramanian On Oct. 11, 2021, 10:30 p.m., Pinal Shah wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73641/ > --- > > (Updated Oct. 11, 2021, 10:30 p.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, and Sarath > Subramanian. > > > Bugs: ATLAS-4447 > https://issues.apache.org/jira/browse/ATLAS-4447 > > > Repository: atlas > > > Description > --- > > If by any chance, commposite index are not enabled, this patch fixes the > composite index from REGISTERED state to ENABLED state > > > Diffs > - > > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphManagement.java > 7e3b2f440 > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphManagement.java > 1cc7f8b55 > intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 20f8f7343 > > repository/src/main/java/org/apache/atlas/repository/patches/AtlasPatchManager.java > d30971afc > > repository/src/main/java/org/apache/atlas/repository/patches/UpdateCompositeIndexStatusPatch.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/73641/diff/4/ > > > Testing > --- > > manually tested > > > Thanks, > > Pinal Shah > >
Re: Review Request 73641: Repair Composite indexes
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73641/#review223595
---
Fix it, then Ship it!
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphManagement.java
Lines 38 (patched)
<https://reviews.apache.org/r/73641/#comment312702>
nit: unused import; consider removing
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphManagement.java
Lines 324 (patched)
<https://reviews.apache.org/r/73641/#comment312703>
nit: add import statement for SchemaStatus and SchemaAction.
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphManagement.java
Lines 329 (patched)
<https://reviews.apache.org/r/73641/#comment312701>
Add more detail to log statement:
LOG.info("Index schema status updated for index: {}, from {} to {}.",
index.name(), SchemaStatus.REGISTERED, SchemaStatus.ENABLED);
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphManagement.java
Lines 332 (patched)
<https://reviews.apache.org/r/73641/#comment312700>
consider adding else if block to handle WARN logging for "INSTALLED" schema
status
repository/src/main/java/org/apache/atlas/repository/patches/RepairCompositeIndexPatch.java
Lines 27 (patched)
<https://reviews.apache.org/r/73641/#comment312704>
consider renaming class to "UpdateCompositeIndexStatusPatch", since this
JAVA patch deals with UPDATING the schema status of certain composite indexes.
- Sarath Subramanian
On Oct. 8, 2021, 9:05 a.m., Pinal Shah wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73641/
> ---
>
> (Updated Oct. 8, 2021, 9:05 a.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, and Sarath
> Subramanian.
>
>
> Bugs: ATLAS-4447
> https://issues.apache.org/jira/browse/ATLAS-4447
>
>
> Repository: atlas
>
>
> Description
> ---
>
> If by any chance, commposite index are not enabled, this patch fixes the
> composite index from REGISTERED state to ENABLED state
>
>
> Diffs
> -
>
>
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphManagement.java
> 7e3b2f440
>
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphManagement.java
> 1cc7f8b55
>
> repository/src/main/java/org/apache/atlas/repository/patches/AtlasPatchManager.java
> d30971afc
>
> repository/src/main/java/org/apache/atlas/repository/patches/RepairCompositeIndexPatch.java
> PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/73641/diff/1/
>
>
> Testing
> ---
>
> manually tested
>
>
> Thanks,
>
> Pinal Shah
>
>
[jira] [Updated] (ATLAS-4408) Dynamic handling of failure in updating index
[ https://issues.apache.org/jira/browse/ATLAS-4408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4408: -- Affects Version/s: 3.0.0 2.2.0 > Dynamic handling of failure in updating index > - > > Key: ATLAS-4408 > URL: https://issues.apache.org/jira/browse/ATLAS-4408 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: 3.0.0, 2.2.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Attachments: IndexRecovery.png, IndexRecovery_FunctionalFlow.png > > > *Index failure resilience:* dynamic handling of failure in updating index > (i.e. HBase commit succeeds but index commit fails). > In case of secondary persistence failure scenario, there will be > inconsistency with indexes for all the transactions failed at Solr. And to > repair that, the existing option is re-indexing all the data which is time > consuming as it involves indexing the entire database. > To recover such inconsistencies we can use the *transaction write-ahead log > option*. By enabling write-ahead log(tx.log-tx), JanusGraph maintains all the > transaction log data which can be used to recover indices in case of > failures. With this approach, it’s extra overhead to maintain the log data > for all transactions but with this approach we can guarantee the system is > more resilient and proactive. So advantages of this approach can nullify the > overhead of maintaining log data. > Design details as below. > # Start new service - IndexRecoveryService at Atlas startup. > ## Continuously monitor for Solr(Index Client) health for every retryTime > millisecs > ### If Solr is healthy and recovery start time is available, > Start Transaction Recovery with available recovery start time(which is > noted when Solr became unhealthy) > Persist current recovery time as previous which can be used later by > passing as custom recovery time to start index recovery if required. > Reset current recovery start time > Continue with Solr health checkup. > ### If Solr is unhealthy and no recovery start time is available, > Shutdown the existing transaction recovery process. > Note down the time which should be the next recovery start time and > persist in graph. > Continue with Solr health checkup. > Configuration properties to be used for this feature. > 1.To enable or disable index recovery(By default index recovery will be > enabled on Atlas startup) > *atlas.graph.enable.index.recovery=true* > 2.To configure how frequently SOLR health check should be done > *atlas.graph.index.search.solr.status.retry.interval=* > 3.To start index recovery by custom recovery time as user provided > *atlas.graph.index.search.solr.recovery.start.time=1630086622* > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4408) Dynamic handling of failure in updating index
[ https://issues.apache.org/jira/browse/ATLAS-4408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4408: -- Labels: indexing (was: ) > Dynamic handling of failure in updating index > - > > Key: ATLAS-4408 > URL: https://issues.apache.org/jira/browse/ATLAS-4408 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: 3.0.0, 2.2.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Labels: indexing > Fix For: 3.0.0, 2.3.0 > > Attachments: IndexRecovery.png, IndexRecovery_FunctionalFlow.png > > > *Index failure resilience:* dynamic handling of failure in updating index > (i.e. HBase commit succeeds but index commit fails). > In case of secondary persistence failure scenario, there will be > inconsistency with indexes for all the transactions failed at Solr. And to > repair that, the existing option is re-indexing all the data which is time > consuming as it involves indexing the entire database. > To recover such inconsistencies we can use the *transaction write-ahead log > option*. By enabling write-ahead log(tx.log-tx), JanusGraph maintains all the > transaction log data which can be used to recover indices in case of > failures. With this approach, it’s extra overhead to maintain the log data > for all transactions but with this approach we can guarantee the system is > more resilient and proactive. So advantages of this approach can nullify the > overhead of maintaining log data. > Design details as below. > # Start new service - IndexRecoveryService at Atlas startup. > ## Continuously monitor for Solr(Index Client) health for every retryTime > millisecs > ### If Solr is healthy and recovery start time is available, > Start Transaction Recovery with available recovery start time(which is > noted when Solr became unhealthy) > Persist current recovery time as previous which can be used later by > passing as custom recovery time to start index recovery if required. > Reset current recovery start time > Continue with Solr health checkup. > ### If Solr is unhealthy and no recovery start time is available, > Shutdown the existing transaction recovery process. > Note down the time which should be the next recovery start time and > persist in graph. > Continue with Solr health checkup. > Configuration properties to be used for this feature. > 1.To enable or disable index recovery(By default index recovery will be > enabled on Atlas startup) > *atlas.graph.enable.index.recovery=true* > 2.To configure how frequently SOLR health check should be done > *atlas.graph.index.search.solr.status.retry.interval=* > 3.To start index recovery by custom recovery time as user provided > *atlas.graph.index.search.solr.recovery.start.time=1630086622* > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4408) Dynamic handling of failure in updating index
[ https://issues.apache.org/jira/browse/ATLAS-4408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4408: -- Fix Version/s: 2.3.0 3.0.0 > Dynamic handling of failure in updating index > - > > Key: ATLAS-4408 > URL: https://issues.apache.org/jira/browse/ATLAS-4408 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: 3.0.0, 2.2.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: IndexRecovery.png, IndexRecovery_FunctionalFlow.png > > > *Index failure resilience:* dynamic handling of failure in updating index > (i.e. HBase commit succeeds but index commit fails). > In case of secondary persistence failure scenario, there will be > inconsistency with indexes for all the transactions failed at Solr. And to > repair that, the existing option is re-indexing all the data which is time > consuming as it involves indexing the entire database. > To recover such inconsistencies we can use the *transaction write-ahead log > option*. By enabling write-ahead log(tx.log-tx), JanusGraph maintains all the > transaction log data which can be used to recover indices in case of > failures. With this approach, it’s extra overhead to maintain the log data > for all transactions but with this approach we can guarantee the system is > more resilient and proactive. So advantages of this approach can nullify the > overhead of maintaining log data. > Design details as below. > # Start new service - IndexRecoveryService at Atlas startup. > ## Continuously monitor for Solr(Index Client) health for every retryTime > millisecs > ### If Solr is healthy and recovery start time is available, > Start Transaction Recovery with available recovery start time(which is > noted when Solr became unhealthy) > Persist current recovery time as previous which can be used later by > passing as custom recovery time to start index recovery if required. > Reset current recovery start time > Continue with Solr health checkup. > ### If Solr is unhealthy and no recovery start time is available, > Shutdown the existing transaction recovery process. > Note down the time which should be the next recovery start time and > persist in graph. > Continue with Solr health checkup. > Configuration properties to be used for this feature. > 1.To enable or disable index recovery(By default index recovery will be > enabled on Atlas startup) > *atlas.graph.enable.index.recovery=true* > 2.To configure how frequently SOLR health check should be done > *atlas.graph.index.search.solr.status.retry.interval=* > 3.To start index recovery by custom recovery time as user provided > *atlas.graph.index.search.solr.recovery.start.time=1630086622* > -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 73589: Indexing Monitor and Recovery
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73589/#review223594 --- Ship it! Ship It! - Sarath Subramanian On Sept. 21, 2021, 8:40 a.m., Ashutosh Mestry wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73589/ > --- > > (Updated Sept. 21, 2021, 8:40 a.m.) > > > Review request for atlas, Nikhil Bonte, Nixon Rodrigues, Radhika Kundam, and > Sarath Subramanian. > > > Bugs: ATLAS-4408 > https://issues.apache.org/jira/browse/ATLAS-4408 > > > Repository: atlas > > > Description > --- > > This implementation builds up on Radhika's earlier implementation. > https://reviews.apache.org/r/73555/ > > Indexing > > atlas.graph.index.status.check.frequency: Frequence of checking Solr for > health status. > atlas.graph.index.recovery.start.time: User can set recovery time in UTC. > E.g. > ``` > atlas.graph.index.recovery.start.time=2014-12-11T02:35:58.440Z > ``` > > **Implementation** > New: _updateGlobalConfiguration_: Updating global configuration variables. > > > Diffs > - > > common/src/main/java/org/apache/atlas/repository/Constants.java aea0c1311 > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphIndexClient.java > 9960d8909 > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphManagement.java > 7e3b2f440 > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphDatabase.java > 11267c438 > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphIndexClient.java > ef42dbd3d > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphManagement.java > 1cc7f8b55 > intg/src/main/java/org/apache/atlas/ApplicationProperties.java 682206dad > > repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java > ddfb0089e > > repository/src/main/java/org/apache/atlas/repository/graph/IndexRecoveryService.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/graph/RecoveryInfoManagementTest.java > PRE-CREATION > > server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java > ba8f0886c > > > Diff: https://reviews.apache.org/r/73589/diff/2/ > > > Testing > --- > > Additional unit tests. > > > Thanks, > > Ashutosh Mestry > >
Re: Review Request 73603: ATLAS-4436 Basic Search : Search By Query '*' gets internal entities also
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73603/#review223591 --- Ship it! Ship It! - Sarath Subramanian On Sept. 27, 2021, 11:02 p.m., Shraddha Pardeshi wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73603/ > --- > > (Updated Sept. 27, 2021, 11:02 p.m.) > > > Review request for atlas, Jayendra Parab, Mandar Ambawane, and Pinal Shah. > > > Bugs: ATLAS-4436 > https://issues.apache.org/jira/browse/ATLAS-4436 > > > Repository: atlas > > > Description > --- > > when we add "*" in search by query in response it brings all the internal > entities like '__AtlasAuditEntry'. But expected behaviour is, it should not > show internal types. > So here filtering those internal types in basic search > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/discovery/FreeTextSearchProcessor.java > 86f2cea23 > > repository/src/main/java/org/apache/atlas/discovery/FullTextSearchProcessor.java > 2d8a448b1 > > repository/src/test/java/org/apache/atlas/discovery/FreeTextSearchProcessorTest.java > e497bb786 > > > Diff: https://reviews.apache.org/r/73603/diff/3/ > > > Testing > --- > > Local testing done > > > Thanks, > > Shraddha Pardeshi > >
Re: Review Request 73585: ATLAS-4425: Migration import should be able to import multiple zip files present in a particular path
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73585/#review223590 --- Fix it, then Ship it! repository/src/main/java/org/apache/atlas/repository/migration/ZipFileMigrationImporter.java Lines 71 (patched) <https://reviews.apache.org/r/73585/#comment312694> add a comment about fileName can support wildcards. fileNameMightBeWithWildcard => fileName repository/src/main/java/org/apache/atlas/repository/migration/ZipFileMigrationImporter.java Lines 74 (patched) <https://reviews.apache.org/r/73585/#comment312698> consider moving this method to create archive directory inside getAllFilesToImport(); The parent File objects are already constructed in getAllFilesToImport(). repository/src/main/java/org/apache/atlas/repository/migration/ZipFileMigrationImporter.java Lines 112 (patched) <https://reviews.apache.org/r/73585/#comment312695> Move line 109 and 112 inside 'if' block (after line 115). Construct this object only after checking if it is a valid directory. also move line 117 inside 'if' block (after line 119) repository/src/main/java/org/apache/atlas/repository/migration/ZipFileMigrationImporter.java Lines 122 (patched) <https://reviews.apache.org/r/73585/#comment312696> eachFile => importFile repository/src/main/java/org/apache/atlas/repository/migration/ZipFileMigrationImporter.java Lines 123 (patched) <https://reviews.apache.org/r/73585/#comment312697> extract to method to check - isValidImportFile(importFile) repository/src/main/java/org/apache/atlas/repository/migration/ZipFileMigrationImporter.java Lines 162 (patched) <https://reviews.apache.org/r/73585/#comment312699> check if you have write/move permission on the destination archive dir as well (newFile) - Sarath Subramanian On Oct. 5, 2021, 4:30 p.m., Sidharth Mishra wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73585/ > --- > > (Updated Oct. 5, 2021, 4:30 p.m.) > > > Review request for atlas, Ashutosh Mestry, Disha Talreja, Radhika Kundam, and > Sarath Subramanian. > > > Bugs: ATLAS-4425 > https://issues.apache.org/jira/browse/ATLAS-4425 > > > Repository: atlas > > > Description > --- > > ATLAS-442: Added support for Migration import to run on a directory with > multip[le zip files > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/repository/impexp/ZipSourceDirect.java > 04342fa52 > > repository/src/main/java/org/apache/atlas/repository/migration/ZipFileMigrationImporter.java > d56261f78 > > > Diff: https://reviews.apache.org/r/73585/diff/3/ > > > Testing > --- > > Manually tested with multiple zip files at a location which was imported > successfully. > > > Thanks, > > Sidharth Mishra > >
[jira] [Commented] (ATLAS-4246) Make Kafka Interface aware of Kafka Schema Registry
[ https://issues.apache.org/jira/browse/ATLAS-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425734#comment-17425734 ] Sarath Subramanian commented on ATLAS-4246: --- [~aileeen], added some review comments. Please review/update. > Make Kafka Interface aware of Kafka Schema Registry > --- > > Key: ATLAS-4246 > URL: https://issues.apache.org/jira/browse/ATLAS-4246 > Project: Atlas > Issue Type: Improvement > Components: kafka-integration >Affects Versions: 2.1.0, 3.0.0 >Reporter: Aileen Toleikis >Assignee: Viktor Somogyi-Vass >Priority: Major > Labels: Kafka > Fix For: 3.0.0, 2.3.0 > > > Kafka Community is using Schema Registry more and more heavily but as Atlas > is currently unaware of this, this extension helps Atlas make use of the > Schemas. > > We have tested this extension and we have production environments where Atlas > will not be allowed without schema registry access. We have received feedback > that this extension would be sufficient to allow production use. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 73286: Initial commit
ache/atlas/kafka/bridge/SchemaRegistryConnector.java
Lines 65 (patched)
<https://reviews.apache.org/r/73286/#comment312674>
LOG.error should be added here.
Consider print error stack within logger message and avoid line 63.
LOG.error("..", e)
addons/kafka-bridge/src/main/java/org/apache/atlas/kafka/bridge/SchemaRegistryConnector.java
Lines 69 (patched)
<https://reviews.apache.org/r/73286/#comment312677>
replace 404 with HttpStatus.SC_NOT_FOUND
addons/kafka-bridge/src/main/java/org/apache/atlas/kafka/bridge/SchemaRegistryConnector.java
Lines 72 (patched)
<https://reviews.apache.org/r/73286/#comment312678>
"Can not find versions to schema: {} in Kafka" => "No schema versions found
for schema: {} in Schema Registry"
addons/kafka-bridge/src/main/java/org/apache/atlas/kafka/bridge/SchemaRegistryConnector.java
Lines 102 (patched)
<https://reviews.apache.org/r/73286/#comment312679>
create static constant for "schema"
addons/kafka-bridge/src/main/java/org/apache/atlas/kafka/bridge/SchemaRegistryConnector.java
Lines 104 (patched)
<https://reviews.apache.org/r/73286/#comment312680>
avoid printing stacktrace here and print in log:
LOG.error ("", e)
addons/kafka-bridge/src/main/java/org/apache/atlas/kafka/bridge/SchemaRegistryConnector.java
Lines 113 (patched)
<https://reviews.apache.org/r/73286/#comment312682>
"Can not" -> "Cannot"
- Sarath Subramanian
On Oct. 5, 2021, 1:06 a.m., Aileen Toleikis wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73286/
> ---
>
> (Updated Oct. 5, 2021, 1:06 a.m.)
>
>
> Review request for atlas.
>
>
> Repository: atlas
>
>
> Description
> ---
>
> The Kafka Community is using Schema Registry more and more heavily but as
> Atlas is currently unaware of this, this extension helps Atlas make use of
> the Schemas.
>
> We have tested this extension and we have production environments where Atlas
> will not be allowed without schema registry access. We have received feedback
> that this extension would be sufficient to allow production use.
>
> Changes:
>
> initial commit
>
>
> added schema creation in Atlas
>
>
> added Schema creation, Schema linkage and started to work on versions
>
>
> avro_field creation is working. However, there is no connection to schemas yet
>
>
> smaller refactorings and improved logging
>
>
> added field reference in schemas
>
>
> deleted second README and added gitignore
>
>
> Smaller changes after rebasing on new version
>
>
> moved HTTP REST calls to a separate class (SchemaRegistryConnector.java)
>
>
> added testCreateSchema() as test case
>
>
> added testCreateField() and testUpdateField() as test case
>
>
> added testGetSchemas() and testGetSchemaVersions() as test case
>
>
> added testUpdateSchema() as test case
>
>
> removed unnecessary comments
>
>
> Diffs
> -
>
> addons/kafka-bridge/pom.xml 2ac19fd20e0c0322b0d104641ddd5a2ef89bf9d0
>
> addons/kafka-bridge/src/main/java/org/apache/atlas/kafka/bridge/KafkaBridge.java
> f9548244364a8f79f346411739348d0d53298c99
>
> addons/kafka-bridge/src/main/java/org/apache/atlas/kafka/bridge/SchemaRegistryConnector.java
> PRE-CREATION
>
> addons/kafka-bridge/src/main/java/org/apache/atlas/kafka/model/KafkaDataTypes.java
> 0f81b4c37a1d8f84c587a914eb460b290ae36987
>
> addons/kafka-bridge/src/test/java/org/apache/atlas/kafka/bridge/KafkaBridgeTest.java
> f86ceb58fd250d80f3ae3c71085c8c0adbc9116b
> pom.xml 47768e9fa7d8abcfc81cd148917400ac478a333a
>
>
> Diff: https://reviews.apache.org/r/73286/diff/3/
>
>
> Testing
> ---
>
>
> Thanks,
>
> Aileen Toleikis
>
>
Re: Review Request 73595: ATLAS-4428 : Enhance Atlas hook messages to capture hook version & message source
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73595/#review223579 --- Fix it, then Ship it! Add tests for cross compatability. If older Atlas server can deserialize the new message structure added by new hook and the other way (older hook sending message to new Atlas server). addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java Line 56 (original) <https://reviews.apache.org/r/73595/#comment312658> nit: revert new line change addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java Line 259 (original) <https://reviews.apache.org/r/73595/#comment312659> nit: revert new line change intg/src/main/java/org/apache/atlas/model/notification/MessageSource.java Lines 62 (patched) <https://reviews.apache.org/r/73595/#comment312660> move private methods after public methods - Sarath Subramanian On Oct. 6, 2021, 8:13 a.m., chaitali wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73595/ > --- > > (Updated Oct. 6, 2021, 8:13 a.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, Pinal Shah, and > Sarath Subramanian. > > > Bugs: ATLAS-4428 > https://issues.apache.org/jira/browse/ATLAS-4428 > > > Repository: atlas > > > Description > --- > > Atlas hook messages doesn't specify the source information of the messages > coming i.e. which hook source - HiveServer2, HiveMetastore, Impala, Spark, > HBase, Kafka. > > Each hook message already captures, the msgSourceIP, msgCreatedBy and > msgCreationTime info. In addition to these, we should capture the msgSource > information as well. > > When messages come from different hook versions, Atlas server should be aware > of the version of different hook messages to process the message based on the > hook version. > > > Diffs > - > > > addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java > 8c09d3394 > > addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java > 0ab06f2c5 > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java > 3cc7b3bc6 > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveMetastoreHookImpl.java > 6a492c214 > > addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/ImpalaLineageHook.java > 10ae08f6d > > addons/sqoop-bridge/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java > 4785960a7 > > addons/storm-bridge/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java > 779c5cb20 > common/src/main/java/org/apache/atlas/repository/Constants.java aea0c1311 > > intg/src/main/java/org/apache/atlas/model/notification/AtlasNotificationBaseMessage.java > ff45d5713 > > intg/src/main/java/org/apache/atlas/model/notification/AtlasNotificationMessage.java > 5869910cb > intg/src/main/java/org/apache/atlas/model/notification/MessageSource.java > PRE-CREATION > intg/src/main/resources/atlas-buildinfo.properties PRE-CREATION > notification/src/main/java/org/apache/atlas/hook/AtlasHook.java 9162ac144 > > notification/src/main/java/org/apache/atlas/notification/AbstractNotification.java > c45a1da95 > > notification/src/main/java/org/apache/atlas/notification/NotificationInterface.java > 3d8d9cc0a > > notification/src/main/java/org/apache/atlas/notification/spool/AtlasFileSpool.java > 0c92c300e > notification/src/test/java/org/apache/atlas/hook/AtlasHookTest.java > 1ae7c278c > > notification/src/test/java/org/apache/atlas/notification/AbstractNotificationTest.java > 8078a6ceb > > notification/src/test/java/org/apache/atlas/notification/entity/EntityNotificationDeserializerTest.java > 13eafb6b1 > > notification/src/test/java/org/apache/atlas/notification/hook/HookNotificationDeserializerTest.java > d04817024 > > > Diff: https://reviews.apache.org/r/73595/diff/7/ > > > Testing > --- > > mvn clean install > mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > testcases running successfully > > > Thanks, > > chaitali > >
[jira] [Commented] (ATLAS-4440) Upgrade Atlas's Kafka dependency to 2.8
[ https://issues.apache.org/jira/browse/ATLAS-4440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17424224#comment-17424224 ] Sarath Subramanian commented on ATLAS-4440: --- [~viktorsomogyi], is this Kafka version backward compatible? Is there any compatibility issue when older Kafka client (hook) communicates with newer Kafka version or vice-versa? > Upgrade Atlas's Kafka dependency to 2.8 > --- > > Key: ATLAS-4440 > URL: https://issues.apache.org/jira/browse/ATLAS-4440 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Viktor Somogyi-Vass >Assignee: Viktor Somogyi-Vass >Priority: Major > Attachments: 0001-ATLAS-4440-Update-Kafka-dependency-to-2.8.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4440) Upgrade Atlas's Kafka dependency to 2.8
[ https://issues.apache.org/jira/browse/ATLAS-4440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17424223#comment-17424223 ] Sarath Subramanian commented on ATLAS-4440: --- Thanks for the Precommit run [~amestry]. +1 > Upgrade Atlas's Kafka dependency to 2.8 > --- > > Key: ATLAS-4440 > URL: https://issues.apache.org/jira/browse/ATLAS-4440 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Viktor Somogyi-Vass >Assignee: Viktor Somogyi-Vass >Priority: Major > Attachments: 0001-ATLAS-4440-Update-Kafka-dependency-to-2.8.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 73595: ATLAS-4428 : Enhance Atlas hook messages to capture hook version & message source
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73595/#review223553
---
addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java
Lines 49 (patched)
<https://reviews.apache.org/r/73595/#comment312624>
instead of returning the implementing class name as message source,
consider returning a static constant:
SQOOP_SOURCE="sqoop"
FALCON_SOURCE="falcon"
HBASE_SOURCE="hbase"
HS2_SOURCE="hive_server2"
HMS_SOURCE="hive_metastore"
IMPALA_SOURCE="impala"
STORM_SOURCE="storm"
Consider defining this constant in common module -
org.apache.atlas.repository.Constants, we could reuse this at server side for
additional actions based on message source.
`return Constants.FALCON_SOURCE;`
intg/src/main/java/org/apache/atlas/model/notification/MessageSource.java
Lines 72 (patched)
<https://reviews.apache.org/r/73595/#comment312625>
getHookVersion() method initializes 'hookVersion' variable, usage of
getters here is confusing:
public MessageSource(String messageSource) {
this.messageSource = messageSource;
this.hookVersion = properties.getProperty("build.version", "UNKNOWN");
}
Also, can't we initialize hookVersion in the static block in line #52?
- Sarath Subramanian
On Sept. 29, 2021, 12:30 a.m., chaitali wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73595/
> ---
>
> (Updated Sept. 29, 2021, 12:30 a.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, Pinal Shah, and
> Sarath Subramanian.
>
>
> Bugs: ATLAS-4428
> https://issues.apache.org/jira/browse/ATLAS-4428
>
>
> Repository: atlas
>
>
> Description
> ---
>
> Atlas hook messages doesn't specify the source information of the messages
> coming i.e. which hook source - HiveServer2, HiveMetastore, Impala, Spark,
> HBase, Kafka.
>
> Each hook message already captures, the msgSourceIP, msgCreatedBy and
> msgCreationTime info. In addition to these, we should capture the msgSource
> information as well.
>
> When messages come from different hook versions, Atlas server should be aware
> of the version of different hook messages to process the message based on the
> hook version.
>
>
> Diffs
> -
>
>
> addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java
> 8c09d3394
>
> addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java
> 0ab06f2c5
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java
> 3cc7b3bc6
>
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveMetastoreHookImpl.java
> 6a492c214
>
> addons/impala-bridge/src/main/java/org/apache/atlas/impala/hook/ImpalaLineageHook.java
> 10ae08f6d
>
> addons/sqoop-bridge/src/main/java/org/apache/atlas/sqoop/hook/SqoopHook.java
> 4785960a7
>
> addons/storm-bridge/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java
> 779c5cb20
> intg/pom.xml 05914
>
> intg/src/main/java/org/apache/atlas/model/notification/AtlasNotificationBaseMessage.java
> ff45d5713
>
> intg/src/main/java/org/apache/atlas/model/notification/AtlasNotificationMessage.java
> 5869910cb
> intg/src/main/java/org/apache/atlas/model/notification/MessageSource.java
> PRE-CREATION
> notification/src/main/java/org/apache/atlas/hook/AtlasHook.java 9162ac144
>
> notification/src/main/java/org/apache/atlas/notification/AbstractNotification.java
> c45a1da95
>
> notification/src/main/java/org/apache/atlas/notification/NotificationInterface.java
> 3d8d9cc0a
>
> notification/src/main/java/org/apache/atlas/notification/spool/AtlasFileSpool.java
> 0c92c300e
> notification/src/test/java/org/apache/atlas/hook/AtlasHookTest.java
> 1ae7c278c
>
> notification/src/test/java/org/apache/atlas/notification/AbstractNotificationTest.java
> 8078a6ceb
>
> notification/src/test/java/org/apache/atlas/notification/entity/EntityNotificationDeserializerTest.java
> 13eafb6b1
>
> notification/src/test/java/org/apache/atlas/notification/hook/HookNotificationDeserializerTest.java
> d04817024
>
>
> Diff: https://reviews.apache.org/r/73595/diff/3/
>
>
> Testing
> ---
>
> mvn clean install
> mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip
> testcases running successfully
>
>
> Thanks,
>
> chaitali
>
>
[jira] [Updated] (ATLAS-4351) Maven-jetty throwing warnings when running Integration tests for conflicting jars in classpath
[ https://issues.apache.org/jira/browse/ATLAS-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4351: -- Component/s: atlas-core > Maven-jetty throwing warnings when running Integration tests for conflicting > jars in classpath > --- > > Key: ATLAS-4351 > URL: https://issues.apache.org/jira/browse/ATLAS-4351 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 3.0.0, 2.2.0 >Reporter: chaitali borole >Assignee: chaitali borole >Priority: Major > Fix For: 3.0.0 > > > Below warnings are thrown when integration tests executed : > [INFO] jetty-9.4.31.v20200723; built: 2020-07-23T17:57:36.812Z; git: > 450ba27947e13e66baa8cd1ce7e85a4461cacc1d; jvm 1.8.0_291-b10 > [WARNING] javax.activation.ActivationDataFlavor scanned from multiple > locations: jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/jakarta.activation-api-1.2.1.jar!/javax/activation/ActivationDataFlavor.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/javax.activation-api-1.2.0.jar!/javax/activation/ActivationDataFlavor.class] > [WARNING] com.google.common.base.Stopwatch$1 scanned from multiple locations: > jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/guava-25.1-jre.jar!/com/google/common/base/Stopwatch$1.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/janusgraph-hbase-0.5.3.jar!/com/google/common/base/Stopwatch$1.class] > [WARNING] javax.servlet.AsyncContext scanned from multiple locations: jar: > [file:///home/jenkins/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncContext.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncContext.class] > [WARNING] javax.servlet.AsyncEvent scanned from multiple locations: jar: > [file:///home/jenkins/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncEvent.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncEvent.class] > [WARNING] javax.servlet.AsyncListener scanned from multiple locations: jar: > [file:///home/jenkins/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncListener.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncListener.class] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4351) Maven-jetty throwing warnings when running Integration tests for conflicting jars in classpath
[ https://issues.apache.org/jira/browse/ATLAS-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4351: -- Affects Version/s: 2.2.0 > Maven-jetty throwing warnings when running Integration tests for conflicting > jars in classpath > --- > > Key: ATLAS-4351 > URL: https://issues.apache.org/jira/browse/ATLAS-4351 > Project: Atlas > Issue Type: Improvement >Affects Versions: 3.0.0, 2.2.0 >Reporter: chaitali borole >Assignee: chaitali borole >Priority: Major > Fix For: 3.0.0 > > > Below warnings are thrown when integration tests executed : > [INFO] jetty-9.4.31.v20200723; built: 2020-07-23T17:57:36.812Z; git: > 450ba27947e13e66baa8cd1ce7e85a4461cacc1d; jvm 1.8.0_291-b10 > [WARNING] javax.activation.ActivationDataFlavor scanned from multiple > locations: jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/jakarta.activation-api-1.2.1.jar!/javax/activation/ActivationDataFlavor.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/javax.activation-api-1.2.0.jar!/javax/activation/ActivationDataFlavor.class] > [WARNING] com.google.common.base.Stopwatch$1 scanned from multiple locations: > jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/guava-25.1-jre.jar!/com/google/common/base/Stopwatch$1.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/janusgraph-hbase-0.5.3.jar!/com/google/common/base/Stopwatch$1.class] > [WARNING] javax.servlet.AsyncContext scanned from multiple locations: jar: > [file:///home/jenkins/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncContext.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncContext.class] > [WARNING] javax.servlet.AsyncEvent scanned from multiple locations: jar: > [file:///home/jenkins/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncEvent.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncEvent.class] > [WARNING] javax.servlet.AsyncListener scanned from multiple locations: jar: > [file:///home/jenkins/.m2/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncListener.class] > , jar: > [file:///home/jenkins/jenkins-agent/workspace/Atlas/PreCommit-ATLAS-Build-Test/webapp/target/atlas-webapp-3.0.0-SNAPSHOT/WEB-INF/lib/javax.servlet-api-3.1.0.jar!/javax/servlet/AsyncListener.class] -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 73448: ATLAS-4351 : Maven-jetty throwing warnings when running Integration tests for conflicting jars in classpath
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73448/#review223552 --- Ship it! Ship It! - Sarath Subramanian On Sept. 29, 2021, 11:33 p.m., chaitali wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73448/ > --- > > (Updated Sept. 29, 2021, 11:33 p.m.) > > > Review request for atlas, Jayendra Parab, Nixon Rodrigues, and Sarath > Subramanian. > > > Bugs: ATLAS-4351 > https://issues.apache.org/jira/browse/ATLAS-4351 > > > Repository: atlas > > > Description > --- > > When we run integration tests maven jetty plugin throws warnings for > conflicting jars in classpath > > > Diffs > - > > addons/falcon-bridge/pom.xml de22fa5a0 > addons/hbase-bridge/pom.xml 50fb9e838 > addons/hbase-testing-util/pom.xml 982e9c85e > addons/hive-bridge/pom.xml ea2de4d3b > addons/impala-bridge/pom.xml a1bd5920b > addons/kafka-bridge/pom.xml 7fe97eb39 > addons/sqoop-bridge/pom.xml 9d5ac52d9 > addons/storm-bridge-shim/pom.xml f4e75927f > addons/storm-bridge/pom.xml 71525438b > common/pom.xml 616f66c53 > graphdb/janus-hbase2/pom.xml 92a7104b9 > graphdb/janus/pom.xml 8f1ed543a > intg/pom.xml 05914 > notification/pom.xml 28d13bc13 > pom.xml 083dc4632 > repository/pom.xml 1607d9aed > test-tools/pom.xml 991fa14c9 > webapp/pom.xml 81bbb89fb > > > Diff: https://reviews.apache.org/r/73448/diff/3/ > > > Testing > --- > > Latest PC > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/754/consoleFull > latestPC > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/845/console > mvn clean install -DskipTests -Drat.skip > mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > > > Thanks, > > chaitali > >
Re: Review Request 73448: ATLAS-4351 : Maven-jetty throwing warnings when running Integration tests for conflicting jars in classpath
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73448/#review223549 --- Ship it! Ship It! - Sarath Subramanian On Sept. 13, 2021, 10:58 p.m., chaitali wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73448/ > --- > > (Updated Sept. 13, 2021, 10:58 p.m.) > > > Review request for atlas, Jayendra Parab, Nixon Rodrigues, and Sarath > Subramanian. > > > Bugs: ATLAS-4351 > https://issues.apache.org/jira/browse/ATLAS-4351 > > > Repository: atlas > > > Description > --- > > When we run integration tests maven jetty plugin throws warnings for > conflicting jars in classpath > > > Diffs > - > > addons/falcon-bridge/pom.xml e0d2f3be7 > addons/hbase-bridge/pom.xml 50fb9e838 > addons/hbase-testing-util/pom.xml 982e9c85e > addons/hive-bridge/pom.xml ea2de4d3b > addons/impala-bridge/pom.xml a1bd5920b > addons/kafka-bridge/pom.xml 7fe97eb39 > addons/sqoop-bridge/pom.xml 9d5ac52d9 > addons/storm-bridge-shim/pom.xml f4e75927f > addons/storm-bridge/pom.xml 71525438b > common/pom.xml 616f66c53 > graphdb/janus-hbase2/pom.xml 92a7104b9 > graphdb/janus/pom.xml 8f1ed543a > intg/pom.xml 05914 > notification/pom.xml 28d13bc13 > pom.xml 19db43146 > repository/pom.xml bf34efee2 > test-tools/pom.xml c861ea67f > webapp/pom.xml ccbcc3141 > > > Diff: https://reviews.apache.org/r/73448/diff/2/ > > > Testing > --- > > Latest PC > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/754/consoleFull > latestPC > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/845/console > mvn clean install -DskipTests -Drat.skip > mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > > > Thanks, > > chaitali > >
[jira] [Updated] (ATLAS-4421) [Atlas: Hive Import] When import-hive is run with incorrect input the information is not conveyed to the user
[
https://issues.apache.org/jira/browse/ATLAS-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4421:
--
Component/s: atlas-core
> [Atlas: Hive Import] When import-hive is run with incorrect input the
> information is not conveyed to the user
> -
>
> Key: ATLAS-4421
> URL: https://issues.apache.org/jira/browse/ATLAS-4421
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.2.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Disha Talreja
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: ATLAS-4421.patch
>
>
> {code:java}
> [root@quasar-cxwzxp-3 bin]#
> /opt/cloudera/parcels/CDH/lib/atlas/hook-bin/import-hive.sh -f /tmp/file2.txt
> ...
> Log file for import is /var/log/atlas/import-hive.log
> ...
> Hive Meta Data imported successfully!!! {code}
> In the above example, file */tmp/file2.txt* does not exists.
> The log file is not created in the expected location
> /var/log/atlas/import-hive.log which is tracked by Jira**
> Here, this gives the user an impression that the import is success while
> nothing has actually happened.
> It would be good to convey that no data was imported in such cases
> This is true even when the database name/pattern provided to -d or table
> name/pattern provided to to -t are incorrect
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4421) [Atlas: Hive Import] When import-hive is run with incorrect input the information is not conveyed to the user
[
https://issues.apache.org/jira/browse/ATLAS-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4421:
--
Labels: hive (was: )
> [Atlas: Hive Import] When import-hive is run with incorrect input the
> information is not conveyed to the user
> -
>
> Key: ATLAS-4421
> URL: https://issues.apache.org/jira/browse/ATLAS-4421
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 2.2.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Disha Talreja
>Priority: Major
> Labels: hive
> Fix For: 3.0.0, 2.3.0
>
> Attachments: ATLAS-4421.patch
>
>
> {code:java}
> [root@quasar-cxwzxp-3 bin]#
> /opt/cloudera/parcels/CDH/lib/atlas/hook-bin/import-hive.sh -f /tmp/file2.txt
> ...
> Log file for import is /var/log/atlas/import-hive.log
> ...
> Hive Meta Data imported successfully!!! {code}
> In the above example, file */tmp/file2.txt* does not exists.
> The log file is not created in the expected location
> /var/log/atlas/import-hive.log which is tracked by Jira**
> Here, this gives the user an impression that the import is success while
> nothing has actually happened.
> It would be good to convey that no data was imported in such cases
> This is true even when the database name/pattern provided to -d or table
> name/pattern provided to to -t are incorrect
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4421) [Atlas: Hive Import] When import-hive is run with incorrect input the information is not conveyed to the user
[
https://issues.apache.org/jira/browse/ATLAS-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4421:
--
Fix Version/s: 2.3.0
3.0.0
> [Atlas: Hive Import] When import-hive is run with incorrect input the
> information is not conveyed to the user
> -
>
> Key: ATLAS-4421
> URL: https://issues.apache.org/jira/browse/ATLAS-4421
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Disha Talreja
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: ATLAS-4421.patch
>
>
> {code:java}
> [root@quasar-cxwzxp-3 bin]#
> /opt/cloudera/parcels/CDH/lib/atlas/hook-bin/import-hive.sh -f /tmp/file2.txt
> ...
> Log file for import is /var/log/atlas/import-hive.log
> ...
> Hive Meta Data imported successfully!!! {code}
> In the above example, file */tmp/file2.txt* does not exists.
> The log file is not created in the expected location
> /var/log/atlas/import-hive.log which is tracked by Jira**
> Here, this gives the user an impression that the import is success while
> nothing has actually happened.
> It would be good to convey that no data was imported in such cases
> This is true even when the database name/pattern provided to -d or table
> name/pattern provided to to -t are incorrect
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4421) [Atlas: Hive Import] When import-hive is run with incorrect input the information is not conveyed to the user
[
https://issues.apache.org/jira/browse/ATLAS-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4421:
--
Affects Version/s: 2.2.0
> [Atlas: Hive Import] When import-hive is run with incorrect input the
> information is not conveyed to the user
> -
>
> Key: ATLAS-4421
> URL: https://issues.apache.org/jira/browse/ATLAS-4421
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Disha Talreja
>Priority: Major
> Attachments: ATLAS-4421.patch
>
>
> {code:java}
> [root@quasar-cxwzxp-3 bin]#
> /opt/cloudera/parcels/CDH/lib/atlas/hook-bin/import-hive.sh -f /tmp/file2.txt
> ...
> Log file for import is /var/log/atlas/import-hive.log
> ...
> Hive Meta Data imported successfully!!! {code}
> In the above example, file */tmp/file2.txt* does not exists.
> The log file is not created in the expected location
> /var/log/atlas/import-hive.log which is tracked by Jira**
> Here, this gives the user an impression that the import is success while
> nothing has actually happened.
> It would be good to convey that no data was imported in such cases
> This is true even when the database name/pattern provided to -d or table
> name/pattern provided to to -t are incorrect
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Review Request 73616: ATLAS-4421: [Atlas: Hive Import] When import-hive is run with incorrect input the information is not conveyed to the user
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73616/#review223541 --- Fix it, then Ship it! please add UT/IT test coverage. addons/hive-bridge/src/bin/import-hive.sh Line 159 (original), 159 (patched) <https://reviews.apache.org/r/73616/#comment312609> "Meta Data" => "metadata" - Sarath Subramanian On Sept. 27, 2021, 10:29 p.m., Disha Talreja wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73616/ > --- > > (Updated Sept. 27, 2021, 10:29 p.m.) > > > Review request for atlas, Ashutosh Mestry and Sarath Subramanian. > > > Repository: atlas > > > Description > --- > > Even if a non-existent file name/ database name/ table name is passed to the > import-hive script, hive meta data gets imported successfully. This gives the > user an impression that the import is successful while nothing has actually > happened. With this patch, the program gets terminated for any incorrect > argument passed and the issue is logged in the log file. > > > Diffs > - > > addons/hive-bridge/src/bin/import-hive.sh fde486f28 > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java > 08471b480 > > > Diff: https://reviews.apache.org/r/73616/diff/1/ > > > Testing > --- > > Patched in a cluster. > > > Thanks, > > Disha Talreja > >
Re: Review Request 73606: AtlasPatchManager: Avoid Race Condition in HA Mode
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73606/#review223540 --- Ship it! Spring Bean initialization is not guaranteed, even though order is specified for the following spring components: * GraphBackedSearchIndexer (Order-1) * AtlasPatchService (Order-3) AtlasPatchService need the component 'AtlasPatchManager' to be injected and is therefore initialized before the indexes are created in GraphBackedSearchIndexer. This patch will address 2 issues: 1. Avoid full table scan in getAllPatches() in AtlasPatchRegistry which happens when query is fired before indexes are created for the first time. 2. Avoid post-construct of PatchContext initalization, which caused individual patch queries to be fired during initialization (before indexes got created) -> moved to applyAll() method. Good fix! - Sarath Subramanian On Sept. 24, 2021, 11:14 a.m., Ashutosh Mestry wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73606/ > --- > > (Updated Sept. 24, 2021, 11:14 a.m.) > > > Review request for atlas, Jayendra Parab, Mandar Ambawane, Nikhil Bonte, > Nixon Rodrigues, and Sarath Subramanian. > > > Bugs: ATLAS-4358 > https://issues.apache.org/jira/browse/ATLAS-4358 > > > Repository: atlas > > > Description > --- > > **Problem** > During start-up in HA mode, the _AtlasPatchManager_ tries to read the patch > data while another instanace of Atlas is trying to create the indexes. > > This causes indexes not to be created correctly. When this happens, the patch > registry ends up doing a a full database scan. With sizable data, this can > result in long startup times. This is detremental for setup that use > containerized cluster setup. > > **Approach** > Initialize the _PatchContext_ in the _applyAll_ method. > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/repository/patches/AtlasPatchManager.java > e2a38ab52 > > > Diff: https://reviews.apache.org/r/73606/diff/1/ > > > Testing > --- > > **PC Build** > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/864/ > > > Thanks, > > Ashutosh Mestry > >
Re: Review Request 73599: Conditionally Disable 'Session Inactivity Timeout'
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73599/#review223517
---
Fix it, then Ship it!
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationFilter.java
Line 311 (original), 325 (patched)
<https://reviews.apache.org/r/73599/#comment312598>
would 'sessionTimeout' be initialized by then? I see it getting initialized
only at getConfiguration(), so won't it be always take default ('0'). line 326
will never be reached.
webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationFilter.java
Line 360 (original), 375 (patched)
<https://reviews.apache.org/r/73599/#comment312597>
add null check ('logoutHandler') to inner 'if' - closer to where it is used
for better readabality.
- Sarath Subramanian
On Sept. 21, 2021, 11:38 a.m., Ashutosh Mestry wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73599/
> ---
>
> (Updated Sept. 21, 2021, 11:38 a.m.)
>
>
> Review request for atlas, Nikhil Bonte, Nixon Rodrigues, Prasad Pawar, and
> Sarath Subramanian.
>
>
> Bugs: ATLAS-4435
> https://issues.apache.org/jira/browse/ATLAS-4435
>
>
> Repository: atlas
>
>
> Description
> ---
>
> Please see JIRA for details.
>
> Additional info:
> * UI pop-up dialog looks at the value passed in the response of
> 'admin/session' REST endpoint. Not passing the configuration value, disables
> the feature from UI (Smart!).
>
>
> Diffs
> -
>
> intg/src/main/java/org/apache/atlas/AtlasConfiguration.java fa519ef5a
>
> webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationFilter.java
> b8d21b9bb
> webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java
> baa040f4c
>
> webapp/src/main/java/org/apache/atlas/web/security/AtlasAuthenticationSuccessHandler.java
> 1b1a80826
>
>
> Diff: https://reviews.apache.org/r/73599/diff/1/
>
>
> Testing
> ---
>
> Manual testing:
> * Verified against Knox SSO enabled cluster.
> * User-name password enabled cluster.
>
>
> Thanks,
>
> Ashutosh Mestry
>
>
Re: Review Request 73589: Indexing Monitor and Recovery
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73589/#review223508 --- graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphDatabase.java Lines 215 (patched) <https://reviews.apache.org/r/73589/#comment312588> in that case - only if user explicitly sets the config in atlas application properties to true/false we should change the global config. It takes a default value here and always updates the global config even if user haven't set it. global config is a site-wide setting for Atlas/Janusgraph and should not be changed unless explicitly specified. - Sarath Subramanian On Sept. 21, 2021, 8:40 a.m., Ashutosh Mestry wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73589/ > --- > > (Updated Sept. 21, 2021, 8:40 a.m.) > > > Review request for atlas, Nikhil Bonte, Nixon Rodrigues, Radhika Kundam, and > Sarath Subramanian. > > > Bugs: ATLAS-4408 > https://issues.apache.org/jira/browse/ATLAS-4408 > > > Repository: atlas > > > Description > --- > > This implementation builds up on Radhika's earlier implementation. > https://reviews.apache.org/r/73555/ > > Indexing > > atlas.graph.index.status.check.frequency: Frequence of checking Solr for > health status. > atlas.graph.index.recovery.start.time: User can set recovery time in UTC. > E.g. > ``` > atlas.graph.index.recovery.start.time=2014-12-11T02:35:58.440Z > ``` > > **Implementation** > New: _updateGlobalConfiguration_: Updating global configuration variables. > > > Diffs > - > > common/src/main/java/org/apache/atlas/repository/Constants.java aea0c1311 > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphIndexClient.java > 9960d8909 > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphManagement.java > 7e3b2f440 > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphDatabase.java > 11267c438 > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphIndexClient.java > ef42dbd3d > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphManagement.java > 1cc7f8b55 > intg/src/main/java/org/apache/atlas/ApplicationProperties.java 682206dad > > repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java > ddfb0089e > > repository/src/main/java/org/apache/atlas/repository/graph/IndexRecoveryService.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/graph/RecoveryInfoManagementTest.java > PRE-CREATION > > server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java > ba8f0886c > > > Diff: https://reviews.apache.org/r/73589/diff/2/ > > > Testing > --- > > Additional unit tests. > > > Thanks, > > Ashutosh Mestry > >
[jira] [Commented] (ATLAS-4358) Mapping for some internal Atlas attributes ( like __patch.type , __timestamp, etc) does not exist in Elasticsearch
[
https://issues.apache.org/jira/browse/ATLAS-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17417946#comment-17417946
]
Sarath Subramanian commented on ATLAS-4358:
---
Thanks for the details [~mehtaanshul]. I have a question:
Internal patch attributes like "__createdBy, __modifiedBy, __timestamp,
__modificationTimestamp, __patch.id, __patch.description, __patch.type,
__patch.action, __patch.state" are created in GraphBackedSearchIndexer which is
Spring annotated component with Order(1)
The AtlasPatchService has an Order(3), so the internal indexes should be
created before Patch Manager is initialized right? Do you see a race scenario
where the init() happens during dependency injection of
AtlasPatchManager in AtlasPatchService?
> Mapping for some internal Atlas attributes ( like __patch.type , __timestamp,
> etc) does not exist in Elasticsearch
> --
>
> Key: ATLAS-4358
> URL: https://issues.apache.org/jira/browse/ATLAS-4358
> Project: Atlas
> Issue Type: Bug
>Reporter: Anshul Mehta
>Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> *Impact of the problem -*
> * Atlas pod taking much longer to become active and this time keep
> increasing as the assets increase. This basically means a downtime on every
> Atlas release.
> * Not able to filter via basic search on attributes like {{__timestamp}} ,
> {{__modificationTimestamp}} , {{createdBy}} and {{modifiedBy}} .
> *Issue -*
> So just before creating the mapping in the mixed index (ES index) Atlas
> creates something called {{propertyKey}} and this propertyKey is used to
> create the mapping. The code is written in a way that checks if propertyKey
> for the current property is null or not. If it is null it creates the
> propertyKey and then adds it to mixed index. If it is not Null it assumes
> that the property has already been added to the index and so skips adding it.
> Now in our case when Atlas checked the propertyKey it was not null (which
> should not have been the case) therefore Atlas skipped adding it to the mixed
> index and so these properties never got added to the mixed index. This
> basically meant propertyKey for these properties were getting created
> somewhere else. We looked into the entire codebase but could not find the use
> of makePropertyKey method ( which is used to create propertyKey) or any other
> similar method.
> Then I saw certain java patch vertices getting created even before these
> internal attributes are added to various indices. Though these patches were
> applied later once all internal attributes were added to all the indices.
> Now, these patch vertices have 9 attributes and we releaized these 9
> attributes are the only attributes missing from ES. So basically when patch
> vertices got created and these vertices with their attributes got added to
> cassandra via janusgraph, janusgraph automatically created propertyKey for
> all these attributes (the janusgraph's makePropertyKey method is not called
> during this process anywhere in the Atlas code). And because internal
> attributes were getting added to indices in another thread at the same time,
> when code checked for propertyKey, it was not null and so it did not add the
> property to the mixed index.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4431) Random NPE when retrieving tasks
[
https://issues.apache.org/jira/browse/ATLAS-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4431:
--
Fix Version/s: 2.3.0
3.0.0
> Random NPE when retrieving tasks
>
>
> Key: ATLAS-4431
> URL: https://issues.apache.org/jira/browse/ATLAS-4431
> Project: Atlas
> Issue Type: Bug
> Components: atlas-intg
>Affects Versions: 2.2.0
>Reporter: Disha Talreja
>Assignee: Disha Talreja
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: ATLAS-4431.patch
>
>
> System tests fail randomly because retrieving tasks using the "admin/tasks"
> endpoint sometimes returns HTTP 500.
> Atlas logs:
> {noformat}
> 2021-09-06 13:51:28,938 INFO - [etp985324122-24:] ~ Logged into Atlas as =
> hrt_qa, by proxyUser = null
> (AtlasAuthenticationFilter$KerberosFilterChainWrapper:724)
> 2021-09-06 13:51:28,939 INFO -
> [etp985324122-24:knox:POST/api/atlas/v2/entity/bulk/classification] ~ Request
> from authenticated user: knox,
> URL=/api/atlas/v2/entity/bulk/classification?doAs=hrt_qa
> (AtlasAuthenticationFilter$KerberosFilterChainWrapper:739)
> 2021-09-06 13:51:28,969 INFO - [etp985324122-24 -
> 78a12f6b-7472-4600-a35d-bfe660358712:knox:POST/api/atlas/v2/entity/bulk/classification]
> ~ updateClassificationText: tag_iuanl: tag_iuanl date Mon Sep 06 13:51:28
> UTC 2021 hive_principal_type USER boolean true string XXJid byte -58 double
> 9.475147275250526E37 arr_string fEOKO short -29141 arr_int 781216025 float
> -3.09E38 int 1369026004 long -2808157562510626816 (EntityGraphMapper:2732)
> 2021-09-06 13:51:29,173 INFO - [etp985324122-24 -
> 78a12f6b-7472-4600-a35d-bfe660358712:knox:POST/api/atlas/v2/entity/bulk/classification]
> ~
> {"type":"CLASSIFICATION_PROPAGATION_ADD","guid":"a4ed45a8-cc15-41dd-9340-ebaca95b07f1","createdBy":"hrt_qa","createdTime":1630936288947,"updatedTime":1630936288947,"parameters":{"relationshipGuid":null,"entityGuid":"5c89916f-9dbc-4be8-afd5-9e7659bbb745","classificationVertexId":"44486696"},"attemptCount":0,"status":"PENDING"}
> (TaskExecutor$TaskLogger:170)
> 2021-09-06 13:51:29,190 INFO - [atlas-task-0-etp985324122-24 -
> c6aa9190-8454-43a4-a862-1f61393994a6:] ~ updateClassificationText: tag_iuanl:
> tag_iuanl date Mon Sep 06 13:51:28 UTC 2021 hive_principal_type USER boolean
> true string XXJid byte -58 double 9.475147275250526E37 arr_string fEOKO short
> -29141 arr_int 781216025 float -3.09E38 int 1369026004 long
> -2808157562510626816 (EntityGraphMapper:2732)
> 2021-09-06 13:51:29,193 INFO - [atlas-task-0-etp985324122-24 -
> c6aa9190-8454-43a4-a862-1f61393994a6:] ~ updateClassificationText: tag_iuanl:
> tag_iuanl date Mon Sep 06 13:51:28 UTC 2021 hive_principal_type USER boolean
> true string XXJid byte -58 double 9.475147275250526E37 arr_string fEOKO short
> -29141 arr_int 781216025 float -3.09E38 int 1369026004 long
> -2808157562510626816 (EntityGraphMapper:2732)
> 2021-09-06 13:51:29,366 ERROR - [etp985324122-594:] ~ URL not supported in HA
> mode: /api/atlas/admin/tasks (ActiveServerFilter:121)
> 2021-09-06 13:51:29,366 INFO - [etp985324122-594:] ~ Logged into Atlas as =
> hrt_qa, by proxyUser = null
> (AtlasAuthenticationFilter$KerberosFilterChainWrapper:724)
> 2021-09-06 13:51:29,366 INFO -
> [etp985324122-594:knox:GET/api/atlas/admin/tasks] ~ Request from
> authenticated user: knox, URL=/api/atlas/admin/tasks?doAs=hrt_qa
> (AtlasAuthenticationFilter$KerberosFilterChainWrapper:739)
> 2021-09-06 13:51:29,375 ERROR - [etp985324122-594 -
> 73d22020-c612-4a00-8f47-b2ebd149e976:knox:GET/api/atlas/admin/tasks] ~ Error
> handling a request: f1318c48aa5440a9 (ExceptionMapperUtil:32)
> java.lang.NullPointerException
> at
> org.apache.atlas.tasks.TaskRegistry.toAtlasTask(TaskRegistry.java:188)
> at org.apache.atlas.tasks.TaskRegistry.getAll(TaskRegistry.java:155)
> at
> org.apache.atlas.tasks.TaskManagement.getAll(TaskManagement.java:120)
> at
> org.apache.atlas.web.resources.AdminResource.getTaskStatus(AdminResource.java:772)
> at jdk.internal.reflect.GeneratedMethodAccessor324.invoke(Unknown
> Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.sun.jer
[jira] [Updated] (ATLAS-4431) Random NPE when retrieving tasks
[
https://issues.apache.org/jira/browse/ATLAS-4431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4431:
--
Affects Version/s: 2.2.0
> Random NPE when retrieving tasks
>
>
> Key: ATLAS-4431
> URL: https://issues.apache.org/jira/browse/ATLAS-4431
> Project: Atlas
> Issue Type: Bug
> Components: atlas-intg
>Affects Versions: 2.2.0
>Reporter: Disha Talreja
>Assignee: Disha Talreja
>Priority: Major
> Attachments: ATLAS-4431.patch
>
>
> System tests fail randomly because retrieving tasks using the "admin/tasks"
> endpoint sometimes returns HTTP 500.
> Atlas logs:
> {noformat}
> 2021-09-06 13:51:28,938 INFO - [etp985324122-24:] ~ Logged into Atlas as =
> hrt_qa, by proxyUser = null
> (AtlasAuthenticationFilter$KerberosFilterChainWrapper:724)
> 2021-09-06 13:51:28,939 INFO -
> [etp985324122-24:knox:POST/api/atlas/v2/entity/bulk/classification] ~ Request
> from authenticated user: knox,
> URL=/api/atlas/v2/entity/bulk/classification?doAs=hrt_qa
> (AtlasAuthenticationFilter$KerberosFilterChainWrapper:739)
> 2021-09-06 13:51:28,969 INFO - [etp985324122-24 -
> 78a12f6b-7472-4600-a35d-bfe660358712:knox:POST/api/atlas/v2/entity/bulk/classification]
> ~ updateClassificationText: tag_iuanl: tag_iuanl date Mon Sep 06 13:51:28
> UTC 2021 hive_principal_type USER boolean true string XXJid byte -58 double
> 9.475147275250526E37 arr_string fEOKO short -29141 arr_int 781216025 float
> -3.09E38 int 1369026004 long -2808157562510626816 (EntityGraphMapper:2732)
> 2021-09-06 13:51:29,173 INFO - [etp985324122-24 -
> 78a12f6b-7472-4600-a35d-bfe660358712:knox:POST/api/atlas/v2/entity/bulk/classification]
> ~
> {"type":"CLASSIFICATION_PROPAGATION_ADD","guid":"a4ed45a8-cc15-41dd-9340-ebaca95b07f1","createdBy":"hrt_qa","createdTime":1630936288947,"updatedTime":1630936288947,"parameters":{"relationshipGuid":null,"entityGuid":"5c89916f-9dbc-4be8-afd5-9e7659bbb745","classificationVertexId":"44486696"},"attemptCount":0,"status":"PENDING"}
> (TaskExecutor$TaskLogger:170)
> 2021-09-06 13:51:29,190 INFO - [atlas-task-0-etp985324122-24 -
> c6aa9190-8454-43a4-a862-1f61393994a6:] ~ updateClassificationText: tag_iuanl:
> tag_iuanl date Mon Sep 06 13:51:28 UTC 2021 hive_principal_type USER boolean
> true string XXJid byte -58 double 9.475147275250526E37 arr_string fEOKO short
> -29141 arr_int 781216025 float -3.09E38 int 1369026004 long
> -2808157562510626816 (EntityGraphMapper:2732)
> 2021-09-06 13:51:29,193 INFO - [atlas-task-0-etp985324122-24 -
> c6aa9190-8454-43a4-a862-1f61393994a6:] ~ updateClassificationText: tag_iuanl:
> tag_iuanl date Mon Sep 06 13:51:28 UTC 2021 hive_principal_type USER boolean
> true string XXJid byte -58 double 9.475147275250526E37 arr_string fEOKO short
> -29141 arr_int 781216025 float -3.09E38 int 1369026004 long
> -2808157562510626816 (EntityGraphMapper:2732)
> 2021-09-06 13:51:29,366 ERROR - [etp985324122-594:] ~ URL not supported in HA
> mode: /api/atlas/admin/tasks (ActiveServerFilter:121)
> 2021-09-06 13:51:29,366 INFO - [etp985324122-594:] ~ Logged into Atlas as =
> hrt_qa, by proxyUser = null
> (AtlasAuthenticationFilter$KerberosFilterChainWrapper:724)
> 2021-09-06 13:51:29,366 INFO -
> [etp985324122-594:knox:GET/api/atlas/admin/tasks] ~ Request from
> authenticated user: knox, URL=/api/atlas/admin/tasks?doAs=hrt_qa
> (AtlasAuthenticationFilter$KerberosFilterChainWrapper:739)
> 2021-09-06 13:51:29,375 ERROR - [etp985324122-594 -
> 73d22020-c612-4a00-8f47-b2ebd149e976:knox:GET/api/atlas/admin/tasks] ~ Error
> handling a request: f1318c48aa5440a9 (ExceptionMapperUtil:32)
> java.lang.NullPointerException
> at
> org.apache.atlas.tasks.TaskRegistry.toAtlasTask(TaskRegistry.java:188)
> at org.apache.atlas.tasks.TaskRegistry.getAll(TaskRegistry.java:155)
> at
> org.apache.atlas.tasks.TaskManagement.getAll(TaskManagement.java:120)
> at
> org.apache.atlas.web.resources.AdminResource.getTaskStatus(AdminResource.java:772)
> at jdk.internal.reflect.GeneratedMethodAccessor324.invoke(Unknown
> Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvoke
Re: Review Request 73593: ATLAS-4431: Random NPE when retrieving tasks
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73593/#review223505 --- Ship it! Ship It! - Sarath Subramanian On Sept. 20, 2021, 1:57 p.m., Disha Talreja wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73593/ > --- > > (Updated Sept. 20, 2021, 1:57 p.m.) > > > Review request for atlas, Ashutosh Mestry, Sarath Subramanian, and Sidharth > Mishra. > > > Repository: atlas > > > Description > --- > > System tests fail randomly because retrieving tasks using the "admin/tasks" > endpoint sometimes returns HTTP 500. This patch includes a few null checks to > avoid NPE. > > > Diffs > - > > repository/src/main/java/org/apache/atlas/tasks/TaskRegistry.java fae8a4fed > > > Diff: https://reviews.apache.org/r/73593/diff/1/ > > > Testing > --- > > Local Build Successful. > PC Build: > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/852/ > > > Thanks, > > Disha Talreja > >
Re: Review Request 73584: ATLAS-4422 : Quick Search: Provide sorting
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73584/#review223492
---
Ship it!
Ship It!
- Sarath Subramanian
On Sept. 15, 2021, 9:03 a.m., Pinal Shah wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73584/
> ---
>
> (Updated Sept. 15, 2021, 9:03 a.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, and Sarath
> Subramanian.
>
>
> Bugs: ATLAS-4422
> https://issues.apache.org/jira/browse/ATLAS-4422
>
>
> Repository: atlas
>
>
> Description
> ---
>
> **Improvement:**
>
> This will add a sorting functionality in quick search API and get the result
> list in Sorted manner.
>
> **Note:**
>
> - Result will be sorted only when 'typeName' has a valid entity type name as
> a value.
> - Only string attribute type (not TEXT) guarantees sorted results
>
> **Example:**
>
> { "query":"sales",
> "typeName":"Asset",
> "sortBy":"__typeName",
> "sortOrder":"ASCENDING" }
>
>
> Diffs
> -
>
>
> intg/src/main/java/org/apache/atlas/model/discovery/QuickSearchParameters.java
> 9e5e9778a
>
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
> 5b3b9486b
>
> repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
> 5e47295cb
> webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java e4c74a9d1
>
>
> Diff: https://reviews.apache.org/r/73584/diff/2/
>
>
> Testing
> ---
>
> added Unit tests
>
>
> Thanks,
>
> Pinal Shah
>
>
Re: Review Request 73584: ATLAS-4422 : Quick Search: Provide sorting
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73584/#review223486
---
Ship it!
lgtm. Can you include UT/IT tests around this sorting.
- Sarath Subramanian
On Sept. 13, 2021, 11:38 p.m., Pinal Shah wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73584/
> ---
>
> (Updated Sept. 13, 2021, 11:38 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, and Sarath
> Subramanian.
>
>
> Bugs: ATLAS-4422
> https://issues.apache.org/jira/browse/ATLAS-4422
>
>
> Repository: atlas
>
>
> Description
> ---
>
> **Improvement:**
>
> This will add a sorting functionality in quick search API and get the result
> list in Sorted manner.
>
> **Note:**
>
> - Result will be sorted only when 'typeName' has a valid entity type name as
> a value.
> - Only string attribute type (not TEXT) guarantees sorted results
>
> **Example:**
>
> { "query":"sales",
> "typeName":"Asset",
> "sortBy":"__typeName",
> "sortOrder":"ASCENDING" }
>
>
> Diffs
> -
>
>
> intg/src/main/java/org/apache/atlas/model/discovery/QuickSearchParameters.java
> 9e5e9778a
>
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
> 5b3b9486b
> webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java e4c74a9d1
>
>
> Diff: https://reviews.apache.org/r/73584/diff/1/
>
>
> Testing
> ---
>
> Tested manually
>
>
> Thanks,
>
> Pinal Shah
>
>
[jira] [Updated] (ATLAS-4369) import-hive.sh does not write logs to /var/log/atlas/import-hive.log file, instead write into ${atlas.log.file} under the directory "${atlas.log.dir}"
[
https://issues.apache.org/jira/browse/ATLAS-4369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4369:
--
Fix Version/s: 2.3.0
3.0.0
> import-hive.sh does not write logs to /var/log/atlas/import-hive.log file,
> instead write into ${atlas.log.file} under the directory "${atlas.log.dir}"
> ---
>
> Key: ATLAS-4369
> URL: https://issues.apache.org/jira/browse/ATLAS-4369
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Umesh Padashetty
>Assignee: Disha Talreja
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: ATLAS-4369.patch
>
>
> When the import scripts like
> * import-kafka.sh
> * import-hive.sh
> * import-hbase.sh
> are run, the corresponding logs are written into
> * /var/log/atlas/import-kafka.log
> * /var/log/atlas/import-hive.log
> * /var/log/atlas/import-hbase.log
> /var/log/atlas/import-kafka.log and /var/log/atlas/import-hbase.log logs are
> correctly written. But when import-hive.sh is ran,
> /var/log/atlas/import-hive.log is not created or written to
> {code:java}
> [root@ccycloud-1 atlas]# ls -ltr
> total 10988
> drwxr-xr-x 2 cstm_atlas cstm_atlas 4096 Jul 16 06:55 jstacks
> drwxr-xr-x 4 root root 4096 Jul 16 06:55 audit
> -rw-r--r-- 1 cstm_atlas cstm_atlas0 Jul 16 06:55 metric.log
> -rw-r--r-- 1 cstm_atlas cstm_atlas0 Jul 16 06:55 failed.log
> drwxr-xr-x 2 cstm_atlas cstm_atlas 4096 Jul 18 03:52 support-bundle
> -rw-r--r-- 1 root root 21573 Jul 22 09:56 import-kafka.log
> -rw-r--r-- 1 root root 159459 Jul 22 09:58 import-hbase.log
> -rw-r--r-- 1 cstm_atlas cstm_atlas19429 Jul 22 10:12
> gc-worker.log.0.current
> -rw-r--r-- 1 cstm_atlas cstm_atlas 179081 Jul 22 10:22 audit.log
> -rw-r--r-- 1 cstm_atlas cstm_atlas 10840903 Jul 22 10:23 application.log
> {code}
> whereas it creates a directory with the name ${atlas.log.dir} and a file with
> the name ${atlas.log.file} under it
> {code:java}
> [root@ccycloud-1 hook-bin]# ls -ltr
> total 44
> -rwxr-xr-x 1 root root 3892 Jul 14 13:09 import-kafka.sh
> -rwxr-xr-x 1 root root 4547 Jul 14 13:09 import-hive.sh
> -rwxr-xr-x 1 root root 4341 Jul 14 13:09 import-hbase.sh
> drwxr-sr-x 2 root root 4096 Jul 22 10:23 ${atlas.log.dir}
> -rw-r--r-- 1 root root 20094 Jul 22 10:23 derby.log {code}
> The import-hive.sh logs are written into ${atlas.log.file}
> {code:java}
> [root@ccycloud-1 ${atlas.log.dir}]# ls -ltra
> total 48
> drwxr-sr-x 3 root root 4096 Jul 22 10:23 ..
> drwxr-sr-x 2 root root 4096 Jul 22 10:23 .
> -rw-r--r-- 1 root root 40960 Jul 22 10:23 ${atlas.log.file} {code}
> The import-hive.sh logs should be written into /var/log/atlas/import-hive.log
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4369) import-hive.sh does not write logs to /var/log/atlas/import-hive.log file, instead write into ${atlas.log.file} under the directory "${atlas.log.dir}"
[
https://issues.apache.org/jira/browse/ATLAS-4369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarath Subramanian updated ATLAS-4369:
--
Affects Version/s: 3.0.0
2.2.0
> import-hive.sh does not write logs to /var/log/atlas/import-hive.log file,
> instead write into ${atlas.log.file} under the directory "${atlas.log.dir}"
> ---
>
> Key: ATLAS-4369
> URL: https://issues.apache.org/jira/browse/ATLAS-4369
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Umesh Padashetty
>Assignee: Disha Talreja
>Priority: Major
> Attachments: ATLAS-4369.patch
>
>
> When the import scripts like
> * import-kafka.sh
> * import-hive.sh
> * import-hbase.sh
> are run, the corresponding logs are written into
> * /var/log/atlas/import-kafka.log
> * /var/log/atlas/import-hive.log
> * /var/log/atlas/import-hbase.log
> /var/log/atlas/import-kafka.log and /var/log/atlas/import-hbase.log logs are
> correctly written. But when import-hive.sh is ran,
> /var/log/atlas/import-hive.log is not created or written to
> {code:java}
> [root@ccycloud-1 atlas]# ls -ltr
> total 10988
> drwxr-xr-x 2 cstm_atlas cstm_atlas 4096 Jul 16 06:55 jstacks
> drwxr-xr-x 4 root root 4096 Jul 16 06:55 audit
> -rw-r--r-- 1 cstm_atlas cstm_atlas0 Jul 16 06:55 metric.log
> -rw-r--r-- 1 cstm_atlas cstm_atlas0 Jul 16 06:55 failed.log
> drwxr-xr-x 2 cstm_atlas cstm_atlas 4096 Jul 18 03:52 support-bundle
> -rw-r--r-- 1 root root 21573 Jul 22 09:56 import-kafka.log
> -rw-r--r-- 1 root root 159459 Jul 22 09:58 import-hbase.log
> -rw-r--r-- 1 cstm_atlas cstm_atlas19429 Jul 22 10:12
> gc-worker.log.0.current
> -rw-r--r-- 1 cstm_atlas cstm_atlas 179081 Jul 22 10:22 audit.log
> -rw-r--r-- 1 cstm_atlas cstm_atlas 10840903 Jul 22 10:23 application.log
> {code}
> whereas it creates a directory with the name ${atlas.log.dir} and a file with
> the name ${atlas.log.file} under it
> {code:java}
> [root@ccycloud-1 hook-bin]# ls -ltr
> total 44
> -rwxr-xr-x 1 root root 3892 Jul 14 13:09 import-kafka.sh
> -rwxr-xr-x 1 root root 4547 Jul 14 13:09 import-hive.sh
> -rwxr-xr-x 1 root root 4341 Jul 14 13:09 import-hbase.sh
> drwxr-sr-x 2 root root 4096 Jul 22 10:23 ${atlas.log.dir}
> -rw-r--r-- 1 root root 20094 Jul 22 10:23 derby.log {code}
> The import-hive.sh logs are written into ${atlas.log.file}
> {code:java}
> [root@ccycloud-1 ${atlas.log.dir}]# ls -ltra
> total 48
> drwxr-sr-x 3 root root 4096 Jul 22 10:23 ..
> drwxr-sr-x 2 root root 4096 Jul 22 10:23 .
> -rw-r--r-- 1 root root 40960 Jul 22 10:23 ${atlas.log.file} {code}
> The import-hive.sh logs should be written into /var/log/atlas/import-hive.log
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Review Request 73580: ATLAS-4369: import-hive.sh does not write logs to /var/log/atlas/import-hive.log file, instead write into ${atlas.log.file} under the directory "${atlas.log.dir}"
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73580/#review223484
---
Ship it!
Ship It!
- Sarath Subramanian
On Sept. 13, 2021, 10:13 a.m., Disha Talreja wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73580/
> ---
>
> (Updated Sept. 13, 2021, 10:13 a.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Sarath Subramanian, and Sidharth
> Mishra.
>
>
> Repository: atlas
>
>
> Description
> ---
>
> Due to the conflicting log4j jars from hive conf, import-hive.sh did not
> write logs to /var/log/atlas/import-hive.log and wrote to
> ${atlas.log.dir}/${atlas.log.file} instead. With this patch, the logs are
> written to the correct/desired location (/var/log/atlas/import-hive.log).
>
>
> Diffs
> -
>
> addons/hive-bridge/src/bin/import-hive.sh 693a79bd0
>
>
> Diff: https://reviews.apache.org/r/73580/diff/1/
>
>
> Testing
> ---
>
> Patched a cluster
>
>
> Thanks,
>
> Disha Talreja
>
>
Re: Review Request 73528: ATLAS-4377 : Atlas - Upgrade Spring Security to 5.4.4+/5.3.8+/5.2.9+
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73528/#review223466 --- Ship it! Ship It! - Sarath Subramanian On Sept. 6, 2021, 1:45 a.m., chaitali wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73528/ > --- > > (Updated Sept. 6, 2021, 1:45 a.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nixon Rodrigues, > and Sarath Subramanian. > > > Bugs: ATLAS-4377 > https://issues.apache.org/jira/browse/ATLAS-4377 > > > Repository: atlas > > > Description > --- > > Atlas is pulling in Spring security 4.2.17.RELEASE upgrade to 5.4.4, 5.3.8, > or 5.2.9 > Upgrading spring-security version to 5.5.1 this is the latest > Upgrading spring-core version to 5.3.8 current version 4.3.29.RELEASE > > > Diffs > - > > addons/falcon-bridge-shim/pom.xml 02da049db > addons/falcon-bridge/pom.xml e0d2f3be7 > authorization/pom.xml 8e5a85db4 > common/src/main/java/org/apache/atlas/utils/OnAtlasPropertyCondition.java > ece6e370a > pom.xml 8ae338cf2 > repository/pom.xml bf34efee2 > repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java > 343d00d72 > test-tools/pom.xml c861ea67f > webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java e16796ff6 > webapp/src/main/java/org/apache/atlas/web/util/Servlets.java 71eca2e50 > webapp/src/main/webapp/WEB-INF/web.xml 7fb56efde > webapp/src/test/java/org/apache/atlas/web/security/UserDaoTest.java > 519be804e > webapp/src/test/webapp/WEB-INF/web.xml 2388fbfbb > > > Diff: https://reviews.apache.org/r/73528/diff/3/ > > > Testing > --- > > mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > Atlas server runnning > mvn clean install -DskipTests > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/811/console > > > Thanks, > > chaitali > >
Re: Review Request 73544: ATLAS-4400: Lower version of aws_s3_v2_directory (1.2) of Atlas is is compatible with new Hooks
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73544/#review223414 --- Ship it! Ship It! - Sarath Subramanian On Aug. 26, 2021, 4:39 p.m., Sidharth Mishra wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73544/ > --- > > (Updated Aug. 26, 2021, 4:39 p.m.) > > > Review request for atlas, Ashutosh Mestry, Disha Talreja, Radhika Kundam, and > Sarath Subramanian. > > > Bugs: ATLAS-4400 > https://issues.apache.org/jira/browse/ATLAS-4400 > > > Repository: atlas > > > Description > --- > > ATLAS-4400: Fixed Hook and Atlas Preprocessor to handle S3 V2 directory > objectPrefix Issue with Atlas Server and Hook versions mismatch > > > Diffs > - > > common/src/main/java/org/apache/atlas/utils/AtlasPathExtractorUtil.java > a9f2e5092 > common/src/test/java/org/apache/atlas/utils/AtlasPathExtractorUtilTest.java > 6bf5d5779 > > webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java > 5643af95c > > webapp/src/main/java/org/apache/atlas/notification/preprocessor/AWSS3V2Preprocessor.java > PRE-CREATION > > webapp/src/main/java/org/apache/atlas/notification/preprocessor/EntityPreprocessor.java > 7f0cafe86 > > webapp/src/main/java/org/apache/atlas/notification/preprocessor/PreprocessorContext.java > 59f6440d5 > > webapp/src/test/java/org/apache/atlas/notification/preprocessor/AWSS3V2PreprocessorTest.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/73544/diff/3/ > > > Testing > --- > > Manually tested the Newer Atlas with old Hook and vice versa. Precommit - > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/824/ > > > Thanks, > > Sidharth Mishra > >
Re: Review Request 73549: ATLAS-4403: Improve find entity by unique attributes logic - when multiple unique attributes exist for entity type
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73549/ --- (Updated Aug. 26, 2021, 1:32 p.m.) Review request for atlas, Ashutosh Mestry, Disha Talreja, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, Radhika Kundam, and Sidharth Mishra. Changes --- addressed review comments. Bugs: ATLAS-4403 https://issues.apache.org/jira/browse/ATLAS-4403 Repository: atlas Description --- When a entity type has multiple unique attributes defined: unique_attr1 unique_attr2 During creation of entity, we check if entity with unique attributes is already present and return/update them. The current logic check if entity is present - one unique attribute at a time, if present will return immediately. We should improve the lookup logic to look for entity with all unique attributes specified and not just return on the first unique attribute value. Diffs (updated) - repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasGraphUtilsV2.java e73f084b8 Diff: https://reviews.apache.org/r/73549/diff/4/ Changes: https://reviews.apache.org/r/73549/diff/3-4/ Testing --- Pre-commit: https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/822/consoleFull (SUCCESS) Thanks, Sarath Subramanian
Re: Review Request 73547: ATLAS-4401: Upgrade commons-io to 2.8.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73547/#review223406 --- Ship it! Ship It! - Sarath Subramanian On Aug. 25, 2021, 8:33 a.m., Disha Talreja wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73547/ > --- > > (Updated Aug. 25, 2021, 8:33 a.m.) > > > Review request for atlas, Ashutosh Mestry, Radhika Kundam, Sarath > Subramanian, and Sidharth Mishra. > > > Bugs: ATLAS-4401 > https://issues.apache.org/jira/browse/ATLAS-4401 > > > Repository: atlas > > > Description > --- > > This patch upgrades commons-io 2.6 to 2.8.0 > > > Diffs > - > > pom.xml 8ae338cf2 > > > Diff: https://reviews.apache.org/r/73547/diff/1/ > > > Testing > --- > > mvn clean -DskipTests install > mvn clean -DskipTests package -Pdist,embedded-hbase-solr > Atlas server running. > PC Build: > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/820/ > > > Thanks, > > Disha Talreja > >
Re: Review Request 73548: ATLAS-4402: Upgrade junit to 4.13.1
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73548/#review223405 --- Ship it! Ship It! - Sarath Subramanian On Aug. 25, 2021, 12:02 p.m., Disha Talreja wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73548/ > --- > > (Updated Aug. 25, 2021, 12:02 p.m.) > > > Review request for atlas, Ashutosh Mestry, Radhika Kundam, Sarath > Subramanian, and Sidharth Mishra. > > > Bugs: ATLAS-4402 > https://issues.apache.org/jira/browse/ATLAS-4402 > > > Repository: atlas > > > Description > --- > > This patch upgrades junit version 4.13 to 4.13.1 > > > Diffs > - > > pom.xml 8ae338cf2 > > > Diff: https://reviews.apache.org/r/73548/diff/1/ > > > Testing > --- > > mvn clean -DskipTests install > mvn clean -DskipTests package -Pdist,embedded-hbase-solr > Atlas server running. > > > Thanks, > > Disha Talreja > >
Re: Review Request 73544: ATLAS-4400: Lower version of aws_s3_v2_directory (1.2) of Atlas is is compatible with new Hooks
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73544/#review223404 --- Ship it! Ship It! - Sarath Subramanian On Aug. 25, 2021, 8:41 p.m., Sidharth Mishra wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73544/ > --- > > (Updated Aug. 25, 2021, 8:41 p.m.) > > > Review request for atlas, Ashutosh Mestry, Disha Talreja, Radhika Kundam, and > Sarath Subramanian. > > > Bugs: ATLAS-4400 > https://issues.apache.org/jira/browse/ATLAS-4400 > > > Repository: atlas > > > Description > --- > > ATLAS-4400: Fixed Hook and Atlas Preprocessor to handle S3 V2 directory > objectPrefix Issue with Atlas Server and Hook versions mismatch > > > Diffs > - > > common/src/main/java/org/apache/atlas/utils/AtlasPathExtractorUtil.java > a9f2e5092 > common/src/test/java/org/apache/atlas/utils/AtlasPathExtractorUtilTest.java > 6bf5d5779 > > webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java > 5643af95c > > webapp/src/main/java/org/apache/atlas/notification/preprocessor/AWSS3V2Preprocessor.java > PRE-CREATION > > webapp/src/main/java/org/apache/atlas/notification/preprocessor/EntityPreprocessor.java > 7f0cafe86 > > webapp/src/main/java/org/apache/atlas/notification/preprocessor/PreprocessorContext.java > 59f6440d5 > > > Diff: https://reviews.apache.org/r/73544/diff/2/ > > > Testing > --- > > Manually tested the Newer Atlas with old Hook and vice versa. Precommit - > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/824/ > > > Thanks, > > Sidharth Mishra > >
Re: Review Request 73544: ATLAS-4400: Lower version of aws_s3_v2_directory (1.2) of Atlas is is compatible with new Hooks
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73544/#review223398
---
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
Lines 1056 (patched)
<https://reviews.apache.org/r/73544/#comment312465>
replace line 1056-1058:
entities.addAll(context.getEntities())
also replace line 1062-1064:
entities.addAll(context.getReferredEntities().values())
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
Lines 1067 (patched)
<https://reviews.apache.org/r/73544/#comment312466>
'entities' can never be null, we assign it to empty List in line 1053.
Consider checking if List is not empty.
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
Lines 1068 (patched)
<https://reviews.apache.org/r/73544/#comment312467>
for (AtlasEntity entity : entities) {
...
}
webapp/src/main/java/org/apache/atlas/notification/preprocessor/EntityPreprocessor.java
Lines 69 (patched)
<https://reviews.apache.org/r/73544/#comment312463>
S3V2_PREPROCESSOR_MAP => AWS_S3_V2_PREPROCESSOR_MAP
webapp/src/main/java/org/apache/atlas/notification/preprocessor/S3V2Preprocessor.java
Lines 27 (patched)
<https://reviews.apache.org/r/73544/#comment312464>
S3V2Preprocessor => AWSS3V2DirectoryPreprocessor
webapp/src/main/java/org/apache/atlas/notification/preprocessor/S3V2Preprocessor.java
Lines 56 (patched)
<https://reviews.apache.org/r/73544/#comment312469>
consider changing this to INFO log for debugging purpose.
webapp/src/main/java/org/apache/atlas/notification/preprocessor/S3V2Preprocessor.java
Lines 66 (patched)
<https://reviews.apache.org/r/73544/#comment312468>
consider also including the 'name' attribute from aws_s3_v2_directory type
in this check:
&& objectPrefix.contains(name)
qualifiedName :
s3a://my_bucket_1/datalake/warehouse/tablespace/external/hive/employee.csv/@cm
objectPrefix : /datalake/warehouse/tablespace/external/hive/employee.csv/
name : employee.csv
- Sarath Subramanian
On Aug. 24, 2021, 11:21 p.m., Sidharth Mishra wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73544/
> ---
>
> (Updated Aug. 24, 2021, 11:21 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Disha Talreja, Radhika Kundam, and
> Sarath Subramanian.
>
>
> Bugs: ATLAS-4400
> https://issues.apache.org/jira/browse/ATLAS-4400
>
>
> Repository: atlas
>
>
> Description
> ---
>
> ATLAS-4400: Fixed Hook and Atlas Preprocessor to handle S3 V2 directory
> objectPrefix Issue with Atlas Server and Hook versions mismatch
>
>
> Diffs
> -
>
> common/src/main/java/org/apache/atlas/utils/AtlasPathExtractorUtil.java
> a9f2e5092
> common/src/test/java/org/apache/atlas/utils/AtlasPathExtractorUtilTest.java
> 6bf5d5779
>
> webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
> 5643af95c
>
> webapp/src/main/java/org/apache/atlas/notification/preprocessor/EntityPreprocessor.java
> 7f0cafe86
>
> webapp/src/main/java/org/apache/atlas/notification/preprocessor/PreprocessorContext.java
> 59f6440d5
>
> webapp/src/main/java/org/apache/atlas/notification/preprocessor/S3V2Preprocessor.java
> PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/73544/diff/1/
>
>
> Testing
> ---
>
> Manually tested the Newer Atlas with old Hook and vice versa. Precommit -
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/817/
>
>
> Thanks,
>
> Sidharth Mishra
>
>
Review Request 73549: ATLAS-4403: Improve find entity by unique attributes logic - when multiple unique attributes exist for entity type
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73549/ --- Review request for atlas, Ashutosh Mestry, Disha Talreja, Madhan Neethiraj, Radhika Kundam, and Sidharth Mishra. Bugs: ATLAS-4403 https://issues.apache.org/jira/browse/ATLAS-4403 Repository: atlas Description --- When a entity type has multiple unique attributes defined: unique_attr1 unique_attr2 During creation of entity, we check if entity with unique attributes is already present and return/update them. The current logic check if entity is present - one unique attribute at a time, if present will return immediately. We should improve the lookup logic to look for entity with all unique attributes specified and not just return on the first unique attribute value. Diffs - repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasGraphUtilsV2.java e73f084b8 Diff: https://reviews.apache.org/r/73549/diff/1/ Testing --- Pre-commit: https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/822/consoleFull (SUCCESS) Thanks, Sarath Subramanian
[jira] [Created] (ATLAS-4403) Improve find entity by unique attributes logic - when multiple unique attributes exist for entity type
Sarath Subramanian created ATLAS-4403: - Summary: Improve find entity by unique attributes logic - when multiple unique attributes exist for entity type Key: ATLAS-4403 URL: https://issues.apache.org/jira/browse/ATLAS-4403 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 2.2.0 Reporter: Sarath Subramanian Assignee: Sarath Subramanian Fix For: 2.3.0 When a entity type has multiple unique attributes defined: * unique_attr1 * unique_attr2 During creation of entity, we check if entity with unique attributes is already present and return/update them. The current logic check if entity is present - one unique attribute at a time, if present will return immediately. We should improve the lookup logic to look for entity with all unique attributes specified and not just return on the first unique attribute value. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4399) Restore deleted entity
[ https://issues.apache.org/jira/browse/ATLAS-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17403260#comment-17403260 ] Sarath Subramanian commented on ATLAS-4399: --- To answer your questions: # You cannot create an entity with an initial DELETED status. # Currently reactivating deleted entity is not supported in Atlas. Hope this helps. > Restore deleted entity > -- > > Key: ATLAS-4399 > URL: https://issues.apache.org/jira/browse/ATLAS-4399 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Zhang Joseph >Priority: Major > Labels: Restore > > Hi All, > I have met this situation that when I delete one entity and entity status > changed from "ACTIVE" to "DELETED", then I want to re-active the entity with > same qualifiedName, but I got two entities with same qualifiedName, one with > status ACTIVE, another with status DELETED. instead of restoring the old one. > Here is my questions: > # Can I create one entity with initial status "DELETED"? > # Do we have approach to restore deleted entity? I have read some clues > about OMRS ways but with no sample, and not find any code in atlas source > code. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (ATLAS-4399) Restore deleted entity
[ https://issues.apache.org/jira/browse/ATLAS-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17403250#comment-17403250 ] Sarath Subramanian edited comment on ATLAS-4399 at 8/23/21, 4:05 PM: - [~josephz], Atlas is a governance tool and it is important to store the metadata history of deleted entity as well. Currently there is no way to activate deleted entity in Atlas. If you reactive deleted entity, the metadata information of previous deleted entity is overriden and the history is lost. An entity in Atlas can have only one ACTIVE state and any number of DELETED entity states for the same qualifiedName. was (Author: sarath.ku...@gmail.com): [~josephz], Atlas is a governance tool and its key to store the metadata history of deleted entity as well. Currently there is no way to activate deleted entity in Atlas. If you reactive deleted entity, the metadata information of previous deleted entity is overriden and the history is lost. An entity in Atlas can have only one ACTIVE state and any number of DELETED entity states for the same qualifiedName. > Restore deleted entity > -- > > Key: ATLAS-4399 > URL: https://issues.apache.org/jira/browse/ATLAS-4399 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Zhang Joseph >Priority: Major > Labels: Restore > > Hi All, > I have met this situation that when I delete one entity and entity status > changed from "ACTIVE" to "DELETED", then I want to re-active the entity with > same qualifiedName, but I got two entities with same qualifiedName, one with > status ACTIVE, another with status DELETED. instead of restoring the old one. > Here is my questions: > # Can I create one entity with initial status "DELETED"? > # Do we have approach to restore deleted entity? I have read some clues > about OMRS ways but with no sample, and not find any code in atlas source > code. -- This message was sent by Atlassian Jira (v8.3.4#803005)
