Re: Review Request 74925: ATLAS-4225 : Support for Chinese character in entity data.

2024-05-01 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74925/#review226408
---


Fix it, then Ship it!





repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
Lines 472 (patched)
<https://reviews.apache.org/r/74925/#comment314619>

Consider replacing #472 - #482 with the following line:
  String modifiedString = StringUtils.strip(query, "*");



repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
Lines 484 (patched)
<https://reviews.apache.org/r/74925/#comment314620>

"StringUtils.isNotEmpty(modifiedString) && " doesn't seem needed here, as 
subsequent call to hastokenizeChar() already handles this condition.


- Madhan Neethiraj


On April 30, 2024, 12:22 p.m., chaitali wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74925/
> ---
> 
> (Updated April 30, 2024, 12:22 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Deep Singh, Jayendra Parab, Pinal 
> Shah, and Sheetal Shah.
> 
> 
> Bugs: ATLAS-4225
> https://issues.apache.org/jira/browse/ATLAS-4225
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Updated latest patch with synchronized code 
> 
> Creating this review request to track and close 
> https://reviews.apache.org/r/73256/diff/3#1 as the patch here is not in sync 
> with latest code
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 591b31612 
>   intg/src/test/java/org/apache/atlas/type/TestAtlasStructType.java f117fb359 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  9be6517e9 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
>  01b88eb2f 
>   repository/src/test/java/org/apache/atlas/BasicTestSetup.java 593f8c19b 
>   
> repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
>  282a3fd4a 
>   
> repository/src/test/java/org/apache/atlas/discovery/ClassificationSearchProcessorTest.java
>  121dca9f5 
>   
> repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
>  3f9e74d11 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2Test.java
>  1489b2760 
> 
> 
> Diff: https://reviews.apache.org/r/74925/diff/6/
> 
> 
> Testing
> ---
> 
> Testcases are added in the patch
> PC is successful : 
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1621/
> 
> 
> Thanks,
> 
> chaitali
> 
>



Re: Review Request 74925: ATLAS-4225 : Support for Chinese character in entity data.

2024-04-26 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74925/#review226403
---




intg/src/main/java/org/apache/atlas/type/AtlasStructType.java
Line 1078 (original), 1078 (patched)
<https://reviews.apache.org/r/74925/#comment314616>

This change doesn't seem correct. Did you mean the following:
  if (StringUtils.isNotBlank(value)) {
...
  }



intg/src/main/java/org/apache/atlas/type/AtlasStructType.java
Lines 1083 (patched)
<https://reviews.apache.org/r/74925/#comment314617>

All CJK characters are treated as tokenizeChar here. While 
hastokenizeChar() only treats specific characters as tokenizeChar. Consider 
adding unit tests for hastokenizeChar(String) with several inputs.


- Madhan Neethiraj


On April 26, 2024, 9:54 a.m., chaitali wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74925/
> ---
> 
> (Updated April 26, 2024, 9:54 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Deep Singh, Jayendra Parab, Pinal 
> Shah, and Sheetal Shah.
> 
> 
> Bugs: ATLAS-4225
> https://issues.apache.org/jira/browse/ATLAS-4225
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Updated latest patch with synchronized code 
> 
> Creating this review request to track and close 
> https://reviews.apache.org/r/73256/diff/3#1 as the patch here is not in sync 
> with latest code
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 591b31612 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  9be6517e9 
>   repository/src/main/java/org/apache/atlas/discovery/SearchContext.java 
> 9a9a5512a 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
>  01b88eb2f 
>   repository/src/test/java/org/apache/atlas/BasicTestSetup.java 593f8c19b 
>   
> repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
>  282a3fd4a 
>   
> repository/src/test/java/org/apache/atlas/discovery/ClassificationSearchProcessorTest.java
>  121dca9f5 
>   
> repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
>  3f9e74d11 
> 
> 
> Diff: https://reviews.apache.org/r/74925/diff/4/
> 
> 
> Testing
> ---
> 
> Testcases are added in the patch
> PC is successful : 
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1561/consoleFull
> 
> 
> Thanks,
> 
> chaitali
> 
>



Re: Request to join Atlas project.

2024-04-24 Thread Madhan Neethiraj
Darshil Shah,

Thank you for your interest in contributing to Apache Atlas. You now have 
contributor access.

Welcome to Apache Atlas.

Thanks,
Madhan



On 4/24/24, 9:36 AM, "Darshil Shah" mailto:shahdarshil...@gmail.com>> wrote:


Hi Team,


Can you please add me as a contributor.


Thank you,
Darshil Shah.


On Tue, Apr 23, 2024 at 2:13 PM Darshil Shah mailto:shahdarshil...@gmail.com>>
wrote:


> Hi Team,
>
> I would like to contribute to the Atlas project, Can you please add me as
> contributor. My Apache id is : darshilshah04.
>
>
> Thanks and Regards,
> Darshil Shah.
>






Re: Review Request 74608: ATLAS-4797 : Implement custom audit filters in Atlas

2024-04-24 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74608/#review226400
---



Consider adding few samples and test cases, to make it easier to understand 
audit filter functioanlity.


intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java
Lines 221 (patched)
<https://reviews.apache.org/r/74608/#comment314614>

Consider annotating isDiscarded() and setDiscarded() with @JsonIgnore.



repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
Lines 462 (patched)
<https://reviews.apache.org/r/74608/#comment314615>

prepareDataObject() seems expensive - especially given it will be executed 
for each entity operation in Atlas. Are all fields of AtlasEntity used in rules?



repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java
Lines 74 (patched)
<https://reviews.apache.org/r/74608/#comment314611>

this.rules could be updated between #74 and #81, with the call to 
setRules() (#41). Consider adding the following line before #74:

final List rules = this.rules;



repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java
Lines 83 (patched)
<https://reviews.apache.org/r/74608/#comment314612>

This conversion seems expensive. Is this conversion necessary for every 
evalation of a rule?



repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java
Lines 97 (patched)
<https://reviews.apache.org/r/74608/#comment314613>

Rule evaluation continues even after ACCEPT result from a rule; is this 
necessary?



webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java
Lines 1327 (patched)
<https://reviews.apache.org/r/74608/#comment314610>

Doesn't "/audits/rules/all" conflict with "/audits/rules/{ruleGuid}" (line 
#1269 above)?


- Madhan Neethiraj


On April 4, 2024, 5:14 a.m., Sheetal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74608/
> ---
> 
> (Updated April 4, 2024, 5:14 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Prasad 
> Pawar, Radhika Kundam, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-4797
> https://issues.apache.org/jira/browse/ATLAS-4797
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Currently, Atlas has to persist all audit events without any constraints 
> which can make audit data grow exponentially after some time.
>  
> This feature is specifically to reduce Atlas audit storage where generation 
> of audit events can be controlled based on the user's use-case/criteria
> 
> Note: Audit data mentioned here is exclusive of Admin Audit Data which 
> persists in atlas_janus table. Filtering will be applicable to the entity 
> audit data which persists in ATLAS_ENTITY_AUDIT_EVENTS table.
> 
> Precommit details :
> Latest PC build (PASSED) : 
> build:https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1592/
> 
> 
> Diffs
> -
> 
>   addons/hbase-bridge/pom.xml 278b6c6d3 
>   addons/hbase-testing-util/pom.xml c4f3a99ee 
>   addons/hive-bridge-shim/pom.xml 39f16ceee 
>   addons/hive-bridge/pom.xml 356ac9542 
>   addons/kafka-bridge/pom.xml 093db7229 
>   addons/models/-Area0/0010-base_model.json a4a9248ec 
>   addons/sqoop-bridge/pom.xml 4b6eac98f 
>   authorization/pom.xml 7a1108487 
>   distro/src/conf/atlas-application.properties b5734d7a8 
>   graphdb/api/pom.xml 4ba89b20f 
>   graphdb/janus-hbase2/pom.xml 9ec087dbb 
>   intg/pom.xml 43a172c1b 
>   intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 0868f8b6e 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 77a6fd8c3 
>   intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
> 3afd27e7d 
>   repository/pom.xml 8fd744b44 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
>  9b011ab06 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  b7f0dd5e2 
>   repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java 
> e63152475 
>   
> repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java
>  PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/rulesengine/AtlasRule.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/rulesengine/AtlasRuleUtils.java

[DRAFT] [REPORT] Apache Atlas - March 2024

2024-03-13 Thread Madhan Neethiraj
Atlas community,

Please review the following draft report for this quarter.

Thanks,
Madhan


## Description:
The mission of Apache Atlas is the creation and maintenance of software related 
to a scalable and extensible set of core foundational governance services - 
enabling enterprises to effectively and efficiently meet their compliance 
requirements

## Project Status:
Current project status: Ongoing, with low activity
Issues for the board: here are no issues requiring board attention.

## Membership Data:
Apache Atlas was founded 2017-06-20 (7 years ago)
There are currently 48 committers and 32 PMC members in this project.
The Committer-to-PMC ratio is 3:2.

Community changes, past quarter:
- No new PMC members. Last addition was Ashutosh Mestry on 2019-04-16.
- No new committers. Last addition was Umesh Padashetty on 2023-06-14.

## Project Activity:
- fixes in download search results functionality
- UI updates to improve usability of tag-propagation enable/disable
- fix in business metadata changes to update system attributes
- updated versions of Spring Security, netty

## Community Health:
- dev@atlas.apache.org had a 40% decrease in traffic in the past quarter (143
  emails compared to 238)
- u...@atlas.apache.org had a 100% increase in traffic in the past quarter (8
  emails compared to 4)
- 10 commits in the past quarter (-73% change)
- 6 code contributors in the past quarter (-50% change)

## Recent releases:
- Apache Atlas 2.3.0 was released on 2022-12-06
- Apache Atlas 2.2.0 was released on 2021-08-17
- Apache Atlas 2.1.0 was released on 2020-07-15






Re: Review Request 74902: ATLAS-4826: Provide Liveness and Readyness probes in Atlas

2024-02-22 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74902/#review226233
---


Ship it!




Ship It!

- Madhan Neethiraj


On Feb. 22, 2024, 11:26 p.m., Disha Talreja wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74902/
> ---
> 
> (Updated Feb. 22, 2024, 11:26 p.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Pinal Shah, and Radhika Kundam.
> 
> 
> Bugs: ATLAS-4826
> https://issues.apache.org/jira/browse/ATLAS-4826
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> In order to monitor the application health and be able to restart it if it is 
> unhealthy we need to check the:
> 1) Liveness - if Atlas is live and running as expected
> 2) Readyness - if Atlas is live and ready to accept client requests
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/util/AtlasMetricsUtil.java 
> beb90e67b 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
> 4d59fa3d6 
>   webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
> 5b16ba149 
> 
> 
> Diff: https://reviews.apache.org/r/74902/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Disha Talreja
> 
>



Re: Review Request 74902: ATLAS-4826: Provide Liveness and Readyness probes in Atlas

2024-02-21 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74902/#review226232
---




graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphDatabase.java
Lines 438 (patched)
<https://reviews.apache.org/r/74902/#comment314475>

Atlas can be used with other storage backends (like Cassandra) as well. 
This change is HBase specific, and wouldn't work with other storage backends. 
Please review and update with an alternate approach.



webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java
Lines 1151 (patched)
<https://reviews.apache.org/r/74902/#comment314474>

Following methods in AtlasMetricsUtil get the status of backend and index 
stores (HBase/Cassandra/Solr/ElasticSearch/..):
- getBackendStoreStatus()
- getIndexStoreStatus()

Consider adding isHealthy() method in AtlasMetricsUtil and call from here.


- Madhan Neethiraj


On Feb. 21, 2024, 11:13 p.m., Disha Talreja wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74902/
> ---
> 
> (Updated Feb. 21, 2024, 11:13 p.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Pinal Shah, and Radhika Kundam.
> 
> 
> Bugs: ATLAS-4826
> https://issues.apache.org/jira/browse/ATLAS-4826
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> In order to monitor the application health and be able to restart it if it is 
> unhealthy we need to check the:
> 1) Liveness - if Atlas is live and running as expected
> 2) Readyness - if Atlas is live and ready to accept client requests
> 
> 
> Diffs
> -
> 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/GraphDatabase.java
>  83f57d139 
>   graphdb/janus/pom.xml 80fe82bfd 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphDatabase.java
>  115b681cc 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
> 4d59fa3d6 
>   webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
> 5b16ba149 
> 
> 
> Diff: https://reviews.apache.org/r/74902/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Disha Talreja
> 
>



Re: Request to join Atlas project

2024-02-16 Thread Madhan Neethiraj
Priyanshi Shah,

I apologize for the delay in responding to your email. Thank you for your 
interest in contributing to Apache Atlas. You now have contributor access.

Welcome to Apache Atlas.

Thanks,
Madhan



On 2/16/24, 11:33 AM, "Priyanshi Shah" mailto:priyanshi.s...@freestoneinfotech.com>> wrote:


Hello Team,


A gentle reminder on the above requests.


Regards,
Priyanshi Shah


On Wed, Feb 7, 2024 at 6:58 PM Priyanshi Shah <
priyanshi.s...@freestoneinfotech.com 
<mailto:priyanshi.s...@freestoneinfotech.com>> wrote:


> Hello team,
>
> A gentle reminder on the above request.
>
> Regards,
> Priyanshi Shah
>
> On Wed, Nov 1, 2023 at 3:10 PM Priyanshi Shah <
> priyanshi.s...@freestoneinfotech.com 
> <mailto:priyanshi.s...@freestoneinfotech.com>> wrote:
>
>> Hi team,
>>
>> I would like to contribute to the Atlas project, Can you please add me as
>> contributor. My Apache id is : priyanshi.shah
>>
>> Regards,
>> Priyanshi Shah
>>
>






Re: Request to join Atlas project

2024-02-14 Thread Madhan Neethiraj
Brijesh Bhalala,

Thank you for your interest in contributing to Apache Atlas. You now have 
contributor access.

Welcome to Apache Atlas.

Thanks,
Madhan


On 2/14/24, 7:48 AM, "Brijesh Bhalala" mailto:brijeshbhalala2...@gmail.com>> wrote:


Hi team,


I would like to contribute to the Atlas project, Can you please add me as
contributor. My Apache id is : brijesh.bhalala


Regards,
Brijesh Bhalala






Re: Interested in contributing to Apache Atlas.

2024-02-02 Thread Madhan Neethiraj
Amruth S,

Thank you for your interest in contributing to Apache Atlas. You now have 
contributor access.
 
Welcome to Apache Atlas.
 
Thanks,
Madhan

On 1/26/24, 6:18 PM, "Amruth S" mailto:amrut...@cloudera.com.inva>LID> wrote:


Atlas Team,


I am interested in contributing to the Apache Atlas project. Can you please
add me as a contributor?


My Jira id is *amruth_s.*


Thanks,
Amruth S






[DISCUSS] merge RANGER-3923 branch into master branch

2024-01-02 Thread Madhan Neethiraj
Rangers,

 

For more than a year now, Apache Ranger community has been adding significant 
enhancements in RANGER-3923 branch. These enhancements enable business managers 
to manage access to datasets, instead of having data owners to manage access to 
individual resources like tables, columns, files, directories. In addition, 
this approach offers several benefits including:
reduced time for users in getting access to data across multiple services, with 
a single policy update.
business managers like project managers to manage access to datasets instead of 
multiple data owners, which reduces the time taken to get access to data.
separation of responsibilities: data owners focus on building datasets, while 
business managers focus on managing access to datasets.
eliminate the need to update access policies for any changes in datasets like 
add/remove/change resources.
 

Apache Ranger community has built APIs and UI to manage datasets and access to 
datasets. Apache Ranger policy engine has been updated as well to support 
datasets. I propose merging these enhancements into master branch this week, 
and work towards releasing Apache Ranger 3.0 version, by next month. Please 
share your feedback.

 

Looking forward to Apache Ranger 3.0 release!

 

Thanks,

Madhan

 



[DRAFT] [REPORT] Apache Atlas - December 2023

2023-12-13 Thread Madhan Neethiraj
Atlas community,

 

Please review the following draft report for this quarter.

 

Thanks,

Madhan

 

## Description:

The mission of Apache Atlas is the creation and maintenance of software related 

to a scalable and extensible set of core foundational governance services - 

enabling enterprises to effectively and efficiently meet their compliance 

requirements

 

## Project Status:

Current project status: Ongoing, with moderate activity

Issues for the board: There are no issues requiring board attention.

 

## Membership Data:

Apache Atlas was founded 2017-06-20 (6 years ago)

There are currently 48 committers and 32 PMC members in this project.

The Committer-to-PMC ratio is 3:2.

 

Community changes, past quarter:

- No new PMC members. Last addition was Ashutosh Mestry on 2019-04-16.

- No new committers. Last addition was Umesh Padashetty on 2023-06-14.

 

## Project Activity:

- enhancements to support aging of entity audits

- added bridge for Couchbase

- performance improvement in notification processing with optimization in

  fetching relationships

- removed references to Python2 with Python3

- improvements in dynamic index recovery

- fixes in Impala hook

- published Atlas client Python library 0.0.15

- updated versions of Okio, Apache Ivy

 

## Community Health:

- dev@atlas.apache.org had a 41% increase in traffic in the past quarter (247

  emails compared to 174)

- 17 issues opened in JIRA, past quarter (-48% change)

- 14 issues closed in JIRA, past quarter (-30% change)

- 33 commits in the past quarter (3% increase)

- 11 code contributors in the past quarter (22% increase)

- 2 PRs opened on GitHub, past quarter (-71% change)

- 4 PRs closed on GitHub, past quarter (no change)

 

## Recent releases:

Apache Atlas 2.3.0 was released on 2022-12-06.

Apache Atlas 2.2.0 was released on 2021-08-17.

Apache Atlas 2.1.0 was released on 2020-07-15.

 

 



[jira] [Resolved] (ATLAS-4815) Python client: incorrect method name in EntityClient.remove_classification()

2023-12-01 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-4815.
-
Fix Version/s: 2.4.0
   Resolution: Fixed

[Python client library 0.0.15|https://pypi.org/project/apache-atlas/0.0.15/] 
includes this fix. [@asik03 |https://github.com/asik03]- thank you for the fix.

> Python client: incorrect method name in EntityClient.remove_classification()
> 
>
> Key: ATLAS-4815
> URL: https://issues.apache.org/jira/browse/ATLAS-4815
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-intg
>    Reporter: Madhan Neethiraj
>Priority: Major
> Fix For: 2.4.0
>
>
> This Jira tracks pull request #232 https://github.com/apache/atlas/pull/232/.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (ATLAS-4815) Python client: incorrect method name in EntityClient.remove_classification()

2023-11-29 Thread Madhan Neethiraj (Jira)
Madhan Neethiraj created ATLAS-4815:
---

 Summary: Python client: incorrect method name in 
EntityClient.remove_classification()
 Key: ATLAS-4815
 URL: https://issues.apache.org/jira/browse/ATLAS-4815
 Project: Atlas
  Issue Type: Bug
  Components: atlas-intg
Reporter: Madhan Neethiraj


This Jira tracks pull request #232 https://github.com/apache/atlas/pull/232/.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] (ATLAS-4184) [Atlas] Need a way to perform bulk delete typedef when knox proxy is enabled

2023-11-22 Thread Madhan Neethiraj (Jira)


[ https://issues.apache.org/jira/browse/ATLAS-4184 ]


Madhan Neethiraj deleted comment on ATLAS-4184:
-

was (Author: jira-bot):
Commit 483ba3ea23c0b329a30b58460d4450f72f166e62 in atlas's branch 
refs/heads/master from Madhan Neethiraj
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=483ba3ea2 ]

Revert "ATLAS-4184: updated build/python scripts to replace references to 
python with python3"

incorrect JIRA number in commit message

This reverts commit 0f331aa63095bae010f2c4cb0a76e58eb5606e1b.


> [Atlas] Need a way to perform bulk delete typedef when knox proxy is enabled
> 
>
> Key: ATLAS-4184
> URL: https://issues.apache.org/jira/browse/ATLAS-4184
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: Dharshana M Krishnamoorthy
>Priority: Major
>
> While performing bulk delete operation on typedef, NullPointerException is 
> thrown
> {code:java}
>  2021-03-04 08:04:09,523 INFO  - [etp1020520290-753 - 
> a28b425d-7409-4891-b440-6cc41daf3a16:knox:DELETE/api/atlas/v2/types/typedefs] 
> ~ GraphTransaction intercept for 
> org.apache.atlas.repository.store.graph.v2.AtlasTypeDefGraphStoreV2.deleteTypesDef
>  (GraphTransactionAdvisor$1:41)
> 2021-03-04 08:04:09,526 ERROR - [etp1020520290-753 - 
> a28b425d-7409-4891-b440-6cc41daf3a16:knox:DELETE/api/atlas/v2/types/typedefs] 
> ~ graph rollback due to exception  (GraphTransactionInterceptor:192)
> java.lang.NullPointerException
> at 
> org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.deleteTypesDef(AtlasTypeDefGraphStore.java:565)
> at 
> org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$$FastClassBySpringCGLIB$$5226c80b.invoke()
> at 
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:737)
> at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
> at 
> org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540)
> at 
> org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395)
> at 
> org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
> at 
> org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
> at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
> at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
> at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
> at 
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2021-03-04 08:04:03,989 INFO  - [etp1020520290-753:] ~ Logged into Atlas as = 
> HTTP (AtlasAuthenticationFilter$KerberosFilterChainWrapper:726){code}
> Delete call body:
> {code:java}
> {"businessMetadataDefs": [{"name": "bm_xvsym_1"}, {"name": "bm_xvsym_2"}]} 
> {code}
> This is used to perform bulk create and perform bulk delete.
> While knox proxy is enabled, NullPointerException is thrown
> Need an alternate way to perform bulk delete with knox enabled



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] (ATLAS-4184) [Atlas] Need a way to perform bulk delete typedef when knox proxy is enabled

2023-11-22 Thread Madhan Neethiraj (Jira)


[ https://issues.apache.org/jira/browse/ATLAS-4184 ]


Madhan Neethiraj deleted comment on ATLAS-4184:
-

was (Author: jira-bot):
Commit 87fb3a4a4e3eae1b89909ac052139be1e0f164ee in atlas's branch 
refs/heads/branch-2.0 from Madhan Neethiraj
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=87fb3a4a4 ]

ATLAS-4184: updated build/python scripts to replace references to python with 
python3

(cherry picked from commit 0f331aa63095bae010f2c4cb0a76e58eb5606e1b)


> [Atlas] Need a way to perform bulk delete typedef when knox proxy is enabled
> 
>
> Key: ATLAS-4184
> URL: https://issues.apache.org/jira/browse/ATLAS-4184
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: Dharshana M Krishnamoorthy
>Priority: Major
>
> While performing bulk delete operation on typedef, NullPointerException is 
> thrown
> {code:java}
>  2021-03-04 08:04:09,523 INFO  - [etp1020520290-753 - 
> a28b425d-7409-4891-b440-6cc41daf3a16:knox:DELETE/api/atlas/v2/types/typedefs] 
> ~ GraphTransaction intercept for 
> org.apache.atlas.repository.store.graph.v2.AtlasTypeDefGraphStoreV2.deleteTypesDef
>  (GraphTransactionAdvisor$1:41)
> 2021-03-04 08:04:09,526 ERROR - [etp1020520290-753 - 
> a28b425d-7409-4891-b440-6cc41daf3a16:knox:DELETE/api/atlas/v2/types/typedefs] 
> ~ graph rollback due to exception  (GraphTransactionInterceptor:192)
> java.lang.NullPointerException
> at 
> org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.deleteTypesDef(AtlasTypeDefGraphStore.java:565)
> at 
> org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$$FastClassBySpringCGLIB$$5226c80b.invoke()
> at 
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:737)
> at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
> at 
> org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540)
> at 
> org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395)
> at 
> org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
> at 
> org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
> at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
> at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
> at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
> at 
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2021-03-04 08:04:03,989 INFO  - [etp1020520290-753:] ~ Logged into Atlas as = 
> HTTP (AtlasAuthenticationFilter$KerberosFilterChainWrapper:726){code}
> Delete call body:
> {code:java}
> {"businessMetadataDefs": [{"name": "bm_xvsym_1"}, {"name": "bm_xvsym_2"}]} 
> {code}
> This is used to perform bulk create and perform bulk delete.
> While knox proxy is enabled, NullPointerException is thrown
> Need an alternate way to perform bulk delete with knox enabled



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] (ATLAS-4184) [Atlas] Need a way to perform bulk delete typedef when knox proxy is enabled

2023-11-22 Thread Madhan Neethiraj (Jira)


[ https://issues.apache.org/jira/browse/ATLAS-4184 ]


Madhan Neethiraj deleted comment on ATLAS-4184:
-

was (Author: jira-bot):
Commit 0f331aa63095bae010f2c4cb0a76e58eb5606e1b in atlas's branch 
refs/heads/master from Madhan Neethiraj
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=0f331aa63 ]

ATLAS-4184: updated build/python scripts to replace references to python with 
python3


> [Atlas] Need a way to perform bulk delete typedef when knox proxy is enabled
> 
>
> Key: ATLAS-4184
> URL: https://issues.apache.org/jira/browse/ATLAS-4184
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: Dharshana M Krishnamoorthy
>Priority: Major
>
> While performing bulk delete operation on typedef, NullPointerException is 
> thrown
> {code:java}
>  2021-03-04 08:04:09,523 INFO  - [etp1020520290-753 - 
> a28b425d-7409-4891-b440-6cc41daf3a16:knox:DELETE/api/atlas/v2/types/typedefs] 
> ~ GraphTransaction intercept for 
> org.apache.atlas.repository.store.graph.v2.AtlasTypeDefGraphStoreV2.deleteTypesDef
>  (GraphTransactionAdvisor$1:41)
> 2021-03-04 08:04:09,526 ERROR - [etp1020520290-753 - 
> a28b425d-7409-4891-b440-6cc41daf3a16:knox:DELETE/api/atlas/v2/types/typedefs] 
> ~ graph rollback due to exception  (GraphTransactionInterceptor:192)
> java.lang.NullPointerException
> at 
> org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.deleteTypesDef(AtlasTypeDefGraphStore.java:565)
> at 
> org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$$FastClassBySpringCGLIB$$5226c80b.invoke()
> at 
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:737)
> at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
> at 
> org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540)
> at 
> org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395)
> at 
> org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
> at 
> org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
> at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
> at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
> at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> at 
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
> at 
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2021-03-04 08:04:03,989 INFO  - [etp1020520290-753:] ~ Logged into Atlas as = 
> HTTP (AtlasAuthenticationFilter$KerberosFilterChainWrapper:726){code}
> Delete call body:
> {code:java}
> {"businessMetadataDefs": [{"name": "bm_xvsym_1"}, {"name": "bm_xvsym_2"}]} 
> {code}
> This is used to perform bulk create and perform bulk delete.
> While knox proxy is enabled, NullPointerException is thrown
> Need an alternate way to perform bulk delete with knox enabled



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (ATLAS-4814) CI build fails in distro module with error: Cannot run program "python"

2023-11-21 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-4814:

Attachment: ATLAS-4184.patch

> CI build fails in distro module with error: Cannot run program "python"
> ---
>
> Key: ATLAS-4814
> URL: https://issues.apache.org/jira/browse/ATLAS-4814
> Project: Atlas
>  Issue Type: Bug
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: ATLAS-4184.patch
>
>
> Atlas CI build (Atlas-master-IntegrationTests) fails with the following 
> error, which is likely due to the build machine not having python2 installed.
>  
> {noformat}
> [INFO] Apache Atlas Couchbase Bridge .. SUCCESS [ 24.697 
> s]
> [INFO] Apache Atlas Distribution .. FAILURE [ 9.103 s]
> [INFO] atlas-examples . SKIPPED
> [INFO] sample-app . SKIPPED
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 01:56 h
> [INFO] Finished at: 2023-11-21T20:44:37Z
> [INFO] 
> 
> Waiting for Jenkins to finish collecting data[ERROR] Failed to execute goal 
> org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (python-test) on project 
> atlas-distro: Command execution failed. Cannot run program "python" (in 
> directory 
> "/home/jenkins/jenkins-agent/workspace/Atlas/Atlas-master-IntegrationTests/distro/src/test/python"):
>  error=2, No such file or directory -> [Help 1][ERROR] [ERROR] To see the 
> full stack trace of the errors, re-run Maven with the -e switch.[ERROR] 
> Re-run Maven using the -X switch to enable full debug logging.[ERROR] [ERROR] 
> For more information about the errors and possible solutions, please read the 
> following articles:[ERROR] [Help 1] 
> [http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException][ERROR]
>  [ERROR] After correcting the problems, you can resume the build with the 
> command[ERROR] mvn  -rf :atlas-distrochannel stopped
> Exception: java.lang.LinkageError thrown from the UncaughtExceptionHandler in 
> thread "Thread-942"
> Finished: FAILURE
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Review Request 74750: ATLAS-4184: updated build/python scripts to replace references to python with python3

2023-11-21 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74750/
---

Review request for atlas.


Bugs: ATLAS-4184
https://issues.apache.org/jira/browse/ATLAS-4184


Repository: atlas


Description
---

- updated distro/pom.xml to replace reference to python with python3
- updated Docker setup to not install python2


Diffs
-

  dev-support/atlas-docker/Dockerfile.atlas-base 4aa225c16 
  distro/pom.xml 9df238eb3 
  distro/src/bin/atlas_admin.py 795a5716f 
  distro/src/bin/atlas_client_cmdline.py f05a3c8f9 
  distro/src/bin/atlas_config.py 31e6fd043 
  distro/src/bin/atlas_kafka_setup.py c13030e88 
  distro/src/bin/atlas_kafka_setup_hook.py cd8c0c3ca 
  distro/src/bin/atlas_start.py ffbe16271 
  distro/src/bin/atlas_stop.py b9cd63689 
  distro/src/bin/atlas_update_simple_auth_json.py 11c249d9b 
  distro/src/bin/cputil.py 8e99f95bc 
  distro/src/bin/quick_start.py 78517f865 
  distro/src/bin/quick_start_v1.py 1c5775a68 
  distro/src/test/mock/setup.py 3639df4a1 
  distro/src/test/python/scripts/TestMetadata.py b813f927f 
  distro/src/test/python/unitTests.py ec1a78cb9 
  tools/atlas-index-repair/src/main/resources/repair_index.py 0333dadca 
  tools/atlas-migration-exporter/atlas_migration_export.py e1b12f609 


Diff: https://reviews.apache.org/r/74750/diff/1/


Testing
---

- pre-commit tests run: 
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1516/
- successfully built and installed Atlas in docker containers having no 
Python2, using scripts under dev-support/atlas-docker
- verified quick_start.py
- verified Hive and HBase hooks reported entities from respective components
- verified Kafka bridge imported Kafka topics successfully


Thanks,

Madhan Neethiraj



[jira] [Updated] (ATLAS-4814) CI build fails in distro module with error: Cannot run program "python"

2023-11-21 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-4814:

Description: 
Atlas CI build (Atlas-master-IntegrationTests) fails with the following error, 
which is likely due to the build machine not having python2 installed.

 

{noformat}
[INFO] Apache Atlas Couchbase Bridge .. SUCCESS [ 24.697 s]
[INFO] Apache Atlas Distribution .. FAILURE [ 9.103 s]
[INFO] atlas-examples . SKIPPED
[INFO] sample-app . SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 01:56 h
[INFO] Finished at: 2023-11-21T20:44:37Z
[INFO] 
Waiting for Jenkins to finish collecting data[ERROR] Failed to execute goal 
org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (python-test) on project 
atlas-distro: Command execution failed. Cannot run program "python" (in 
directory 
"/home/jenkins/jenkins-agent/workspace/Atlas/Atlas-master-IntegrationTests/distro/src/test/python"):
 error=2, No such file or directory -> [Help 1][ERROR] [ERROR] To see the full 
stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven 
using the -X switch to enable full debug logging.[ERROR] [ERROR] For more 
information about the errors and possible solutions, please read the following 
articles:[ERROR] [Help 1] 
[http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException][ERROR]
 [ERROR] After correcting the problems, you can resume the build with the 
command[ERROR] mvn  -rf :atlas-distrochannel stopped

Exception: java.lang.LinkageError thrown from the UncaughtExceptionHandler in 
thread "Thread-942"
Finished: FAILURE
{noformat}

  was:
Atlas CI build (Atlas-master-IntegrationTests) fails with the following error, 
which is likely due to the build machine not having python2 installed.

 
[INFO] Apache Atlas Couchbase Bridge .. SUCCESS [ 24.697 s]
[INFO] Apache Atlas Distribution .. FAILURE [  9.103 s]
[INFO] atlas-examples . SKIPPED
[INFO] sample-app . SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time:  01:56 h
[INFO] Finished at: 2023-11-21T20:44:37Z
[INFO] 
Waiting for Jenkins to finish collecting data[ERROR] Failed to execute goal 
org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (python-test) on project 
atlas-distro: Command execution failed. Cannot run program "python" (in 
directory 
"/home/jenkins/jenkins-agent/workspace/Atlas/Atlas-master-IntegrationTests/distro/src/test/python"):
 error=2, No such file or directory -> [Help 1][ERROR] [ERROR] To see the full 
stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven 
using the -X switch to enable full debug logging.[ERROR] [ERROR] For more 
information about the errors and possible solutions, please read the following 
articles:[ERROR] [Help 1] 
[http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException][ERROR]
 [ERROR] After correcting the problems, you can resume the build with the 
command[ERROR]   mvn  -rf :atlas-distrochannel stopped

Exception: java.lang.LinkageError thrown from the UncaughtExceptionHandler in 
thread "Thread-942"
Finished: FAILURE


> CI build fails in distro module with error: Cannot run program "python"
> ---
>
> Key: ATLAS-4814
> URL: https://issues.apache.org/jira/browse/ATLAS-4814
> Project: Atlas
>  Issue Type: Bug
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
>
> Atlas CI build (Atlas-master-IntegrationTests) fails with the following 
> error, which is likely due to the build machine not having python2 installed.
>  
> {noformat}
> [INFO] Apache Atlas Couchbase Bridge .. SUCCESS [ 24.697 
> s]
> [INFO] Apache Atlas Distribution .. FAILURE [ 9.103 s]
> [INFO] atlas-examples . SKIPPED
> [INFO] sample-app . SKIPPED
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> -

[jira] [Created] (ATLAS-4814) CI build fails in distro module with error: Cannot run program "python"

2023-11-21 Thread Madhan Neethiraj (Jira)
Madhan Neethiraj created ATLAS-4814:
---

 Summary: CI build fails in distro module with error: Cannot run 
program "python"
 Key: ATLAS-4814
 URL: https://issues.apache.org/jira/browse/ATLAS-4814
 Project: Atlas
  Issue Type: Bug
Reporter: Madhan Neethiraj
Assignee: Madhan Neethiraj


Atlas CI build (Atlas-master-IntegrationTests) fails with the following error, 
which is likely due to the build machine not having python2 installed.

 
[INFO] Apache Atlas Couchbase Bridge .. SUCCESS [ 24.697 s]
[INFO] Apache Atlas Distribution .. FAILURE [  9.103 s]
[INFO] atlas-examples . SKIPPED
[INFO] sample-app . SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time:  01:56 h
[INFO] Finished at: 2023-11-21T20:44:37Z
[INFO] 
Waiting for Jenkins to finish collecting data[ERROR] Failed to execute goal 
org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (python-test) on project 
atlas-distro: Command execution failed. Cannot run program "python" (in 
directory 
"/home/jenkins/jenkins-agent/workspace/Atlas/Atlas-master-IntegrationTests/distro/src/test/python"):
 error=2, No such file or directory -> [Help 1][ERROR] [ERROR] To see the full 
stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven 
using the -X switch to enable full debug logging.[ERROR] [ERROR] For more 
information about the errors and possible solutions, please read the following 
articles:[ERROR] [Help 1] 
[http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException][ERROR]
 [ERROR] After correcting the problems, you can resume the build with the 
command[ERROR]   mvn  -rf :atlas-distrochannel stopped

Exception: java.lang.LinkageError thrown from the UncaughtExceptionHandler in 
thread "Thread-942"
Finished: FAILURE



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (ATLAS-4789) Add Couchbase bridge

2023-11-02 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-4789.
-
Resolution: Fixed

[~chedim]  - thank you for the contribution!

> Add Couchbase bridge
> 
>
> Key: ATLAS-4789
> URL: https://issues.apache.org/jira/browse/ATLAS-4789
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Reporter: Dmitrii Chechetkin
>Assignee: Dmitrii Chechetkin
>Priority: Major
> Attachments: ATLAS-4789.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> We at couchbase developed a bridge for Atlas that connectes to our database 
> and decided to release it into open-source. Needed to create the ticket to 
> open the pull request.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (ATLAS-4789) Add Couchbase bridge

2023-11-02 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj reassigned ATLAS-4789:
---

Assignee: Dmitrii Chechetkin

> Add Couchbase bridge
> 
>
> Key: ATLAS-4789
> URL: https://issues.apache.org/jira/browse/ATLAS-4789
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Reporter: Dmitrii Chechetkin
>Assignee: Dmitrii Chechetkin
>Priority: Major
> Attachments: ATLAS-4789.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> We at couchbase developed a bridge for Atlas that connectes to our database 
> and decided to release it into open-source. Needed to create the ticket to 
> open the pull request.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: Review Request 74706: Adds a Couchbase hook for Apache Atlas

2023-11-02 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74706/#review225930
---


Ship it!




Ship It!

- Madhan Neethiraj


On Nov. 1, 2023, 3:24 p.m., Dmitrii Chechetkin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74706/
> ---
> 
> (Updated Nov. 1, 2023, 3:24 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> We at couchbase developed a bridge for Atlas that connectes to our database 
> using DCP, analyzes metadata for documents that are being stored/updated on 
> the cluster and then sends the extracted metadata onto an Atlas instance. 
> Additional documentation on the bridge is provided in 
> `docs/src/documents/Hook/HookCouchbase.md`
> 
> Thank you for reviewing!
> 
> 
> Diffs
> -
> 
>   addons/couchbase-bridge/pom.xml PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/AtlasConfig.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/CBConfig.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/CouchbaseHook.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseAtlasEntity.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseBucket.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseCluster.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseCollection.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseField.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseFieldType.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseScope.java
>  PRE-CREATION 
>   addons/couchbase-bridge/src/main/resources/atlas-application.properties 
> PRE-CREATION 
>   
> addons/couchbase-bridge/src/test/java/com/couchbase/atlas/connector/CouchbaseHookTest.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/test/java/com/couchbase/atlas/connector/entities/CouchbaseAtlasEntityTest.java
>  PRE-CREATION 
>   addons/couchbase-bridge/src/test/resources/log4j.xml PRE-CREATION 
>   addons/models/5000-Couchbase/5020-couchbase_model.json PRE-CREATION 
>   distro/pom.xml ed477dfbb 
>   distro/src/main/assemblies/atlas-couchbase-hook-package.xml PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 3e2ca1c39 
>   docs/src/documents/Hook/HookCouchbase.md PRE-CREATION 
>   pom.xml f76d0ea04 
> 
> 
> Diff: https://reviews.apache.org/r/74706/diff/2/
> 
> 
> Testing
> ---
> 
> Testing done manually and there's also a couple of integration tests.
> 
> 
> Thanks,
> 
> Dmitrii Chechetkin
> 
>



Re: Review Request 74706: Adds a Couchbase hook for Apache Atlas

2023-10-30 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74706/#review225907
---




addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/TypeGenerator.java
Lines 43 (patched)
<https://reviews.apache.org/r/74706/#comment314316>

Generating typedefs dynamically and adding them in Atlas would require 
Atlas admin privileges. Did you consider creating a model json file and adding 
to  https://github.com/apache/atlas/tree/master/addons/models? This will cause 
Atlas server to create Couchbase typedefs during startup.


- Madhan Neethiraj


On Oct. 30, 2023, 7:44 p.m., Dmitrii Chechetkin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74706/
> ---
> 
> (Updated Oct. 30, 2023, 7:44 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> We at couchbase developed a bridge for Atlas that connectes to our database 
> using DCP, analyzes metadata for documents that are being stored/updated on 
> the cluster and then sends the extracted metadata onto an Atlas instance. 
> Additional documentation on the bridge is provided in 
> `docs/src/documents/Hook/HookCouchbase.md`
> 
> 
> Diffs
> -
> 
>   addons/couchbase-bridge/pom.xml PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/AtlasConfig.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/CBConfig.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/CouchbaseHook.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/TypeGenerator.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseAtlasEntity.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseBucket.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseCluster.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseCollection.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseField.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseFieldType.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseScope.java
>  PRE-CREATION 
>   addons/couchbase-bridge/src/main/resources/atlas-application.properties 
> PRE-CREATION 
>   
> addons/couchbase-bridge/src/test/java/com/couchbase/atlas/connector/CouchbaseHookTest.java
>  PRE-CREATION 
>   
> addons/couchbase-bridge/src/test/java/com/couchbase/atlas/connector/entities/CouchbaseAtlasEntityTest.java
>  PRE-CREATION 
>   addons/couchbase-bridge/src/test/resources/log4j.xml PRE-CREATION 
>   distro/pom.xml ed477dfbb 
>   distro/src/main/assemblies/atlas-couchbase-hook-package.xml PRE-CREATION 
>   distro/src/main/assemblies/standalone-package.xml 3e2ca1c39 
>   docs/src/documents/Hook/HookCouchbase.md PRE-CREATION 
>   pom.xml f76d0ea04 
> 
> 
> Diff: https://reviews.apache.org/r/74706/diff/1/
> 
> 
> Testing
> ---
> 
> Testing done manually and there's also a couple of integration tests.
> 
> 
> Thanks,
> 
> Dmitrii Chechetkin
> 
>



[jira] [Assigned] (ATLAS-4783) Atlas fails to start if indexing backend isn't set to Solr

2023-09-11 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj reassigned ATLAS-4783:
---

Fix Version/s: 3.0.0
   2.4.0
 Assignee: Szymon Orzechowski
   Resolution: Fixed

[~szyorz]  - thank you for the fix!

> Atlas fails to start if indexing backend isn't set to Solr
> --
>
> Key: ATLAS-4783
> URL: https://issues.apache.org/jira/browse/ATLAS-4783
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-intg
>Affects Versions: 3.0.0, 2.3.0
>Reporter: Szymon Orzechowski
>Assignee: Szymon Orzechowski
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
> Attachments: ATLAS-4783.patch
>
>
> If atlas.graph.index.search.backend is set to elasticsearch (which is in tech 
> preview) and atlas.graph.index.search.solr.wait-searcher isn't set then Atlas 
> fails to start and errors with
> {code:java}
> Caused by: java.util.NoSuchElementException: 
> 'atlas.graph.index.search.solr.wait-searcher' doesn't map to an existing 
> object
>         at 
> org.apache.commons.configuration.AbstractConfiguration.getBoolean(AbstractConfiguration.java:644)
>  ~[?:?]
>         at 
> org.apache.atlas.ApplicationProperties.setDefaults(ApplicationProperties.java:374)
>  ~[?:?]
>         at 
> org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:146) 
> ~[?:?]
>         ... 35 more
>  {code}
> This property shouldn't be required in atlas-application.properties when 
> elasticsearch is used



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: Review Request 74545: ATLAS-4783: Fix NoSuchElementException when running different indexing backend than Solr

2023-09-11 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74545/#review225737
---


Ship it!




Ship It!

- Madhan Neethiraj


On Aug. 17, 2023, 8:49 a.m., Szymon Orzechowski wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74545/
> ---
> 
> (Updated Aug. 17, 2023, 8:49 a.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Pinal Shah, Prasad Pawar, and 
> Radhika Kundam.
> 
> 
> Bugs: ATLAS-4783
> https://issues.apache.org/jira/browse/ATLAS-4783
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Elasticsearch is in tech preview as an alternative to Solr as indexing 
> backend. However current Atlas code causes NoSuchElementException when 
> property atlas.graph.index.search.solr.wait-searcher is not set. This however 
> shouldn't be a requirement when not using Solr. This patch fixes that.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/ApplicationProperties.java f83dff699 
> 
> 
> Diff: https://reviews.apache.org/r/74545/diff/1/
> 
> 
> Testing
> ---
> 
> Ran mvn clean test and tested on my cluster with Elasticsearch
> 
> 
> Thanks,
> 
> Szymon Orzechowski
> 
>



Review Request 74596: ATLAS-4783: fix to not require Solr configuration when index-backend is Elasticsearch

2023-09-11 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74596/
---

Review request for atlas, Ashutosh Mestry, Nikhil Bonte, Nixon Rodrigues, Pinal 
Shah, Radhika Kundam, Sarath Subramanian, and Sidharth Mishra.


Bugs: ATLAS-4783
https://issues.apache.org/jira/browse/ATLAS-4783


Repository: atlas


Description
---

moved the LOG statement that refers SOLR_WAIT_SEARCHER_CONF to inside the block 
handling Solr index backend


Diffs
-

  intg/src/main/java/org/apache/atlas/ApplicationProperties.java f83dff699 


Diff: https://reviews.apache.org/r/74596/diff/1/


Testing
---

- verified that Atlas server starts successfully with Solr index backend
- TODO: validation of Atlas server startup with Elasticsearch index backend


Thanks,

Madhan Neethiraj



Review Request 74587: ATLAS-4795: docker setup updated to support Apple M1

2023-09-08 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74587/
---

Review request for atlas, Ashutosh Mestry, Nikhil Bonte, Nixon Rodrigues, 
Radhika Kundam, Sarath Subramanian, Sharmadha Sainath, and Sidharth Mishra.


Bugs: ATLAS-4795
https://issues.apache.org/jira/browse/ATLAS-4795


Repository: atlas


Description
---

- updated docker setup scripts to use TARGETARCH
- added support for building and runing with JDK8 or JDK11


Diffs
-

  dev-support/atlas-docker/.env 440be2a46 
  dev-support/atlas-docker/Dockerfile 7bd79651f 
  dev-support/atlas-docker/Dockerfile.atlas 888779133 
  dev-support/atlas-docker/Dockerfile.atlas-base e68daa69d 
  dev-support/atlas-docker/Dockerfile.atlas-build 4b381c1d9 
  dev-support/atlas-docker/docker-compose.atlas-base.yml 349c306d6 
  dev-support/atlas-docker/docker-compose.atlas-build.yml d2ec7bda5 
  dev-support/atlas-docker/docker-compose.atlas.yml aeb706e59 
  dev-support/atlas-docker/scripts/atlas-build.sh 592a055f7 


Diff: https://reviews.apache.org/r/74587/diff/1/


Testing
---

- verified that Atlas builds and runs successfully via docker setup scripts in 
MacBook Pro with Apple M1


Thanks,

Madhan Neethiraj



[jira] [Created] (ATLAS-4795) Docker setup: support Apple M1 chip env

2023-09-08 Thread Madhan Neethiraj (Jira)
Madhan Neethiraj created ATLAS-4795:
---

 Summary: Docker setup: support Apple M1 chip env
 Key: ATLAS-4795
 URL: https://issues.apache.org/jira/browse/ATLAS-4795
 Project: Atlas
  Issue Type: Improvement
  Components:  atlas-core
Reporter: Madhan Neethiraj
Assignee: Madhan Neethiraj


Docker setup scripts to build and run Atlas fails in environments that use 
Apple M1 chip:
 
{noformat}
atlas-base exited with code 0
atlas-build  | Building from /home/atlas/src
atlas-build  | The JAVA_HOME environment variable is not defined correctly
atlas-build  | This environment variable is needed to run this program
atlas-build  | NB: JAVA_HOME should point to a JDK not a JRE
 {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (ATLAS-4794) HBase hook initialization failure: java.lang.ClassNotFoundException: org.apache.logging.log4j.core.Layout

2023-09-08 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-4794:

Description: 
Atlas Hbase hook fails during initialization with the following error in 
hbase-hbase-master-atlas-hbase.example.com.log:
{noformat}
2023-09-08 19:12:04,599 ERROR [master/atlas-hbase:16000:becomeActiveMaster] 
coprocessor.CoprocessorHost: The coprocessor 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor threw java.io.IOException: 
java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
java.io.IOException: java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
at 
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:280)
at 
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:169)
at 
org.apache.hadoop.hbase.master.MasterCoprocessorHost.(MasterCoprocessorHost.java:157)
at 
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1011)
at 
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2239)
at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:621)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
at 
org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:191)
at 
org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:71)
at 
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:274)
... 6 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at 
org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:178)
... 8 more
Caused by: java.lang.NoClassDefFoundError: org/apache/logging/log4j/core/Layout
at org.apache.atlas.hook.AtlasHook.(AtlasHook.java:96)
at 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.(HBaseAtlasCoprocessor.java:46)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.init(HBaseAtlasCoprocessor.java:80)
at 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.(HBaseAtlasCoprocessor.java:60)
... 13 more
Caused by: java.lang.ClassNotFoundException: 
org.apache.logging.log4j.core.Layout
at java.lang.ClassLoader.findClass(ClassLoader.java:523)
at 
org.apache.atlas.plugin.classloader.AtlasPluginClassLoader$MyClassLoader.findClass(AtlasPluginClassLoader.java:347)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at 
org.apache.atlas.plugin.classloader.AtlasPluginClassLoader.loadClass(AtlasPluginClassLoader.java:144)
... 22 more
2023-09-08 19:12:04,601 ERROR [master/atlas-hbase:16000:becomeActiveMaster] 
master.HMaster: * ABORTING master 
atlas-hbase.example.com,16000,1694200321790: The coprocessor 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor threw java.io.IOException: 
java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor *
java.io.IOException: java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
{noformat}

  was:
Atlas docker setup fails in HBase startup with the following error in 
hbase-hbase-master-atlas-hbase.example.com.log:
{noformat}
2023-09-08 19:12:04,599 ERROR [master/atlas-hbase:16000:becomeActiveMaster] 
coprocessor.CoprocessorHost: The coprocessor 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor threw java.io.IOException: 
java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
java.io.IOException: java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
at 
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:280

[jira] [Updated] (ATLAS-4794) HBase hook initialization failure: java.lang.ClassNotFoundException: org.apache.logging.log4j.core.Layout

2023-09-08 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-4794:

Summary: HBase hook initialization failure: 
java.lang.ClassNotFoundException: org.apache.logging.log4j.core.Layout  (was: 
Atlas docker setup: HBase fails to startup with error 
java.lang.ClassNotFoundException: org.apache.logging.log4j.core.Layout)

> HBase hook initialization failure: java.lang.ClassNotFoundException: 
> org.apache.logging.log4j.core.Layout
> -
>
> Key: ATLAS-4794
> URL: https://issues.apache.org/jira/browse/ATLAS-4794
> Project: Atlas
>  Issue Type: Bug
>    Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: ATLAS-4794.patch
>
>
> Atlas docker setup fails in HBase startup with the following error in 
> hbase-hbase-master-atlas-hbase.example.com.log:
> {noformat}
> 2023-09-08 19:12:04,599 ERROR [master/atlas-hbase:16000:becomeActiveMaster] 
> coprocessor.CoprocessorHost: The coprocessor 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor threw java.io.IOException: 
> java.lang.InstantiationException: 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
> java.io.IOException: java.lang.InstantiationException: 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:280)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:169)
> at 
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.(MasterCoprocessorHost.java:157)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1011)
> at 
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2239)
> at 
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:621)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: java.lang.InstantiationException: 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
> at 
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:191)
> at 
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:71)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:274)
> ... 6 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
> Method)
> at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at 
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:178)
> ... 8 more
> Caused by: java.lang.NoClassDefFoundError: 
> org/apache/logging/log4j/core/Layout
> at org.apache.atlas.hook.AtlasHook.(AtlasHook.java:96)
> at 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.(HBaseAtlasCoprocessor.java:46)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
> Method)
> at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at java.lang.Class.newInstance(Class.java:442)
> at 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.init(HBaseAtlasCoprocessor.java:80)
> at 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.(HBaseAtlasCoprocessor.java:60)
> ... 13 more
> Caused by: java.lang.ClassNotFoundException: 
> org.apache.logging.log4j.core.Layout
> at java.lang.ClassLoader.findClass(ClassLoader.java:523)
> at 
> org.apache.atlas.plugin.classloader.AtlasPluginClassLoader$MyClassLoader.findClass(AtlasPluginClassLoader.java:347)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at 
> org.apache.atlas.plugin.classloader.AtlasPluginC

Review Request 74586: ATLAS-4794: fixed initialization failure in HBase hook

2023-09-08 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74586/
---

Review request for atlas, Ashutosh Mestry, Nikhil Bonte, Nixon Rodrigues, 
Radhika Kundam, Sarath Subramanian, Sharmadha Sainath, Sidharth Mishra, and 
Umesh Padashetty.


Bugs: ATLAS-4794
https://issues.apache.org/jira/browse/ATLAS-4794


Repository: atlas


Description
---

- updated HBase hook packaging to include log4j-api and log4j-core libraries


Diffs
-

  addons/hbase-bridge/pom.xml d78abbb37 


Diff: https://reviews.apache.org/r/74586/diff/1/


Testing
---

- verified that HBase hooks initializes and works successfully with the fix


Thanks,

Madhan Neethiraj



[jira] [Assigned] (ATLAS-4794) Atlas docker setup: HBase fails to startup with error java.lang.ClassNotFoundException: org.apache.logging.log4j.core.Layout

2023-09-08 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj reassigned ATLAS-4794:
---

Assignee: Madhan Neethiraj

> Atlas docker setup: HBase fails to startup with error 
> java.lang.ClassNotFoundException: org.apache.logging.log4j.core.Layout
> 
>
> Key: ATLAS-4794
> URL: https://issues.apache.org/jira/browse/ATLAS-4794
> Project: Atlas
>  Issue Type: Bug
>    Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
>
> Atlas docker setup fails in HBase startup with the following error in 
> hbase-hbase-master-atlas-hbase.example.com.log:
> {noformat}
> 2023-09-08 19:12:04,599 ERROR [master/atlas-hbase:16000:becomeActiveMaster] 
> coprocessor.CoprocessorHost: The coprocessor 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor threw java.io.IOException: 
> java.lang.InstantiationException: 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
> java.io.IOException: java.lang.InstantiationException: 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:280)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:169)
> at 
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.(MasterCoprocessorHost.java:157)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1011)
> at 
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2239)
> at 
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:621)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: java.lang.InstantiationException: 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
> at 
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:191)
> at 
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:71)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:274)
> ... 6 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
> Method)
> at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at 
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:178)
> ... 8 more
> Caused by: java.lang.NoClassDefFoundError: 
> org/apache/logging/log4j/core/Layout
> at org.apache.atlas.hook.AtlasHook.(AtlasHook.java:96)
> at 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.(HBaseAtlasCoprocessor.java:46)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
> Method)
> at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at java.lang.Class.newInstance(Class.java:442)
> at 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.init(HBaseAtlasCoprocessor.java:80)
> at 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.(HBaseAtlasCoprocessor.java:60)
> ... 13 more
> Caused by: java.lang.ClassNotFoundException: 
> org.apache.logging.log4j.core.Layout
> at java.lang.ClassLoader.findClass(ClassLoader.java:523)
> at 
> org.apache.atlas.plugin.classloader.AtlasPluginClassLoader$MyClassLoader.findClass(AtlasPluginClassLoader.java:347)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at 
> org.apache.atlas.plugin.classloader.AtlasPluginClassLoader.loadClass(AtlasPluginClassLoader.java:144)
> ... 22 more
> 2023-09-08 19:12:04,601 ERROR [master/atlas-hbase:16000:becomeActiveMaster] 
> master.HMaster: * ABORTING master 
> atlas-hbase.example.com,16000,1694

[jira] [Updated] (ATLAS-4794) Atlas docker setup: HBase fails to startup with error java.lang.ClassNotFoundException: org.apache.logging.log4j.core.Layout

2023-09-08 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-4794:

Description: 
Atlas docker setup fails in HBase startup with the following error in 
hbase-hbase-master-atlas-hbase.example.com.log:
{noformat}
2023-09-08 19:12:04,599 ERROR [master/atlas-hbase:16000:becomeActiveMaster] 
coprocessor.CoprocessorHost: The coprocessor 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor threw java.io.IOException: 
java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
java.io.IOException: java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
at 
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:280)
at 
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:169)
at 
org.apache.hadoop.hbase.master.MasterCoprocessorHost.(MasterCoprocessorHost.java:157)
at 
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1011)
at 
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2239)
at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:621)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
at 
org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:191)
at 
org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:71)
at 
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:274)
... 6 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at 
org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:178)
... 8 more
Caused by: java.lang.NoClassDefFoundError: org/apache/logging/log4j/core/Layout
at org.apache.atlas.hook.AtlasHook.(AtlasHook.java:96)
at 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.(HBaseAtlasCoprocessor.java:46)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.init(HBaseAtlasCoprocessor.java:80)
at 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.(HBaseAtlasCoprocessor.java:60)
... 13 more
Caused by: java.lang.ClassNotFoundException: 
org.apache.logging.log4j.core.Layout
at java.lang.ClassLoader.findClass(ClassLoader.java:523)
at 
org.apache.atlas.plugin.classloader.AtlasPluginClassLoader$MyClassLoader.findClass(AtlasPluginClassLoader.java:347)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at 
org.apache.atlas.plugin.classloader.AtlasPluginClassLoader.loadClass(AtlasPluginClassLoader.java:144)
... 22 more
2023-09-08 19:12:04,601 ERROR [master/atlas-hbase:16000:becomeActiveMaster] 
master.HMaster: * ABORTING master 
atlas-hbase.example.com,16000,1694200321790: The coprocessor 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor threw java.io.IOException: 
java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor *
java.io.IOException: java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
{noformat}

  was:
Atlas from latest master branch fails during startup with following error:
{noformat}
2023-09-08 19:12:04,599 ERROR [master/atlas-hbase:16000:becomeActiveMaster] 
coprocessor.CoprocessorHost: The coprocessor 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor threw java.io.IOException: 
java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
java.io.IOException: java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
at 
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:280)
at 
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors

[jira] [Created] (ATLAS-4794) Atlas fails to startup with error: java.lang.ClassNotFoundException: org.apache.logging.log4j.core.Layout

2023-09-08 Thread Madhan Neethiraj (Jira)
Madhan Neethiraj created ATLAS-4794:
---

 Summary: Atlas fails to startup with error: 
java.lang.ClassNotFoundException: org.apache.logging.log4j.core.Layout
 Key: ATLAS-4794
 URL: https://issues.apache.org/jira/browse/ATLAS-4794
 Project: Atlas
  Issue Type: Bug
Reporter: Madhan Neethiraj


Atlas from latest master branch fails during startup with following error:
{noformat}
2023-09-08 19:12:04,599 ERROR [master/atlas-hbase:16000:becomeActiveMaster] 
coprocessor.CoprocessorHost: The coprocessor 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor threw java.io.IOException: 
java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
java.io.IOException: java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
at 
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:280)
at 
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:169)
at 
org.apache.hadoop.hbase.master.MasterCoprocessorHost.(MasterCoprocessorHost.java:157)
at 
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1011)
at 
org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2239)
at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:621)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
at 
org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:191)
at 
org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:71)
at 
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:274)
... 6 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at 
org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:178)
... 8 more
Caused by: java.lang.NoClassDefFoundError: org/apache/logging/log4j/core/Layout
at org.apache.atlas.hook.AtlasHook.(AtlasHook.java:96)
at 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.(HBaseAtlasCoprocessor.java:46)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.init(HBaseAtlasCoprocessor.java:80)
at 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.(HBaseAtlasCoprocessor.java:60)
... 13 more
Caused by: java.lang.ClassNotFoundException: 
org.apache.logging.log4j.core.Layout
at java.lang.ClassLoader.findClass(ClassLoader.java:523)
at 
org.apache.atlas.plugin.classloader.AtlasPluginClassLoader$MyClassLoader.findClass(AtlasPluginClassLoader.java:347)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at 
org.apache.atlas.plugin.classloader.AtlasPluginClassLoader.loadClass(AtlasPluginClassLoader.java:144)
... 22 more
2023-09-08 19:12:04,601 ERROR [master/atlas-hbase:16000:becomeActiveMaster] 
master.HMaster: * ABORTING master 
atlas-hbase.example.com,16000,1694200321790: The coprocessor 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor threw java.io.IOException: 
java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor *
java.io.IOException: java.lang.InstantiationException: 
org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
{noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (ATLAS-4794) Atlas docker setup: HBase fails to startup with error java.lang.ClassNotFoundException: org.apache.logging.log4j.core.Layout

2023-09-08 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-4794:

Summary: Atlas docker setup: HBase fails to startup with error 
java.lang.ClassNotFoundException: org.apache.logging.log4j.core.Layout  (was: 
Atlas fails to startup with error: java.lang.ClassNotFoundException: 
org.apache.logging.log4j.core.Layout)

> Atlas docker setup: HBase fails to startup with error 
> java.lang.ClassNotFoundException: org.apache.logging.log4j.core.Layout
> 
>
> Key: ATLAS-4794
> URL: https://issues.apache.org/jira/browse/ATLAS-4794
> Project: Atlas
>  Issue Type: Bug
>    Reporter: Madhan Neethiraj
>Priority: Major
>
> Atlas from latest master branch fails during startup with following error:
> {noformat}
> 2023-09-08 19:12:04,599 ERROR [master/atlas-hbase:16000:becomeActiveMaster] 
> coprocessor.CoprocessorHost: The coprocessor 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor threw java.io.IOException: 
> java.lang.InstantiationException: 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
> java.io.IOException: java.lang.InstantiationException: 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:280)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:169)
> at 
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.(MasterCoprocessorHost.java:157)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:1011)
> at 
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2239)
> at 
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:621)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: java.lang.InstantiationException: 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor
> at 
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:191)
> at 
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:71)
> at 
> org.apache.hadoop.hbase.coprocessor.CoprocessorHost.checkAndLoadInstance(CoprocessorHost.java:274)
> ... 6 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
> Method)
> at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at 
> org.apache.hadoop.hbase.master.MasterCoprocessorHost.checkAndGetInstance(MasterCoprocessorHost.java:178)
> ... 8 more
> Caused by: java.lang.NoClassDefFoundError: 
> org/apache/logging/log4j/core/Layout
> at org.apache.atlas.hook.AtlasHook.(AtlasHook.java:96)
> at 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.(HBaseAtlasCoprocessor.java:46)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
> Method)
> at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at java.lang.Class.newInstance(Class.java:442)
> at 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.init(HBaseAtlasCoprocessor.java:80)
> at 
> org.apache.atlas.hbase.hook.HBaseAtlasCoprocessor.(HBaseAtlasCoprocessor.java:60)
> ... 13 more
> Caused by: java.lang.ClassNotFoundException: 
> org.apache.logging.log4j.core.Layout
> at java.lang.ClassLoader.findClass(ClassLoader.java:523)
> at 
> org.apache.atlas.plugin.classloader.AtlasPluginClassLoader$MyClassLoader.findClass(AtlasPluginClassLoader.java:347)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> at 
> org.apache.atlas.plugin.classloader.AtlasPluginClassLoader.loadClass(AtlasPluginClassLoader.java:144)
> ... 22 more
> 2023-09-08 19:12:04,601 ERROR [master/atlas-hbase:

[jira] [Created] (ATLAS-4792) build failure in Apple M1 env

2023-09-07 Thread Madhan Neethiraj (Jira)
Madhan Neethiraj created ATLAS-4792:
---

 Summary: build failure in Apple M1 env
 Key: ATLAS-4792
 URL: https://issues.apache.org/jira/browse/ATLAS-4792
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Reporter: Madhan Neethiraj
Assignee: Madhan Neethiraj


Building Atlas master branch fails in MacBook Pro with Apple M1 chip, with the 
following:

{noformat}
[INFO] --- frontend-maven-plugin:1.4:install-node-and-npm (install node and 
npm) @ atlas-dashboardv2 ---
[INFO] Installing node version v12.16.0
[INFO] Downloading 
https://nodejs.org/dist/v12.16.0/node-v12.16.0-darwin-arm64.tar.gz to 
~/.m2/repository/com/github/eirslett/node/12.16.0/node-12.16.0-darwin-arm64.tar.gz
[INFO] No proxies configured
[INFO] No proxy was configured, downloading directly
[INFO] 
[INFO] Reactor Summary for apache-atlas 3.0.0-SNAPSHOT:
[INFO]
[INFO] Apache Atlas Server Build Tools  SUCCESS [  1.056 s]
[INFO] apache-atlas ... SUCCESS [  7.362 s]
[INFO] Apache Atlas Integration ... SUCCESS [ 22.299 s]
[INFO] Apache Atlas Test Utility Tools  SUCCESS [  8.182 s]
[INFO] Apache Atlas Common  SUCCESS [  9.194 s]
[INFO] Apache Atlas Client  SUCCESS [  0.410 s]
[INFO] atlas-client-common  SUCCESS [  3.962 s]
[INFO] atlas-client-v1  SUCCESS [  4.495 s]
[INFO] Apache Atlas Server API  SUCCESS [  6.009 s]
[INFO] atlas-client-v2  SUCCESS [  3.849 s]
[INFO] Apache Atlas Notification .. SUCCESS [ 11.231 s]
[INFO] Apache Atlas Graph Database Projects ... SUCCESS [  0.355 s]
[INFO] Apache Atlas Graph Database API  SUCCESS [  3.687 s]
[INFO] Graph Database Common Code . SUCCESS [  3.410 s]
[INFO] Apache Atlas JanusGraph-HBase2 Module .. SUCCESS [  4.272 s]
[INFO] Apache Atlas JanusGraph DB Impl  SUCCESS [ 17.223 s]
[INFO] Apache Atlas Graph Database Implementation Dependencies SUCCESS [  2.734 
s]
[INFO] Apache Atlas Authorization . SUCCESS [  7.162 s]
[INFO] Apache Atlas Repository  SUCCESS [ 38.694 s]
[INFO] Apache Atlas UI  FAILURE [ 35.820 s]
[INFO] Apache Atlas New UI  SKIPPED
[INFO] Apache Atlas Web Application ... SKIPPED
[INFO] Apache Atlas Documentation . SKIPPED
[INFO] Apache Atlas FileSystem Model .. SKIPPED
[INFO] Apache Atlas Plugin Classloader  SKIPPED
[INFO] Apache Atlas Hive Bridge Shim .. SKIPPED
[INFO] Apache Atlas Hive Bridge ... SKIPPED
[INFO] Apache Atlas Falcon Bridge Shim  SKIPPED
[INFO] Apache Atlas Falcon Bridge . SKIPPED
[INFO] Apache Atlas Sqoop Bridge Shim . SKIPPED
[INFO] Apache Atlas Sqoop Bridge .. SKIPPED
[INFO] Apache Atlas Storm Bridge Shim . SKIPPED
[INFO] Apache Atlas Storm Bridge .. SKIPPED
[INFO] Apache Atlas Hbase Bridge Shim . SKIPPED
[INFO] Apache Atlas Hbase Bridge .. SKIPPED
[INFO] Apache HBase - Testing Util  SKIPPED
[INFO] Apache Atlas Kafka Bridge .. SKIPPED
[INFO] Apache Atlas classification updater  SKIPPED
[INFO] Apache Atlas index repair tool . SKIPPED
[INFO] Apache Atlas Impala Hook API ... SKIPPED
[INFO] Apache Atlas Impala Bridge Shim  SKIPPED
[INFO] Apache Atlas Impala Bridge . SKIPPED
[INFO] Apache Atlas Distribution .. SKIPPED
[INFO] atlas-examples . SKIPPED
[INFO] sample-app . SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time:  03:12 min
[INFO] Finished at: 2023-09-07T20:39:20-07:00
[INFO] 
[ERROR] Failed to execute goal 
com.github.eirslett:frontend-maven-plugin:1.4:install-node-and-npm (install 
node and npm) on project atlas-dashboardv2: Could not download Node.js: Got 
error code 404 from the server. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Ma

[jira] [Updated] (ATLAS-4785) AtlasClientV2 label methods use incorrect API_V2

2023-09-03 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-4785:

Fix Version/s: 2.4.0

> AtlasClientV2 label methods use incorrect API_V2
> 
>
> Key: ATLAS-4785
> URL: https://issues.apache.org/jira/browse/ATLAS-4785
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Zhen Wang
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
> Attachments: image-2023-08-18-13-41-31-830.png
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> `AtlasClientV2.addLabels` by unique attribute method and 
> `AtlasClientV2.setLabels` by unique attribute method use incorrect `API_V2`.
> !image-2023-08-18-13-41-31-830.png!
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (ATLAS-4785) AtlasClientV2 label methods use incorrect API_V2

2023-09-03 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-4785.
-
Fix Version/s: 3.0.0
   Resolution: Fixed

> AtlasClientV2 label methods use incorrect API_V2
> 
>
> Key: ATLAS-4785
> URL: https://issues.apache.org/jira/browse/ATLAS-4785
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 2.3.0
>Reporter: Zhen Wang
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: image-2023-08-18-13-41-31-830.png
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> `AtlasClientV2.addLabels` by unique attribute method and 
> `AtlasClientV2.setLabels` by unique attribute method use incorrect `API_V2`.
> !image-2023-08-18-13-41-31-830.png!
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[DRAFT] [REPORT] Apache Atlas - June 2023

2023-06-14 Thread Madhan Neethiraj
Atlas community,

 

Please review the following draft report for this quarter.

 

Thanks,

Madhan

 

 

## Description:

The mission of Apache Atlas is the creation and maintenance of software related 

to a scalable and extensible set of core foundational governance services - 

enabling enterprises to effectively and efficiently meet their compliance 

requirements

 

## Project Status:

Current project status: Ongoing, with moderate activity

Issues for the board: There are no issues requiring board attention.

 

## Membership Data:

Apache Atlas was founded 2017-06-20 (6 years ago)

There are currently 48 committers and 32 PMC members in this project.

The Committer-to-PMC ratio is roughly 3:2.

 

Community changes, past quarter:

- No new PMC members. Last addition was Ashutosh Mestry on 2019-04-16.

- One new committers. Last addition was Umesh Padashetty on 2023-06-13.

 

## Project Activity:

- released Python client 0.0.14

- updated versions of Storm, JanusGraph, TinkerPop

- added APIs to download search results to file; updated UI to download results

- Hive hook fix in handling of notifications having an entity in both inputs

- support building Atlas using JDK versions higher than 8

- UI updates to remove unused libraries from build package

 

## Community Health:

- dev@atlas.apache.org had a 46% increase in traffic in the past quarter (226

  emails compared to 154)

- 29 issues opened in JIRA, past quarter (61% increase)

- 11 issues closed in JIRA, past quarter (no change)

- 33 commits in the past quarter (6% increase)

- 11 code contributors in the past quarter (57% increase)

 

## Most Recent releases:

2.3.0 was released on 2022-12-06

2.2.0 was released on 2021-08-17

2.1.0 was released on 2020-07-15

0.8.4 was released on 2019-06-21

 



[jira] [Resolved] (ATLAS-4759) Atlas Python client suppresses 503 error

2023-06-05 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-4759.
-
Fix Version/s: 3.0.0
   2.4.0
 Assignee: Maxim Martynov
   Resolution: Fixed

[~dolfinus]  - thank you for the fix. The patch is now merged in following 
branches:

 
{noformat}
commit b7849aeb78856f2f8e205c5e784cee84409cbc72 (HEAD -> master, origin/master, 
origin/HEAD)
Author: Maxim Martynov 
Date:   Mon Jun 5 12:33:35 2023 -0700

ATLAS-4759: Python client - do not suppress HTTP 503 (SERVICE_UNAVAILABLE)

Signed-off-by: Madhan Neethiraj 
{noformat}
 

 
{noformat}
commit b1d9450761de70adf9b3ecd35b5f62dc31d70489 (HEAD -> branch-2.0, 
origin/branch-2.0)
Author: Maxim Martynov 
Date:   Mon Jun 5 12:33:35 2023 -0700

ATLAS-4759: Python client - do not suppress HTTP 503 (SERVICE_UNAVAILABLE)

Signed-off-by: Madhan Neethiraj 
(cherry picked from commit b7849aeb78856f2f8e205c5e784cee84409cbc72)
{noformat}

> Atlas Python client suppresses 503 error
> 
>
> Key: ATLAS-4759
> URL: https://issues.apache.org/jira/browse/ATLAS-4759
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-intg
>Affects Versions: 2.1.0
>Reporter: Maxim Martynov
>Assignee: Maxim Martynov
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Currently if Python client for Apache Atlas gets HTTP 503 error code, it 
> always returns {{None}}:
> https://github.com/apache/atlas/blob/b19fba2ad96e3007af79d646ba17b6bd783d9156/intg/src/main/python/apache_atlas/client/base_client.py#L113-L116
> So if someone writes data to Atlas (e.g. update entity properties, save 
> lineage info), there is no distinction if {{None}} is returned because it is 
> supposed to do (e.g. 204 No content) or Atlas is not available.
> It is recommended to raise exception here.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (ATLAS-4658) base_client.py implementation is a bit messy

2023-06-05 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj reassigned ATLAS-4658:
---

Assignee: Maxim Martynov

> base_client.py implementation is a bit messy
> 
>
> Key: ATLAS-4658
> URL: https://issues.apache.org/jira/browse/ATLAS-4658
> Project: Atlas
>  Issue Type: Improvement
>  Components: atlas-intg
>Affects Versions: 2.1.0
>Reporter: Maxim Martynov
>Assignee: Maxim Martynov
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> atlas_client/client/base_client.py looks a bit messy:
> https://github.com/apache/atlas/blob/c1daaa44c3833be31c006753a2cf2f6f2e476cdc/intg/src/main/python/apache_atlas/client/base_client.py#L79-L120
>  # There is no need to make conditions with mapping between HTTPMethod enum 
> values and corresponding {{requests.Session}} methods - there is a 
> {{requests.Session.request}} method which accepts HTTP method as argument
>  # There is no need to use {{elif}} in conditions if previous clauses are 
> ended with {{return}} or {{raise}} statements
>  # There is not need to use {{logger.isEnabledFor(logging.DEBUG)}} before 
> calling {{{}logger.debug{}}}, this is already done in {{.debug}} method itself
>  # Use {{{}logging{}}}, not {{print}}
>  # logging.exception already shows full exception traceback, no need 
> to explicitly show add it into a logging message
>  
> I've creates pull request with a fix for all these items:
> https://github.com/apache/atlas/pull/158/files



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (ATLAS-4658) base_client.py implementation is a bit messy

2023-06-05 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-4658.
-
Fix Version/s: 3.0.0
   2.4.0
   Resolution: Fixed

[~dolfinus]  - thank you for the fix. The patch is now merged in following 
branches:

 
{noformat}
commit ad24235f492066378842cb0c1fbf9bde43da0391 (HEAD -> master, origin/master, 
origin/HEAD)
Author: Maxim Martynov 
Date:   Mon Jun 5 12:07:51 2023 -0700

ATLAS-4658: Python client - simply if clauses in AtlasClient.call_api

Signed-off-by: Madhan Neethiraj 
{noformat}
 
{noformat}
commit 8df8b41a5b9ce7410efe388f3184d109201080d4 (HEAD -> branch-2.0, 
origin/branch-2.0)
Author: Maxim Martynov 
Date:   Mon Jun 5 12:07:51 2023 -0700

ATLAS-4658: Python client - simply if clauses in AtlasClient.call_api

Signed-off-by: Madhan Neethiraj 
(cherry picked from commit ad24235f492066378842cb0c1fbf9bde43da0391)
{noformat}

> base_client.py implementation is a bit messy
> 
>
> Key: ATLAS-4658
> URL: https://issues.apache.org/jira/browse/ATLAS-4658
> Project: Atlas
>  Issue Type: Improvement
>  Components: atlas-intg
>Affects Versions: 2.1.0
>Reporter: Maxim Martynov
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> atlas_client/client/base_client.py looks a bit messy:
> https://github.com/apache/atlas/blob/c1daaa44c3833be31c006753a2cf2f6f2e476cdc/intg/src/main/python/apache_atlas/client/base_client.py#L79-L120
>  # There is no need to make conditions with mapping between HTTPMethod enum 
> values and corresponding {{requests.Session}} methods - there is a 
> {{requests.Session.request}} method which accepts HTTP method as argument
>  # There is no need to use {{elif}} in conditions if previous clauses are 
> ended with {{return}} or {{raise}} statements
>  # There is not need to use {{logger.isEnabledFor(logging.DEBUG)}} before 
> calling {{{}logger.debug{}}}, this is already done in {{.debug}} method itself
>  # Use {{{}logging{}}}, not {{print}}
>  # logging.exception already shows full exception traceback, no need 
> to explicitly show add it into a logging message
>  
> I've creates pull request with a fix for all these items:
> https://github.com/apache/atlas/pull/158/files



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: Handle Hive table data models in Atlas

2023-05-01 Thread Madhan Neethiraj
Hello Martin,

Primary keys and foreign keys are not captured currently by Atlas data model 
for Hive. It will require updates to the data model 
(https://github.com/apache/atlas/blob/master/addons/models/1000-Hadoop/1030-hive_model.json)
 and Hive hook to capture these details.

Can you please file a JIRA at https://issues.apache.org/jira/projects/ATLAS/ to 
track this requirement?

Thanks,
Madhan

On 4/12/23, 8:18 AM, "Martin Guerre" mailto:mgue...@zeenea.com.inva>LID> wrote:


Dear Apache Atlas community,


I am contacting you regarding a requirement for which I could not find any
documentation or JIRA issue for Atlas product. I would like to retrieve
Hive table data models (Primary Keys and Foreign Key) into Atlas through
Hive Hook.


Since Hive 2.1.0, it is possible to declare Primary Keys and Foreign Keys
for Hive tables so I assumed it would be retrievable by Atlas but it is not
the case.


- Are you aware of this limitation ?
- Is it planned to handle Hive data models in Atlas in the near or far
future ?


I am staying at your disposal for additional information if needed.


Best Regards,


Martin Guerre
Connectivity Team - Zeenea


*Nantes*






Re: Apache Atlas many to many relationship between entities

2023-04-17 Thread Madhan Neethiraj
Randheer,

Looking at the definition of relationship 'subscribe', it requires following 
types of entities at its ends:
- end1: client
- end2: application

However, the entity you are trying to create (of type 'service'), requires 
relationship definition with following ends:
- end1: service
- end2: application

Either the relationship definition, or type of entity you are createing needs 
update to be in sync.

Hope this helps.

Madhan




On 4/17/23, 9:08 PM, "Randheer K" mailto:randhee...@thoughtworks.com>> wrote:


Hi,
I'm trying to create many to many relationships between two entities, but
I'm getting the below error errorMessage': 'Relationship end is invalid.


> *Expected application but is NULL *


this is the relationship def that I created
{ "name": "subscribe", "typeName": "subscribe", "description": "Client
provided by the application", "category": "RELATIONSHIP", "typeVersion":
"1.0", "relationshipCategory": "ASSOCIATION", "propagateTags": "NONE",
"endDef1": { "name": "application", "type": "client", "cardinality": "SET",
"isContainer": false, "isLegacyAttribute": false }, "endDef2": { "name":
"client", "type": "application", "cardinality": "SET", "isContainer":
false, "isLegacyAttribute": false } }
and this is the json object that i used to create entity,
{ "entity": { "typeName": "service", "attributes": attributes,
"relationshipAttributes": { "application": {"typeName": "application",
"uniqueAttributes": {"id": application_id}} } } }


If anyone has experience with many to many relationships in apache atlas,
please let me know what Im doing wrong.


Thanks






Review Request 74389: ATLAS-4744: updated version of JanusGraph to 0.6.3 and TinkerPop to 3.5.5

2023-04-10 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74389/
---

Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nikhil Bonte, Nixon 
Rodrigues, Pinal Shah, Radhika Kundam, Sarath Subramanian, and Sidharth Mishra.


Bugs: ATLAS-4744
https://issues.apache.org/jira/browse/ATLAS-4744


Repository: atlas


Description
---

- JanusGraph version updated from 0.6.1 to 0.6.3
- TinkerPop version updated from 3.5.4 to 3.5.5


Diffs
-

  pom.xml 7077ddb23 


Diff: https://reviews.apache.org/r/74389/diff/1/


Testing
---

- verified all unittests and integration tests pass successfully
- quick_start.py completes successfully
- sanity testing of Hive, HBase hooks
- sanity testing of import-hbase and import-kafka utilities


Thanks,

Madhan Neethiraj



[jira] [Created] (ATLAS-4744) update JanusGraph version to 0.6.3

2023-04-10 Thread Madhan Neethiraj (Jira)
Madhan Neethiraj created ATLAS-4744:
---

 Summary: update JanusGraph version to 0.6.3
 Key: ATLAS-4744
 URL: https://issues.apache.org/jira/browse/ATLAS-4744
 Project: Atlas
  Issue Type: Improvement
  Components:  atlas-core
Reporter: Madhan Neethiraj
Assignee: Madhan Neethiraj


JanusGraph 0.6.3 has several fixes and improvements - 
https://github.com/JanusGraph/janusgraph/milestone/24?closed=1.

Atlas should be updated to use the latest available version of JanusGraph 
(0.6.3) and TinkerPop (3.5.5).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (ATLAS-4740) support build using higher version JDK, with 1.8 as target version

2023-04-05 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-4740.
-
Fix Version/s: 3.0.0
   2.4.0
   Resolution: Fixed

> support build using higher version JDK, with 1.8 as target version
> --
>
> Key: ATLAS-4740
> URL: https://issues.apache.org/jira/browse/ATLAS-4740
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>    Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
>
> Atlas requires JDK8 to build. Recent changes in infra, like Jenkins, require 
> using JDK11 or above to build. Building with JDK8 results in the failure 
> given below.
>  
> To address this issue, Atlas should be updated to build using later version 
> JDK, still continuing to generate libraries for JDK8.
>  
> {noformat}
> [Atlas-master-snapshot-publish] $ /home/jenkins/tools/java/latest1.8/bin/java 
> -Xmx16g -Xms16g -Dfile.encoding=UTF-8 -Dproject.build.sourceEncoding=UTF-8 
> -cp 
> /home/jenkins/maven35-agent.jar:/home/jenkins/tools/maven/apache-maven-3.6.3/boot/plexus-classworlds-2.6.0.jar:/home/jenkins/tools/maven/apache-maven-3.6.3/conf/logging
>  jenkins.maven3.agent.Maven35Main 
> /home/jenkins/tools/maven/apache-maven-3.6.3 /home/jenkins/agent.jar 
> /home/jenkins/maven35-interceptor.jar 
> /home/jenkins/maven3-interceptor-commons.jar 36489
> Exception in thread "main" java.lang.UnsupportedClassVersionError: 
> hudson/remoting/Launcher has been compiled by a more recent version of the 
> Java Runtime (class file version 55.0), this version of the Java Runtime only 
> recognizes class file versions up to 52.0
>  {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Review Request 74376: ATLAS-4741: updated Storm version to 2.4.0

2023-03-30 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74376/
---

Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nikhil Bonte, Nixon 
Rodrigues, Pinal Shah, Sarath Subramanian, and Sidharth Mishra.


Bugs: ATLAS-4741
https://issues.apache.org/jira/browse/ATLAS-4741


Repository: atlas


Description
---

- updated Strom version to 2.4.0
- excluded storm-clojure from storm-core


Diffs
-

  addons/storm-bridge-shim/pom.xml f4e75927f 
  addons/storm-bridge/pom.xml d6cec279b 
  pom.xml 15aa57bc4 


Diff: https://reviews.apache.org/r/74376/diff/1/


Testing
---

- pre-commit tests run 
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1336/


Thanks,

Madhan Neethiraj



[jira] [Created] (ATLAS-4741) Build failure: Could not transfer artifact com.twitter:carbonite:pom:1.5.0

2023-03-30 Thread Madhan Neethiraj (Jira)
Madhan Neethiraj created ATLAS-4741:
---

 Summary: Build failure: Could not transfer artifact 
com.twitter:carbonite:pom:1.5.0
 Key: ATLAS-4741
 URL: https://issues.apache.org/jira/browse/ATLAS-4741
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Reporter: Madhan Neethiraj
Assignee: Madhan Neethiraj


Atlas build fails with the following error:

{noformat}
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process 
(process-resource-bundles) on project storm-bridge-shim: Error resolving 
project artifact: Could not transfer artifact com.twitter:carbonite:pom:1.5.0 
from/to java.net-Public (https://maven.java.net/content/groups/public/): 
transfer failed for 
https://maven.java.net/content/groups/public/com/twitter/carbonite/1.5.0/carbonite-1.5.0.pom
 for project com.twitter:carbonite:jar:1.5.0: PKIX path building failed: 
sun.security.provider.certpath.SunCertPathBuilderException: unable to find 
valid certification path to requested target -> [Help 1]
 {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Review Request 74375: ATLAS-4740: enable building Atlas using JDK versions higher than 8

2023-03-30 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74375/
---

Review request for atlas, Ashutosh Mestry, Jayendra Parab, Pinal Shah, Sarath 
Subramanian, and Sidharth Mishra.


Bugs: ATLAS-4740
https://issues.apache.org/jira/browse/ATLAS-4740


Repository: atlas


Description
---

added build properties to generate binaries for 1.8


Diffs
-

  pom.xml 15aa57bc4 


Diff: https://reviews.apache.org/r/74375/diff/1/


Testing
---

pre-commit build and test run: 
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1335/


Thanks,

Madhan Neethiraj



[jira] [Created] (ATLAS-4740) support build using higher version JDK, with 1.8 as target version

2023-03-30 Thread Madhan Neethiraj (Jira)
Madhan Neethiraj created ATLAS-4740:
---

 Summary: support build using higher version JDK, with 1.8 as 
target version
 Key: ATLAS-4740
 URL: https://issues.apache.org/jira/browse/ATLAS-4740
 Project: Atlas
  Issue Type: Improvement
  Components:  atlas-core
Reporter: Madhan Neethiraj
Assignee: Madhan Neethiraj


Atlas requires JDK8 to build. Recent changes in infra, like Jenkins, require 
using JDK11 or above to build. Building with JDK8 results in the failure given 
below.

 

To address this issue, Atlas should be updated to build using later version 
JDK, still continuing to generate libraries for JDK8.

 
{noformat}
[Atlas-master-snapshot-publish] $ /home/jenkins/tools/java/latest1.8/bin/java 
-Xmx16g -Xms16g -Dfile.encoding=UTF-8 -Dproject.build.sourceEncoding=UTF-8 -cp 
/home/jenkins/maven35-agent.jar:/home/jenkins/tools/maven/apache-maven-3.6.3/boot/plexus-classworlds-2.6.0.jar:/home/jenkins/tools/maven/apache-maven-3.6.3/conf/logging
 jenkins.maven3.agent.Maven35Main /home/jenkins/tools/maven/apache-maven-3.6.3 
/home/jenkins/agent.jar /home/jenkins/maven35-interceptor.jar 
/home/jenkins/maven3-interceptor-commons.jar 36489
Exception in thread "main" java.lang.UnsupportedClassVersionError: 
hudson/remoting/Launcher has been compiled by a more recent version of the Java 
Runtime (class file version 55.0), this version of the Java Runtime only 
recognizes class file versions up to 52.0
 {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: Getting updates into version 2.3.0

2023-03-26 Thread Madhan Neethiraj
Umesh,

Apache Atlas uses branch-2.0 for 2.x maintenance releases, like 2.20, 2.3.0. 
So, commits for next maint release (2.4.0) would go in branch-2.0.

To start as a contributor in Apache Atlas, you would need a JIRA account at 
https://issues.apache.org/. If you don’t have an account yet, you can sign up 
for one at https://selfserve.apache.org/jira-account.html.

Once you become a contributor, you will be able to create/update/assign JIRAs 
and attach patches. Apache Atlas uses review board for code review of patches. 
The patches will have to be reviewed/approved/committed by one of Apache Atlas 
committers.

Hope this helps.

Madhan


On 3/6/23, 12:43 PM, "Mallugari, Umesh" mailto:umesh.mallug...@usbank.com.inva>LID> wrote:


Fellow developers,


What strategy do you suggest while we are working on a tag such as 2.3.0 to get 
updates following that tag?


Many thanks!


Umesh




U.S. BANCORP made the following annotations
-
Electronic Privacy Notice. This e-mail, and any attachments, contains 
information that is, or may be, covered by electronic communications privacy 
laws, and is also confidential and proprietary in nature. If you are not the 
intended recipient, please be advised that you are legally prohibited from 
retaining, using, copying, distributing, or otherwise disclosing this 
information in any manner. Instead, please reply to the sender that you have 
received this communication in error, and then immediately delete it. Thank you 
in advance for your cooperation.


-






Re: Consultation on whether Apache Atlas replaceNamespace Attributes is enabled or not

2023-03-15 Thread Madhan Neethiraj
+dev list

 

Kai Lin,

 

There are couple of reasons:
technical metadata reported by Atlas hooks don’t include namespace attributes. 
While processing entity update notification by hooks, it is important to not 
lose existing namespace attributes on these entities.
Atlas deployments might be using Atlas clients built before the namespace 
feature was introduced. Calls to update entities from these clients wouldn’t 
include namespace attributes. Hence it is important to retain namespace 
attributes on entities unless the client overrides with appropriate parameter.
 

Hope this helps.

 

Thanks,

Madhan

 

From: Kai Lin 
Date: Tuesday, March 14, 2023 at 3:31 AM
To: 
Subject: Consultation on whether Apache Atlas replaceNamespace Attributes is 
enabled or not

 

Dear Madhan,

I would like to know why the replaceNamespaceAttributes parameter of the 
AtlasEntityStoreV2.createOrUpdate interface is false?Is there any consideration?

https://github.com/apache/atlas/commit/d3a08b6eabcc50536c468db2d1328df54ee27fce



Can i change the replaceNamespaceAttributes to true?Will there be any problem? 

Looking forward to your answer.

 

Thank you very much.



[DRAFT] [REPORT] Apache Atlas - March 2023

2023-03-12 Thread Madhan Neethiraj
Atlas community,

 

Please review the following draft report for this quarter.

 

Thanks,

Madhan

 

 

## Description:

The mission of Apache Atlas is the creation and maintenance of software related 

to a scalable and extensible set of core foundational governance services - 

enabling enterprises to effectively and efficiently meet their compliance 

requirements

 

## Issues:

There are no issues requiring board attention.

 

## Membership Data:

Apache Atlas was founded 2017-06-20 (6 years ago)

There are currently 47 committers and 32 PMC members in this project.

The Committer-to-PMC ratio is roughly 3:2.

 

Community changes, past quarter:

- No new PMC members. Last addition was Ashutosh Mestry on 2019-04-16.

- No new committers. Last addition was Mandar Ambawane on 2022-09-12.

 

## Project Activity:

- released Python client 0.0.13

- updated versions of Spring-security, Netty, Swagger, Jettison, Snakeyaml,

  Tinkerpop

- UI fixes in handling of deleted entities

- fix to address failure in admin audit APIs due to missed index creation

 

## Community Health:

- dev@atlas.apache.org had a 22% decrease in traffic in the past quarter

 (680 emails compared to 868)

 

## Most Recent releases:

2.3.0 was released on 2022-12-06

2.2.0 was released on 2021-08-17

2.1.0 was released on 2020-07-15

0.8.4 was released on 2019-06-21



[jira] [Resolved] (ATLAS-4657) Atlas Python client is messing up with logs

2023-02-28 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-4657.
-
Fix Version/s: 3.0.0
   2.4.0
   Resolution: Fixed

[~dolfinus]  - thank you for this fix in Apache Atlas Python client. This fix 
is included [Python client 
0.0.13|https://pypi.org/project/apache-atlas/0.0.13/].

> Atlas Python client is messing up with logs
> ---
>
> Key: ATLAS-4657
> URL: https://issues.apache.org/jira/browse/ATLAS-4657
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-intg
>Affects Versions: 2.1.0
>Reporter: Maxim Martynov
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> After importing {{atlas-client}} Python library, I got all my loggers 
> disabled.
> This is caused by this code block:
> [https://github.com/apache/atlas/blob/c1daaa44c3833be31c006753a2cf2f6f2e476cdc/intg/src/main/python/apache_atlas/__init__.py]
> Unlike applications, libraries should not setup logging by themselves. It 
> should be done explicitly by code which uses this client. Or there should be 
> a function to set up desired logging, which also should be explicitly called 
> by application or user.
>  
> I've created a pull request to fix this issue:
> [https://github.com/apache/atlas/pull/157]
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: 2.3.0 - How to put an entity in "In Progress"..

2023-02-06 Thread Madhan Neethiraj
Umesh,

"In Progress" status in lineage diagram means that Apache Atlas is yet to 
receive complete details of the entity i.e., entity.isIncomplete = true. This 
can happen for example when it processes report of a lineage between entities 
and Apache Atlas doesn't have details of some of the entities. Subsequently 
when Atlas receives metadata of such entities, "In Progress" status will be 
cleared.

Hope this helps.

Madhan




On 2/6/23, 11:03 AM, "Mallugari, Umesh" mailto:umesh.mallug...@usbank.com.inva>LID> wrote:


Fellow Atlas Developers & Users,


We are trying out version 2.3.0 in our organization. In this version, there is 
a new status available for entities in the lineage diagram called “In Progress” 
with an hour glass.
How to test this feature so entities show up with the hour glass in the diagram?


Many thanks.


Umesh




U.S. BANCORP made the following annotations
-
Electronic Privacy Notice. This e-mail, and any attachments, contains 
information that is, or may be, covered by electronic communications privacy 
laws, and is also confidential and proprietary in nature. If you are not the 
intended recipient, please be advised that you are legally prohibited from 
retaining, using, copying, distributing, or otherwise disclosing this 
information in any manner. Instead, please reply to the sender that you have 
received this communication in error, and then immediately delete it. Thank you 
in advance for your cooperation.


-






[jira] [Commented] (ATLAS-3756) Error when compile from source code - can not find restlet for transitive dep

2023-01-26 Thread Madhan Neethiraj (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681114#comment-17681114
 ] 

Madhan Neethiraj commented on ATLAS-3756:
-

[~hsaputra]  - thank you for the fix. [~mcbeelen]  - thank you for the nudge to 
get this fix merged.

> Error when compile from source code - can not find restlet for transitive dep
> -
>
> Key: ATLAS-3756
> URL: https://issues.apache.org/jira/browse/ATLAS-3756
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 3.0.0
>Reporter: Henry Saputra
>Assignee: Henry Saputra
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> When trying to compile Apache Atlas from code, got this error messages:
>  
> mvn clean -DskipTests install
>  
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 8.214 s
> [INFO] Finished at: 2020-04-26T01:25:17-07:00
> [INFO] 
> 
> [ERROR] Failed to execute goal on project atlas-testtools: Could not resolve 
> dependencies for project org.apache.atlas:atlas-testtools:jar:3.0.0-SNAPSHOT: 
> The following artifacts could not be resolved: 
> org.restlet.jee:org.restlet:jar:2.3.0, 
> org.restlet.jee:org.restlet.ext.servlet:jar:2.3.0: Failure to find 
> org.restlet.jee:org.restlet:jar:2.3.0 in https://repo1.maven.org/maven2 was 
> cached in the local repository, resolution will not be reattempted until the 
> update interval of central has elapsed or updates are forced -> [Help 1]
>  
> Maven version:
> 3.6.3
>  
> Java version:
> OpenJDK 1.8.0 - 252



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (ATLAS-3756) Error when compile from source code - can not find restlet for transitive dep

2023-01-26 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-3756.
-
Fix Version/s: 2.4.0
   3.0.0
   Resolution: Fixed

> Error when compile from source code - can not find restlet for transitive dep
> -
>
> Key: ATLAS-3756
> URL: https://issues.apache.org/jira/browse/ATLAS-3756
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 3.0.0
>Reporter: Henry Saputra
>Assignee: Henry Saputra
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> When trying to compile Apache Atlas from code, got this error messages:
>  
> mvn clean -DskipTests install
>  
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 8.214 s
> [INFO] Finished at: 2020-04-26T01:25:17-07:00
> [INFO] 
> 
> [ERROR] Failed to execute goal on project atlas-testtools: Could not resolve 
> dependencies for project org.apache.atlas:atlas-testtools:jar:3.0.0-SNAPSHOT: 
> The following artifacts could not be resolved: 
> org.restlet.jee:org.restlet:jar:2.3.0, 
> org.restlet.jee:org.restlet.ext.servlet:jar:2.3.0: Failure to find 
> org.restlet.jee:org.restlet:jar:2.3.0 in https://repo1.maven.org/maven2 was 
> cached in the local repository, resolution will not be reattempted until the 
> update interval of central has elapsed or updates are forced -> [Help 1]
>  
> Maven version:
> 3.6.3
>  
> Java version:
> OpenJDK 1.8.0 - 252



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (ATLAS-3756) Error when compile from source code - can not find restlet for transitive dep

2023-01-26 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj reassigned ATLAS-3756:
---

Assignee: Henry Saputra

> Error when compile from source code - can not find restlet for transitive dep
> -
>
> Key: ATLAS-3756
> URL: https://issues.apache.org/jira/browse/ATLAS-3756
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 3.0.0
>Reporter: Henry Saputra
>Assignee: Henry Saputra
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> When trying to compile Apache Atlas from code, got this error messages:
>  
> mvn clean -DskipTests install
>  
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 8.214 s
> [INFO] Finished at: 2020-04-26T01:25:17-07:00
> [INFO] 
> 
> [ERROR] Failed to execute goal on project atlas-testtools: Could not resolve 
> dependencies for project org.apache.atlas:atlas-testtools:jar:3.0.0-SNAPSHOT: 
> The following artifacts could not be resolved: 
> org.restlet.jee:org.restlet:jar:2.3.0, 
> org.restlet.jee:org.restlet.ext.servlet:jar:2.3.0: Failure to find 
> org.restlet.jee:org.restlet:jar:2.3.0 in https://repo1.maven.org/maven2 was 
> cached in the local repository, resolution will not be reattempted until the 
> update interval of central has elapsed or updates are forced -> [Help 1]
>  
> Maven version:
> 3.6.3
>  
> Java version:
> OpenJDK 1.8.0 - 252



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[DRAFT] [REPORT] Apache Atlas - December 2022

2022-12-10 Thread Madhan Neethiraj
Atlas community,

Please review the following draft report for this quarter.

Thanks,
Madhan

## Description:
The mission of Apache Atlas is the creation and maintenance of software related 
to a scalable and extensible set of core foundational governance services - 
enabling enterprises to effectively and efficiently meet their compliance 
requirements

## Issues:
There are no issues requiring board attention.

## Membership Data:
Apache Atlas was founded 2017-06-20 (5 years ago)
There are currently 47 committers and 32 PMC members in this project.
The Committer-to-PMC ratio is roughly 3:2.

Community changes, past quarter:
- No new PMC members. Last addition was Ashutosh Mestry on 2019-04-16.
- Mandar Ambawane was added as committer on 2022-09-12
- Prasad Pawar was added as committer on 2022-09-12
- 5 new Atlas contributors in last quarter:
  - Sheetal Shah, zhuangchong, yanxu3, Maxim Martynov, Dov Benyomin Sohacheski 

## Project Activity:
- the community released Apache Atlas 2.3.0
- released Python client 0.0.12
- New feature: relationship search API and UI
- docker setup fixes for Kafka bridge
- fixes in import-hbase and import-kafka utilities
- updated versions of commons-text and commons-configuration2 libraries
- multiple UI component version updates: scss-tokenizer, node-sass,
  loader-utils, Snake-yaml
- updated Hive version from 3.1.0 to 3.1.2
- multiple UI fixes


## Community Health:
- dev@atlas.apache.org had a 22% decrease in traffic in the past quarter (680
  emails compared to 868)
- u...@atlas.apache.org had a 37% decrease in traffic in the past quarter (12
  emails compared to 19)
- 46 issues opened in JIRA, past quarter (-16% change)
- 47 issues closed in JIRA, past quarter (30% increase)
- 104 commits in the past quarter (70% increase)
- 17 code contributors in the past quarter (30% increase)
- 22 PRs opened on GitHub, past quarter (29% increase)
- 41 PRs closed on GitHub, past quarter (485% increase)

## Most Recent releases:
2.3.0 was released on 2022-12-06
2.2.0 was released on 2021-08-17
2.1.0 was released on 2020-07-15
0.8.4 was released on 2019-06-21





[jira] [Resolved] (ATLAS-4698) Release Atlas 2.3.0

2022-12-07 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-4698.
-
Resolution: Fixed

> Release Atlas 2.3.0
> ---
>
> Key: ATLAS-4698
> URL: https://issues.apache.org/jira/browse/ATLAS-4698
> Project: Atlas
>  Issue Type: Task
>Reporter: Madhan Neethiraj
>    Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 2.3.0
>
>
> Parent task to capture release tasks for Apache Atlas 2.3.0



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Review Request 74241: ATLAS-4702: updated version in pom.xml from 2.3.0 to 2.4.0-SNAPSHOT

2022-12-07 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74241/
---

Review request for atlas.


Bugs: ATLAS-4702
https://issues.apache.org/jira/browse/ATLAS-4702


Repository: atlas


Description
---

updated version in pom.xml files from 2.3.0 to 2.4.0-SNAPSHOT


Diffs
-

  addons/falcon-bridge-shim/pom.xml 490d812a0 
  addons/falcon-bridge/pom.xml 7072ab06f 
  addons/hbase-bridge-shim/pom.xml a33f78463 
  addons/hbase-bridge/pom.xml ee4ed0471 
  addons/hbase-testing-util/pom.xml 560e4f27e 
  addons/hdfs-model/pom.xml 43362fadf 
  addons/hive-bridge-shim/pom.xml 6c2c04aac 
  addons/hive-bridge/pom.xml 4911f98a0 
  addons/impala-bridge-shim/pom.xml 7c2cae7f5 
  addons/impala-bridge/pom.xml b7a08f2ec 
  addons/impala-hook-api/pom.xml 47c0b7003 
  addons/kafka-bridge/pom.xml fa54a56ae 
  addons/sqoop-bridge-shim/pom.xml 3e1e7c8ba 
  addons/sqoop-bridge/pom.xml 45a27e2f2 
  addons/storm-bridge-shim/pom.xml 098873c3f 
  addons/storm-bridge/pom.xml 01450e795 
  atlas-examples/pom.xml e5c336411 
  atlas-examples/sample-app/pom.xml 7375e6433 
  authorization/pom.xml e580df8f3 
  build-tools/pom.xml 12f1c87eb 
  client/client-v1/pom.xml 4d119ed10 
  client/client-v2/pom.xml 620c95b24 
  client/common/pom.xml f5099b110 
  client/pom.xml 3973f1e67 
  common/pom.xml 25e5a7b87 
  dashboardv2/pom.xml 42647005c 
  dashboardv3/pom.xml bb31b3f72 
  distro/pom.xml ae13b6574 
  docs/pom.xml 019c3fbea 
  graphdb/api/pom.xml 8f0c2dae5 
  graphdb/common/pom.xml dab86a314 
  graphdb/graphdb-impls/pom.xml eb53375a0 
  graphdb/janus-hbase2/pom.xml 73e88d9df 
  graphdb/janus/pom.xml 8ef052639 
  graphdb/pom.xml dceecf47c 
  intg/pom.xml 32acf0286 
  notification/pom.xml 67bae8958 
  plugin-classloader/pom.xml 8dff9c555 
  pom.xml 1217ccc34 
  repository/pom.xml 974c917ad 
  server-api/pom.xml 9e018874f 
  test-tools/pom.xml 838b970c9 
  tools/atlas-index-repair/pom.xml 732d2be3d 
  tools/classification-updater/pom.xml 9399420d2 
  webapp/pom.xml 25214a24d 


Diff: https://reviews.apache.org/r/74241/diff/1/


Testing
---

verified that build completes successfully


Thanks,

Madhan Neethiraj



[ANNOUNCE] Apache Atlas 2.3.0 released

2022-12-07 Thread Madhan Neethiraj
Hi All,

The Apache Atlas team is happy to announce the release of Apache Atlas - 
version 2.3.0.

Atlas is a scalable and extensible set of core foundational governance
services – enabling enterprises to effectively and efficiently meet their
compliance requirements within Hadoop and allows integration with the whole
enterprise data ecosystem.

The release artifacts are available at: https://atlas.apache.org/#/Downloads

The binary artifacts are available from Maven central and its mirrors.

To use these artifacts, please use the following documentation: 
http://atlas.apache.org/2.3.0/index.html

More details on Apache Atlas can be found at: http://atlas.apache.org/

We thank everyone who made this release possible.

Thanks,
Apache Atlas team




Review Request 74240: ATLAS-4700: updated documentation for 2.3.0 release

2022-12-07 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74240/
---

Review request for atlas, Ashutosh Mestry, Jayendra Parab, Mandar Ambawane, 
Nikhil Bonte, Nixon Rodrigues, Pinal Shah, Prasad Pawar, Radhika Kundam, Sarath 
Subramanian, Sharmadha Sainath, and Sidharth Mishra.


Bugs: ATLAS-4700
https://issues.apache.org/jira/browse/ATLAS-4700


Repository: atlas


Description
---

updated documentation for 2.3.0 release


Diffs
-

  docs/docz-lib/config/menu.js 37acf46d4 
  docs/docz-lib/config/versions.js 3ef55f681 
  docs/src/documents/Downloads/Downloads.md e486a1d0b 
  docs/src/documents/Overview.md 8614106ab 
  docs/src/documents/Whats-New/WhatsNew-2.3.md PRE-CREATION 


Diff: https://reviews.apache.org/r/74240/diff/1/


Testing
---

verified the updated site with 'mvn site:run" in docs directory


Thanks,

Madhan Neethiraj



[jira] [Resolved] (ATLAS-4701) Publish release artifacts

2022-12-06 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-4701.
-
Resolution: Fixed

> Publish release artifacts
> -
>
> Key: ATLAS-4701
> URL: https://issues.apache.org/jira/browse/ATLAS-4701
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Madhan Neethiraj
>    Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 2.3.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[RESULT] [VOTE] Release Apache Atlas 2.3.0

2022-12-06 Thread Madhan Neethiraj
Atlas team,

Thanks for taking time to verify, give feedback and vote on Apache Atlas 2.3.0 
release candidate #1.

More than 72 hours have passed since the email to vote for the release; the 
vote is now closed. The vote passes with 8 +1s and no 0s and -1s.

Here are the vote details:

Four +1 from following PMC members:
- Ashutosh Mestry
- Madhan Neethiraj
- Nixon Rodrigues
- Sarath Subramanian

Four +1 votes from following committers/contributors:
- Jayendra Parab
- Nikhil Bonte
- Ramesh Mani
- Sidharth Mishra

Voting thread: https://lists.apache.org/thread/rsvfkvhyrnckl6w10h1l5rom5jjnfso8

I will begin the process of creating the release artifacts for Apache Atlas 
2.3.0.

Thanks,
Madhan




Re: [VOTE] Release Apache Atlas 2.3.0 rc1

2022-12-01 Thread Madhan Neethiraj
Sid,

I see that the certificate in https://maven.restlet.com expired 19 days back. I 
guess this issue wouldn't be impact folks who have .m2 cache populated with 
org.restlet artifacts earlier (than the certificate expiry). Until this 
certificate is renewed, you can use the flags to build Atlas.

Thanks,
Madhan



On 12/1/22, 12:53 PM, "Sidharth Mishra"  wrote:

    Thanks Madhan and Sarath. I tried the below and the signature
verification worked with warning as shown below:
curl -O https://dist.apache.org/repos/dist/dev/atlas/KEYS
gpg --import KEYS
gpg --list madhan # Checked it has FED467D3B01179D4
gpg --verify apache-atlas-2.3.0-sources.tar.gz.asc
apache-atlas-2.3.0-sources.tar.gz
   gpg: Signature made Mon Nov 28 13:14:22 2022 PST
   gpg:using RSA key FED467D3B01179D4
   gpg: Good signature from "Madhan Neethiraj " 
[unknown]
   gpg: WARNING: This key is not certified with a trusted signature!
   gpg:  There is no indication that the signature belongs
to the owner.
   Primary key fingerprint: 0524 DD1F 7940 6654 6D61  0744 FED4
67D3 B011 79D4

Still I am getting below error when I run maven clean install. Adding
the -Dmaven.wagon.http.ssl.ignore.validity.dates=true
-Dmaven.wagon.http.ssl.insecure=true
-Dmaven.wagon.http.ssl.allowall=true is ignoring this issue. Do you
have any ideas on this?

   [ERROR] Failed to execute goal on project atlas-testtools:
Could not resolve dependencies for project
org.apache.atlas:atlas-testtools:jar:2.3.0: Failed to collect
dependencies at org.apache.solr:solr-test-framework:jar:8.6.3 ->
org.restlet.jee:org.restlet:jar:2.4.3: Failed to read artifact
descriptor for org.restlet.jee:org.restlet:jar:2.4.3: Could not
transfer artifact org.restlet.jee:org.restlet:pom:2.4.3 from/to
maven-restlet (https://maven.restlet.com): transfer failed for

https://maven.restlet.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-2.4.3.pom:
sun.security.validator.ValidatorException: PKIX path validation
failed: java.security.cert.CertPathValidatorException: validity check
failed: NotAfter: Sun Nov 13 09:05:56 PST 2022 -> [Help 1]

Thanks,
Sid

On Thu, Dec 1, 2022 at 10:04 AM Sarath Subramanian  
wrote:
>
> Sid,
>
> I followed the below steps to verify signature. I'm running on Mac OS
> Monterey.
>
>  wget https://dist.apache.org/repos/dist/dev/atlas/KEYS
> >  gpg --import KEYS
> >  gpg --verify apache-atlas-2.3.0-sources.tar.gz.asc
> > apache-atlas-2.3.0-sources.tar.gz
> >
>
> Thanks,
> Sarath
>
> On Wed, Nov 30, 2022 at 11:50 PM Sidharth Mishra 
> wrote:
>
> >  Thank you Madhan for the release candidate.
> >
> > Validated the following and faced some issues as mentioned below:
> >
> > - Able to download source, signature, md5 and sha512 files and verify
> > the checksum hash
> > - I am not able to verify the signature of the source. Steps I followed:
> > curl -O https://downloads.apache.org/atlas/KEYS
> > gpg --import KEYS
> > gpg --list-sigs madhan # Gives the correct output
> > pub   rsa4096 2020-06-10 [SC] [expires: 2024-06-09]
> > 1B6007E9CDEC4913DFB5031B630E02BA8823016D
> > uid   [ unknown] Madhan Neethiraj <
> > mad...@apache.org>
> > sig 3630E02BA8823016D 2020-06-10
> > Madhan Neethiraj 
> >sub   rsa4096 2020-06-10 [E] [expires: 2024-06-09]
> >sig  630E02BA8823016D 2020-06-10  Madhan
> > Neethiraj 
> >
> >pub   rsa2048 2014-11-10 [SC] [expired: 2018-11-10]
> > 4BDEE6708B5F5216CF0BA42754EA1B4FFA03B08A
> >uid   [ expired] Madhan Neethiraj <
> > mad...@apache.org>
> >sig 354EA1B4FFA03B08A 2014-11-10  Madhan
> > Neethiraj 
> >sig  9C0596B11E19B762 2014-11-10  [User ID not
> > found]
> >sig  2C1CD6311ED05C4A 2016-02-16  [User ID not
> > found]
> >sig 3 X  EB4200BBD4393DE8 2016-04-30  [User ID not
> > found]
> >
> > gpg --verify apache-atlas-2.3.0-sources.tar.gz.asc
> > apache-atlas-2.3.0-sources.tar.gz
> >  gpg: Signature made Mon Nov 28 13:14:22 2022 PST
> >  gpg:using RSA key FED467D3B01179D4
> > 

Re: [VOTE] Release Apache Atlas 2.3.0 rc1

2022-12-01 Thread Madhan Neethiraj
Sid,

Can you please use keys from https://dist.apache.org/repos/dist/dev/atlas/KEYS?

Thanks,
Madhan




On 11/30/22, 11:49 PM, "Sidharth Mishra"  wrote:

 Thank you Madhan for the release candidate.

Validated the following and faced some issues as mentioned below:

- Able to download source, signature, md5 and sha512 files and verify
the checksum hash
- I am not able to verify the signature of the source. Steps I followed:
curl -O https://downloads.apache.org/atlas/KEYS
gpg --import KEYS
gpg --list-sigs madhan # Gives the correct output
pub   rsa4096 2020-06-10 [SC] [expires: 2024-06-09]
1B6007E9CDEC4913DFB5031B630E02BA8823016D
uid   [ unknown] Madhan Neethiraj 

sig 3630E02BA8823016D 2020-06-10
Madhan Neethiraj 
   sub   rsa4096 2020-06-10 [E] [expires: 2024-06-09]
   sig  630E02BA8823016D 2020-06-10  Madhan
Neethiraj 

   pub   rsa2048 2014-11-10 [SC] [expired: 2018-11-10]
4BDEE6708B5F5216CF0BA42754EA1B4FFA03B08A
   uid   [ expired] Madhan Neethiraj 
   sig 354EA1B4FFA03B08A 2014-11-10  Madhan
Neethiraj 
   sig  9C0596B11E19B762 2014-11-10  [User ID not found]
   sig  2C1CD6311ED05C4A 2016-02-16  [User ID not found]
   sig 3 X  EB4200BBD4393DE8 2016-04-30  [User ID not found]

gpg --verify apache-atlas-2.3.0-sources.tar.gz.asc
apache-atlas-2.3.0-sources.tar.gz
 gpg: Signature made Mon Nov 28 13:14:22 2022 PST
 gpg:using RSA key FED467D3B01179D4
 gpg: Can't check signature: No public key

   # Even tried this and no help as it didn't download any keys
   gpg --keyserver
https://dist.apache.org/repos/dist/release/atlas/KEYS --recv-keys
FED467D3B01179D4

- Build failed for the source using embedded-hbase-solr profile -

  [ERROR] Failed to execute goal on project atlas-testtools: Could not
resolve dependencies for project
org.apache.atlas:atlas-testtools:jar:2.3.0: Failed to collect
dependencies at org.apache.solr:solr-test-framework:jar:8.6.3 ->
org.restlet.jee:org.restlet:jar:2.4.3: Failed to read artifact
descriptor for org.restlet.jee:org.restlet:jar:2.4.3: Could not
transfer artifact org.restlet.jee:org.restlet:pom:2.4.3 from/to
maven-restlet (https://maven.restlet.com): transfer failed for

https://maven.restlet.com/org/restlet/jee/org.restlet/2.4.3/org.restlet-2.4.3.pom:
sun.security.validator.ValidatorException: PKIX path validation
failed: java.security.cert.CertPathValidatorException: validity check
failed: NotAfter: Sun Nov 13 09:05:56 PST 2022 -> [Help 1]
  [ERROR]
  [ERROR] To see the full stack trace of the errors, re-run Maven with
the -e switch.
  [ERROR] Re-run Maven using the -X switch to enable full debug logging.
  [ERROR]
  [ERROR] For more information about the errors and possible
solutions, please read the following articles:
  [ERROR] [Help 1]

http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
  [ERROR]
  [ERROR] After correcting the problems, you can resume the build with
the command
  [ERROR]   mvn  -rf :atlas-testtools

By running the below command the above issue was resolved as it
ignored all ssl verifications (Similar issue -

https://stackoverflow.com/questions/68199459/maven-build-failed-pkix-path-validation-failed-java-security-cert-certpathval):
 ln -s /usr/local/bin/python3 /usr/local/bin/python  # As by
default python3 gets installed when we upgrade to mac ventura and no
python symlink
 mvn clean install
-Dmaven.wagon.http.ssl.ignore.validity.dates=true
-Dmaven.wagon.http.ssl.insecure=true
-Dmaven.wagon.http.ssl.allowall=true
 mvn clean package -Pdist
-Dmaven.wagon.http.ssl.ignore.validity.dates=true
-Dmaven.wagon.http.ssl.insecure=true
-Dmaven.wagon.http.ssl.allowall=true

- started Atlas and ran quickstart script to preload sample types and
entities
- validated relation search, basic and advanced search

P.S. Recently I upgraded my mac to ventura and I'm not sure if these
issues are due to the upgrade. If someone else has a similar mac
version and it works for them then we are good to proceed.
Please check once if the signature is correct or else I will debug
further at my end.

Thanks,
Sid



On Tue, Nov 29, 2022 at 10:17 PM Sarath Subramanian  
wrote:
>
    > Thank you Madhan for the release candidate.
>
> +1 for Apache Atlas 2.3.0 release candidate #1
>
> validated the following:
> - Able to download so

Re: Apache Atlas v2.2.0 setup with HBase v1.4.9 as backend giving - org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend

2022-11-30 Thread Madhan Neethiraj
Ashok,

 

Apache Atlas 2.2.0 release is built and tested with HBase 2.3.3. Your 
deployment uses HBase 1.4.9; looking at errors (below), I guess HBase 2.3.3 
client used in Atlas fails to work with HBase 1.4.9. Please consider moving to 
HBase version 2.3.3 or later.

 

Hope this helps.

 

Madhan

 

Caused by: 
org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException):
 org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column 
family table does not exist in region hbase:meta,,1.1588230740 in table 
'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => 
'|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}, 
{NAME => 'info', BLOOMFILTER => 'NONE', VERSIONS => '3', IN_MEMORY => 'true', 
KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', 
COMPRESSION => 'NONE', CACHE_DATA_IN_L1 => 'true', MIN_VERSIONS => '0', 
BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}
at org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:8355)


 

From: Ashok Kumar 
Date: Monday, November 28, 2022 at 12:35 AM
To: , 
Cc: , , 

Subject: Apache Atlas v2.2.0 setup with HBase v1.4.9 as backend giving - 
org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in 
storage backend

 

Hi Madan & Team,

 

We are trying to deploy Atlas using docker containers, the below is the tech 
stack used for the deployment

1.   Apache Atlas v2.2.0 (Downloaded from 
https://downloads.apache.org/atlas/2.2.0/apache-atlas-2.2.0-sources.tar.gz), 
compiled and created a docker image out of it.

2.   Solr v8.11.1

3.   HBase (v1.4.9)

4.   Apache Kafka & Zookeeper (v2.6.1)

When we tried to deploy Atlas, we were facing the following issue. The 
following is the stack trace of the error:

 

2022-11-28 06:20:12,162 DEBUG - [main:] ~ Substituted default CF name 
"system_properties" with short form "s" to reduce HBase KeyValue size 
(HBaseStoreManager:684)
2022-11-28 06:20:12,527 WARN  - [main:] ~ Unexpected exception during 
getDeployment() (HBaseStoreManager:399)
java.lang.RuntimeException: 
org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in 
storage backend
at 
org.janusgraph.diskstorage.hbase2.HBaseStoreManager.getDeployment(HBaseStoreManager.java:358)
at 
org.janusgraph.diskstorage.hbase2.HBaseStoreManager.getFeatures(HBaseStoreManager.java:397)
at 
org.janusgraph.graphdb.configuration.builder.GraphDatabaseConfigurationBuilder.build(GraphDatabaseConfigurationBuilder.java:51)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:161)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:132)
at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:112)
at 
org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphDatabase.initJanusGraph(AtlasJanusGraphDatabase.java:182)
at 
org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphDatabase.getGraphInstance(AtlasJanusGraphDatabase.java:169)
at 
org.apache.atlas.repository.graphdb.janus.AtlasJanusGraphDatabase.getGraph(AtlasJanusGraphDatabase.java:278)
at 
org.apache.atlas.repository.graph.AtlasGraphProvider.getGraphInstance(AtlasGraphProvider.java:52)
at 
org.apache.atlas.repository.graph.AtlasGraphProvider.get(AtlasGraphProvider.java:98)
at 
org.apache.atlas.repository.graph.AtlasGraphProvider$$EnhancerBySpringCGLIB$$72fdd918.CGLIB$get$1()
at 
org.apache.atlas.repository.graph.AtlasGraphProvider$$EnhancerBySpringCGLIB$$72fdd918$$FastClassBySpringCGLIB$$71b13aef.invoke()
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
at 
org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:358)
at 
org.apache.atlas.repository.graph.AtlasGraphProvider$$EnhancerBySpringCGLIB$$72fdd918.get()
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162)
at 
org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:588)
at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1176)
at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1071)
at 
org.springframework.beans.factory.support.AbstractAutowir

[VOTE] Release Apache Atlas 2.3.0 rc1

2022-11-28 Thread Madhan Neethiraj
Atlas team,

 

Apache Atlas 2.3.0 release candidate #1 is now available for a vote within dev 
community. Links to the release artifacts are given below. Please review and 
vote.

 

The vote will be open for at least 72 hours or until necessary votes are 
reached.

  [ ] +1 approve

  [ ] +0 no opinion

  [ ] -1 disapprove (and reason why)

 

Thanks,

Madhan

 

 

List of issues addressed in this release: 
https://issues.apache.org/jira/issues/?jql=project%20%3D%20Atlas%20AND%20fixVersion%20%3D%202.3.0%20ORDER%20BY%20key%20DESC

 

Git tag for the release: https://github.com/apache/atlas/tree/release-2.3.0-rc1

Sources for the release: 
https://dist.apache.org/repos/dist/dev/atlas/2.3.0-rc1/apache-atlas-2.3.0-sources.tar.gz

 

Source release verification:

  PGP Signature: 
https://dist.apache.org/repos/dist/dev/atlas/2.3.0-rc1/apache-atlas-2.3.0-sources.tar.gz.asc

  MD5 Hash:  
https://dist.apache.org/repos/dist/dev/atlas/2.3.0-rc1/apache-atlas-2.3.0-sources.tar.gz.md5

  SHA512 Hash:   
https://dist.apache.org/repos/dist/dev/atlas/2.3.0-rc1/apache-atlas-2.3.0-sources.tar.gz.sha512
 

 

Keys to verify the signature of the release artifacts are available at: 
https://dist.apache.org/repos/dist/dev/atlas/KEYS



[VOTE] Release Apache Atlas 2.3.0 rc1

2022-11-28 Thread Madhan Neethiraj
Atlas team,

 

Apache Atlas 2.3.0 release candidate #1 is now available for a vote within dev 
community. Links to the release artifacts are given below. Please review and 
vote.

 

The vote will be open for at least 72 hours or until necessary votes are 
reached.

  [ ] +1 approve

  [ ] +0 no opinion

  [ ] -1 disapprove (and reason why)

 

Thanks,

Madhan

 

 

List of issues addressed in this release: 
https://issues.apache.org/jira/issues/?jql=project%20%3D%20Atlas%20AND%20fixVersion%20%3D%202.3.0%20ORDER%20BY%20key%20DESC

 

Git tag for the release: https://github.com/apache/atlas/tree/release-2.3.0-rc1

 

Sources for the release: 
https://dist.apache.org/repos/dist/dev/atlas/2.3.0-rc1/apache-atlas-2.3.0-sources.tar.gz

 

Source release verification:

  PGP Signature: 
https://dist.apache.org/repos/dist/dev/atlas/2.3.0-rc1/apache-atlas-2.3.0-sources.tar.gz.asc

  MD5 Hash: 
https://dist.apache.org/repos/dist/dev/atlas/2.3.0-rc1/apache-atlas-2.3.0-sources.tar.gz.md5

 

Keys to verify the signature of the release artifacts are available at: 
https://dist.apache.org/repos/dist/dev/atlas/KEYS



[jira] [Updated] (ATLAS-4714) Atlas source package to exclude downloaded archives

2022-11-22 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-4714:

Attachment: ATLAS-4714.patch

> Atlas source package to exclude downloaded archives
> ---
>
> Key: ATLAS-4714
> URL: https://issues.apache.org/jira/browse/ATLAS-4714
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>    Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: ATLAS-4714.patch
>
>
> Atlas source packaging currently includes downloaded archive files, for 
> examples the files used to build docker images. These files should be 
> excluded from source package.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Review Request 74212: ATLAS-4714: exclude downloaded/generated files from source package

2022-11-22 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74212/
---

Review request for atlas, Ayub Pathan, Ashutosh Mestry, Jayendra Parab, Mandar 
Ambawane, Pinal Shah, Prasad Pawar, Radhika Kundam, Sarath Subramanian, 
Sharmadha Sainath, and Sidharth Mishra.


Bugs: ATLAS-4714
https://issues.apache.org/jira/browse/ATLAS-4714


Repository: atlas


Description
---

Following downloaded/generated files are excluded from source package:
- distro/elasticsearch/**
- distro/zk/**
- dashboardv2/node_modules/**
- dashboardv3/node_modules/**
- dev-support/atlas-docker/downloads/*


Diffs
-

  distro/src/main/assemblies/src-package.xml b53487155 


Diff: https://reviews.apache.org/r/74212/diff/1/


Testing
---

- Built and deployed Atlas using the generated source package in docker
- Sanity testing of Atlas UI, Hive/HBase hooks, import utilities for 
Hive/Hbase/Kafka/Glossaries


Thanks,

Madhan Neethiraj



[jira] [Assigned] (ATLAS-4712) atlas-hive image shows "No such file or directory error"

2022-11-17 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj reassigned ATLAS-4712:
---

Assignee: Madhan Neethiraj  (was: Snehal Ambavkar)

> atlas-hive image shows  "No such file or directory error"
> -
>
> Key: ATLAS-4712
> URL: https://issues.apache.org/jira/browse/ATLAS-4712
> Project: Atlas
>  Issue Type: Bug
>Reporter: Snehal Ambavkar
>Assignee: Madhan Neethiraj
>Priority: Minor
> Fix For: 3.0.0, 2.3.0
>
> Attachments: ATLAS-4712.patch
>
>
> Running docker-compose 
> {code:java}
>  docker-compose -f docker-compose.atlas-base.yml -f docker-compose.atlas.yml 
> -f docker-compose.atlas-hadoop.yml -f docker-compose.atlas-hbase.yml -f 
> docker-compose.atlas-kafka.yml -f docker-compose.atlas-hive.yml up -d
>  {code}
> shows with:
> {code:java}
> /home/atlas/scripts/atlas-hive-setup.sh: line 44: cd: 
> /opt/atlas/atlas-hive-plugin: No such file or directory
> /home/atlas/scripts/atlas-hive-setup.sh: line 45: ./enable-hive-plugin.sh: No 
> such file or directory {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (ATLAS-3498) Update basic-search to enable search using labels

2022-11-17 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-3498:

Fix Version/s: 3.0.0
   (was: 2.3.0)

> Update basic-search to enable search using labels
> -
>
> Key: ATLAS-3498
> URL: https://issues.apache.org/jira/browse/ATLAS-3498
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 3.0.0
>
>
> Update basic-search to enable search using labels



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (ATLAS-3501) Update guava version to 28.0-jre

2022-11-17 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-3501:

Fix Version/s: 3.0.0
   (was: 2.3.0)

> Update guava version to 28.0-jre
> 
>
> Key: ATLAS-3501
> URL: https://issues.apache.org/jira/browse/ATLAS-3501
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Aadarsh Jajodia
>Assignee: Aadarsh Jajodia
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: 0001-ATLAS-3501-Update-guava-version-to-28.0-jre.patch
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (ATLAS-3610) Enable logging to multiple targets (logdir, stdout)

2022-11-17 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-3610:

Fix Version/s: (was: 2.3.0)

> Enable logging to multiple targets (logdir, stdout)
> ---
>
> Key: ATLAS-3610
> URL: https://issues.apache.org/jira/browse/ATLAS-3610
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 3.0.0
>Reporter: Mariusz Górski
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: ATLAS-3610-1.patch, ATLAS-3610-2.patch, 
> ATLAS-3610-3.patch, ATLAS-3610.patch
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> *Problem description*
> When starting Atlas, Python scripts are invoked to spawn all necessary 
> processes. This is achieved with the use of _subprocess.Popen()_ in 
> _runProcess()_ function of _atlas_config.py_ script. While doing that, stdout 
> is being redirected to log files, which (for java processes) can override 
> log4j configuration (if configured with stdout handler). This creates 
> situation when log4j configuration cannot be relied upon and prevents logging 
> to stdout.
> *Proposed solution*
>  * Add possibility to log to stdout by using multiple stdout/stderr handlers
>  * Adjust _runProcess()_ function from _atlas_config.py_ script (and all 
> functions relying on it) to provide _logconsole_ kwarg which will add 
> additional handler to Popen stdout/stderr redirection
>  * Add _ENABLE_LOGGING_TO_CONSOLE_ env variable (defaulting to _False_) for 
> backwards compatibility.
>  * Adjust _atlas_start.py_ script to make use of _ENABLE_LOGGING_TO_CONSOLE_  
> and provide it in functions invoking atlas/solr/elasticsearch/zookeeper
> *Predicted Benefits*
> This improvement would bring the benefit of enabling logging to stdout (which 
> is supressed for now even if configured in spawned components), which could 
> be very useful when running Atlas in docker container or in Kubernetes/OC, 
> where some logging scraping is present.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (ATLAS-3890) Enhancement for all BulkAPIs.

2022-11-17 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-3890:

Fix Version/s: 3.0.0
   (was: 2.3.0)

> Enhancement for all BulkAPIs.
> -
>
> Key: ATLAS-3890
> URL: https://issues.apache.org/jira/browse/ATLAS-3890
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: chaitali borole
>Assignee: chaitali borole
>Priority: Major
>  Labels: api-change
> Fix For: 3.0.0
>
>
> According to below linked Atlas Jira two Bulk Apis are enhanced to return 
> authorised entities or modify them when some of then entity types are blocked 
> by policies in Ranger for access to user.Hence we are improving the other 
> bulk Apis too.
> Previously user only used to get 403 error if he tries to access or modify 
> entities which comprised of both authorised as well as unauthorised typenames.
> DELETE /bulk - EntityREST.deleteByGuids()
> GET /bulk/uniqueAttribute/type/\{typeName} - 
> EntityREST.getEntitiesByUniqueAttributes()



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (ATLAS-4679) Indexing of deleted relationship edges prolongs entity update time

2022-11-17 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-4679.
-
Resolution: Fixed

> Indexing of deleted relationship edges prolongs entity update time
> --
>
> Key: ATLAS-4679
> URL: https://issues.apache.org/jira/browse/ATLAS-4679
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Reporter: VINAYAK MARRAIYA
>Assignee: Sheetal Shah
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Review Request 74205: ATLAS-4699: updated version in pom.xml from 2.3.0-SNAPSHOT to 2.3.0

2022-11-17 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74205/
---

Review request for atlas, Ashutosh Mestry, Jayendra Parab, Mandar Ambawane, 
Nikhil Bonte, Nixon Rodrigues, Pinal Shah, Radhika Kundam, Sarath Subramanian, 
and Sidharth Mishra.


Bugs: ATLAS-4699
https://issues.apache.org/jira/browse/ATLAS-4699


Repository: atlas


Description
---

updated Atlas version from 2.3.0-SNAPSHOT to 2.3.0


Diffs
-

  addons/falcon-bridge-shim/pom.xml 981e76418 
  addons/falcon-bridge/pom.xml 8e25111d4 
  addons/hbase-bridge-shim/pom.xml ef3c1bad9 
  addons/hbase-bridge/pom.xml 5a0f62fb3 
  addons/hbase-testing-util/pom.xml 947ca477f 
  addons/hdfs-model/pom.xml 226b0d49b 
  addons/hive-bridge-shim/pom.xml cb4dbb3ae 
  addons/hive-bridge/pom.xml 80a2252c5 
  addons/impala-bridge-shim/pom.xml 247dc10ae 
  addons/impala-bridge/pom.xml cacdf4a98 
  addons/impala-hook-api/pom.xml 8b61994f7 
  addons/kafka-bridge/pom.xml 31f5d5b3a 
  addons/sqoop-bridge-shim/pom.xml 4ba77d521 
  addons/sqoop-bridge/pom.xml 62e2eee00 
  addons/storm-bridge-shim/pom.xml 510718c8d 
  addons/storm-bridge/pom.xml a5017e0b2 
  atlas-examples/pom.xml 12cd8c401 
  atlas-examples/sample-app/pom.xml e2eb42f54 
  authorization/pom.xml 177f2dde3 
  build-tools/pom.xml 01c573258 
  client/client-v1/pom.xml fc80a4294 
  client/client-v2/pom.xml deb991001 
  client/common/pom.xml e9959bf47 
  client/pom.xml 5057e4c02 
  common/pom.xml 554ab67d2 
  dashboardv2/pom.xml 965e5856e 
  dashboardv3/pom.xml 2e1729988 
  dev-support/atlas-docker/.env ca597ac04 
  distro/pom.xml a892736de 
  docs/pom.xml 9b99d5caf 
  graphdb/api/pom.xml 741cf8d89 
  graphdb/common/pom.xml 0e96d46f9 
  graphdb/graphdb-impls/pom.xml 897a48dc8 
  graphdb/janus-hbase2/pom.xml 4da5ae8e4 
  graphdb/janus/pom.xml 75eed0f16 
  graphdb/pom.xml 0fde3510a 
  intg/pom.xml 3a34983a5 
  notification/pom.xml e8abd453e 
  plugin-classloader/pom.xml 2057d3554 
  pom.xml 4cc2201c1 
  repository/pom.xml eeb7af9de 
  server-api/pom.xml 0c5e66ec2 
  test-tools/pom.xml 6622512fe 
  tools/atlas-index-repair/pom.xml 9298f91ba 
  tools/classification-updater/pom.xml 04e766bd2 
  webapp/pom.xml 30c7b416b 


Diff: https://reviews.apache.org/r/74205/diff/1/


Testing
---

verified that Atlas builds successfully


Thanks,

Madhan Neethiraj



Re: Review Request 74201: ATLAS-4445: fix import-hive.sh failure due to NoClassDefFoundError: javax/ws/rs/core/Link$Builder

2022-11-16 Thread Madhan Neethiraj


> On Nov. 16, 2022, 11:33 p.m., Sidharth Mishra wrote:
> > addons/hive-bridge/src/bin/import-hive.sh
> > Line 97 (original), 97 (patched)
> > <https://reviews.apache.org/r/74201/diff/1/?file=2271812#file2271812line97>
> >
> > javax.ws.rs-api was added to exclude list to fix 
> > https://issues.apache.org/jira/browse/ATLAS-4354. Could you please confirm 
> > if this is not needed to be excluded?

Prior to this patch, jsr311-api-1.1 library was packaged in Atlas Hive hook, 
and javax.ws.rs-api library from Hive lib was excluded from import-hive 
classpath. This resulted in import-hive to fail with error 
"java.lang.NoClassDefFoundError: javax/ws/rs/core/Link$Builder".

Addressing this failure requires jsr311-api to be not packaged with Atlas Hive 
hook, and include javax.ws.rs-api library from Hive lib to import-hive 
classpath.


- Madhan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74201/#review224868
-------


On Nov. 15, 2022, 8:08 p.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74201/
> ---
> 
> (Updated Nov. 15, 2022, 8:08 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, keval bhatt, 
> Nikhil Bonte, Nixon Rodrigues, Pinal Shah, Radhika Kundam, Ramesh Mani, 
> Sarath Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-4445
> https://issues.apache.org/jira/browse/ATLAS-4445
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> - updated hive version from 3.1.0 to 3.1.2
> - removed packaging jsr311-api in Hive hook
> 
> 
> Diffs
> -
> 
>   addons/hive-bridge/pom.xml 856e5d95d 
>   addons/hive-bridge/src/bin/import-hive.sh ebe6976f0 
>   pom.xml ca221fe29 
> 
> 
> Diff: https://reviews.apache.org/r/74201/diff/1/
> 
> 
> Testing
> ---
> 
> - verified import-hive.sh in docker setup
> - all existing tests pass successfully
> - pre-commit tests run: 
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1288/
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>



Review Request 74204: ATLAS-3820: fix import-kafka.sh failures

2022-11-15 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74204/
---

Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nikhil Bonte, Nixon 
Rodrigues, Pinal Shah, Radhika Kundam, Ramesh Mani, Sarath Subramanian, 
Sharmadha Sainath, Sidharth Mishra, and Umesh Padashetty.


Bugs: ATLAS-3820
https://issues.apache.org/jira/browse/ATLAS-3820


Repository: atlas


Description
---

- added missing libraries to Kafka bridge package
- updated docker setup to include Kafka bridge


Diffs
-

  addons/kafka-bridge/pom.xml 9ce22a757 
  addons/kafka-bridge/src/bin/import-kafka.sh ada3b4ae2 
  
addons/kafka-bridge/src/main/java/org/apache/atlas/kafka/bridge/KafkaBridge.java
 8be2fca3a 
  dev-support/atlas-docker/.dockerignore d4a3db26b 
  dev-support/atlas-docker/Dockerfile.atlas-kafka e06d62845 
  dev-support/atlas-docker/scripts/atlas-build.sh e53ebbba9 
  dev-support/atlas-docker/scripts/atlas-kafka-application.properties 
PRE-CREATION 


Diff: https://reviews.apache.org/r/74204/diff/1/


Testing
---

- verified that import-kafka.sh sucessfully added Kafka topics to Atlas


Thanks,

Madhan Neethiraj



[jira] [Assigned] (ATLAS-3820) run import-kafka.sh throws Expection

2022-11-15 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj reassigned ATLAS-3820:
---

Assignee: Madhan Neethiraj

> run import-kafka.sh throws Expection
> 
>
> Key: ATLAS-3820
> URL: https://issues.apache.org/jira/browse/ATLAS-3820
> Project: Atlas
>  Issue Type: Bug
>  Components: kafka-integration
>Affects Versions: 1.0.0
> Environment: Kafka version: 1.1.0
>Reporter: fa
>Assignee: Madhan Neethiraj
>Priority: Major
>
> maven build command: mvn clean -DskipTests package -Pdist,embedded-hbase-solr
> I run `bin/atlas_stat.py`, and I can see the web-ui.
> Then I run 
> apache-atlas-1.0.0-kafka-hook/apache-atlas-kafka-hook-1.0.0/hook-bin/import-kafka.sh.
> stack trace
> Exception in thread "main" java.lang.NoClassDefFoundError: 
> org/apache/commons/lang/exception/NestableRuntimeExceptionException in thread 
> "main" java.lang.NoClassDefFoundError: 
> org/apache/commons/lang/exception/NestableRuntimeException at 
> java.lang.ClassLoader.defineClass1(Native Method) at 
> java.lang.ClassLoader.defineClass(ClassLoader.java:763) at 
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at 
> java.net.URLClassLoader.defineClass(URLClassLoader.java:468) at 
> java.net.URLClassLoader.access$100(URLClassLoader.java:74) at 
> java.net.URLClassLoader$1.run(URLClassLoader.java:369) at 
> java.net.URLClassLoader$1.run(URLClassLoader.java:363) at 
> java.security.AccessController.doPrivileged(Native Method) at 
> java.net.URLClassLoader.findClass(URLClassLoader.java:362) at 
> java.lang.ClassLoader.loadClass(ClassLoader.java:424) at 
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at 
> java.lang.ClassLoader.loadClass(ClassLoader.java:357) at 
> java.lang.ClassLoader.defineClass1(Native Method) at 
> java.lang.ClassLoader.defineClass(ClassLoader.java:763) at 
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at 
> java.net.URLClassLoader.defineClass(URLClassLoader.java:468) at 
> java.net.URLClassLoader.access$100(URLClassLoader.java:74) at 
> java.net.URLClassLoader$1.run(URLClassLoader.java:369) at 
> java.net.URLClassLoader$1.run(URLClassLoader.java:363) at 
> java.security.AccessController.doPrivileged(Native Method) at 
> java.net.URLClassLoader.findClass(URLClassLoader.java:362) at 
> java.lang.ClassLoader.loadClass(ClassLoader.java:424) at 
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at 
> java.lang.ClassLoader.loadClass(ClassLoader.java:357) at 
> org.apache.atlas.kafka.bridge.KafkaBridge.main(KafkaBridge.java:100)Caused 
> by: java.lang.ClassNotFoundException: 
> org.apache.commons.lang.exception.NestableRuntimeException at 
> java.net.URLClassLoader.findClass(URLClassLoader.java:382) at 
> java.lang.ClassLoader.loadClass(ClassLoader.java:424) at 
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at 
> java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 25 more



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Review Request 74203: ATLAS-4241: fix import-hbase errors

2022-11-15 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74203/
---

Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nikhil Bonte, Nixon 
Rodrigues, Pinal Shah, Radhika Kundam, Ramesh Mani, Sarath Subramanian, and 
Sidharth Mishra.


Bugs: ATLAS-4241
https://issues.apache.org/jira/browse/ATLAS-4241


Repository: atlas


Description
---

- fixed classpath for import-hbase to exclude couple of libraries present in 
HBase lib directory
- added following libraries in HBase plugin impl: commons-collections, 
commons-logging, jackson-jaxrs-json-provider, jackson-jaxrs-base, jersey-bundle


Diffs
-

  addons/hbase-bridge/pom.xml f51a7d6f1 
  addons/hbase-bridge/src/bin/import-hbase.sh a343036fa 


Diff: https://reviews.apache.org/r/74203/diff/1/


Testing
---

verified the following using Docker setup
 - HBase hook successfully reports changes to Atlas server
 - import-hbase utility successfully imports namespaces and tables to Atlas 
server


Thanks,

Madhan Neethiraj



[jira] [Assigned] (ATLAS-4241) import-hbase.sh get map error

2022-11-15 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj reassigned ATLAS-4241:
---

Assignee: Madhan Neethiraj  (was: Radhika Kundam)

> import-hbase.sh get map error
> -
>
> Key: ATLAS-4241
> URL: https://issues.apache.org/jira/browse/ATLAS-4241
> Project: Atlas
>  Issue Type: Bug
>  Components: hbase-integration
>Affects Versions: 2.1.0
>Reporter: junsionzhang
>Assignee: Madhan Neethiraj
>Priority: Major
>
> when run import-hbase.sh ,errors is as follows:
> 2021-04-12 14:34:25,865 ERROR - [main:] ~ ImportHBaseEntities failed 
> (HBaseBridge:197)
> org.apache.atlas.AtlasException: Failed to load application properties
>  at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:147)
>  at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:100)
>  at org.apache.atlas.hbase.bridge.HBaseBridge.main(HBaseBridge.java:139)
> Caused by: org.apache.commons.configuration.ConversionException: 
> 'atlas.graph.index.search.solr.wait-searcher' doesn't map to a List object: 
> true, a java.lang.Boolean
>  at 
> org.apache.commons.configuration.AbstractConfiguration.getList(AbstractConfiguration.java:1144)
>  at 
> org.apache.commons.configuration.AbstractConfiguration.getList(AbstractConfiguration.java:1109)
>  at 
> org.apache.commons.configuration.AbstractConfiguration.interpolatedConfiguration(AbstractConfiguration.java:1274)
>  at org.apache.atlas.ApplicationProperties.get(ApplicationProperties.java:142)
>  ... 2 more
>  
> It seems the common-configuration does not work correctly . the hbasebridge 
> use common-configuration 1.10 ,but the version also exists.  could somebody 
> upgrade from 1.x to 2.x
>  
> https://commons.apache.org/proper/commons-configuration/userguide/upgradeto2_0.html



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Review Request 74202: ATLAS-4713: docker setup updates to be in sync with component versions in project pom.xml

2022-11-15 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74202/
---

Review request for atlas, Ashutosh Mestry, Jayendra Parab, Mandar Ambawane, 
Nikhil Bonte, Nixon Rodrigues, Pinal Shah, Radhika Kundam, Sarath Subramanian, 
and Sidharth Mishra.


Bugs: ATLAS-4713
https://issues.apache.org/jira/browse/ATLAS-4713


Repository: atlas


Description
---

- updated version of Kafka, Hive-hadoop
- fixed incorrect Atlas URL in hook configuration setup


Diffs
-

  dev-support/atlas-docker/.env 19175c2ed 
  dev-support/atlas-docker/scripts/atlas-hbase-application.properties 8c5b9e088 
  dev-support/atlas-docker/scripts/atlas-hive-application.properties 8c5b9e088 


Diff: https://reviews.apache.org/r/74202/diff/1/


Testing
---

- verified that Atlas docker setup builds successfully
- performed sanity testing of Atlas, Hive hook, HBase hook


Thanks,

Madhan Neethiraj



[jira] [Created] (ATLAS-4713) docker setup - update component versions

2022-11-15 Thread Madhan Neethiraj (Jira)
Madhan Neethiraj created ATLAS-4713:
---

 Summary: docker setup - update component versions
 Key: ATLAS-4713
 URL: https://issues.apache.org/jira/browse/ATLAS-4713
 Project: Atlas
  Issue Type: Bug
  Components: atlas-docker
Reporter: Madhan Neethiraj
Assignee: Madhan Neethiraj


Docker setup needs updates to be in sync with component versions in project 
pom.xml file. For example:
 * ATLAS_VERSION=2.2.0 should be 2.3.0-SNAPSHOT
 * KAFKA_VERSION=2.5.0 should be 2.8.1
 * HIVE_HADOOP_VERSION=3.1.0 should be 3.1.1



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Review Request 74201: ATLAS-4445: fix import-hive.sh failure due to NoClassDefFoundError: javax/ws/rs/core/Link$Builder

2022-11-15 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74201/
---

Review request for atlas, Ashutosh Mestry, Jayendra Parab, keval bhatt, Nikhil 
Bonte, Nixon Rodrigues, Pinal Shah, Radhika Kundam, Ramesh Mani, Sarath 
Subramanian, and Sidharth Mishra.


Bugs: ATLAS-4445
https://issues.apache.org/jira/browse/ATLAS-4445


Repository: atlas


Description
---

- updated hive version from 3.1.0 to 3.1.2
- removed packaging jsr311-api in Hive hook


Diffs
-

  addons/hive-bridge/pom.xml 856e5d95d 
  addons/hive-bridge/src/bin/import-hive.sh ebe6976f0 
  pom.xml ca221fe29 


Diff: https://reviews.apache.org/r/74201/diff/1/


Testing
---

- verified import-hive.sh in docker setup
- all existing tests pass successfully
- pre-commit tests run: 
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1286/


Thanks,

Madhan Neethiraj



[jira] [Assigned] (ATLAS-4445) unable to run import-hive.sh successfully

2022-11-15 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj reassigned ATLAS-4445:
---

Assignee: Madhan Neethiraj

> unable to run import-hive.sh successfully
> -
>
> Key: ATLAS-4445
> URL: https://issues.apache.org/jira/browse/ATLAS-4445
> Project: Atlas
>  Issue Type: Bug
>  Components: hive-integration
>Affects Versions: 2.2.0
> Environment: RHEL 7.x
>Reporter: Tim Hunter
>Assignee: Madhan Neethiraj
>Priority: Major
>
> Same configuration this worked in 2.1.0, unable to figure out why it doesn't 
> in 2.2.0
>  
> {{$ ./import-hive.sh $ ./import-hive.sh Using Hive configuration directory 
> [/usr/local/hive/conf]Log file for import is 
> /var/log/atlas/import-hive.logSLF4J: Class path contains multiple SLF4J 
> bindings.SLF4J: Found binding in 
> [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J:
>  Found binding in 
> [jar:file:/home/ec2-user/hadoop-3.3.1/share/hadoop/common/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]SLF4J:
>  See http://www.slf4j.org/codes.html#multiple_bindings for an 
> explanation.SLF4J: Actual binding is of type 
> [org.apache.logging.slf4j.Log4jLoggerFactory]2021-10-05T12:57:32,810 INFO 
> [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - delete non existing 
> flag : false 2021-10-05T12:57:32,841 INFO [main] 
> org.apache.atlas.ApplicationProperties - Looking for 
> atlas-application.properties in classpath2021-10-05T12:57:32,843 INFO [main] 
> org.apache.atlas.ApplicationProperties - Loading atlas-application.properties 
> from 
> file:/opt/atlas/apache-atlas-2.2.0/conf/atlas-application.properties2021-10-05T12:57:32,903
>  INFO [main] org.apache.atlas.ApplicationProperties - Using graphdb backend 
> 'janus'2021-10-05T12:57:32,903 INFO [main] 
> org.apache.atlas.ApplicationProperties - Using storage backend 
> 'hbase2'2021-10-05T12:57:32,903 INFO [main] 
> org.apache.atlas.ApplicationProperties - Using index backend 
> 'solr'2021-10-05T12:57:32,903 INFO [main] 
> org.apache.atlas.ApplicationProperties - Atlas is running in MODE: 
> PROD.2021-10-05T12:57:32,904 INFO [main] 
> org.apache.atlas.ApplicationProperties - Setting solr.wait-searcher property 
> 'false'2021-10-05T12:57:32,904 INFO [main] 
> org.apache.atlas.ApplicationProperties - Setting index.search.map-name 
> property 'false'2021-10-05T12:57:32,904 INFO [main] 
> org.apache.atlas.ApplicationProperties - Setting 
> atlas.graph.index.search.max-result-set-size = 1502021-10-05T12:57:32,904 
> INFO [main] org.apache.atlas.ApplicationProperties - Setting 
> atlas.graph.index.search.solr.wait-searcher = false2021-10-05T12:57:32,905 
> INFO [main] org.apache.atlas.ApplicationProperties - Property (set to 
> default) atlas.graph.cache.db-cache = true2021-10-05T12:57:32,905 INFO [main] 
> org.apache.atlas.ApplicationProperties - Property (set to default) 
> atlas.graph.cache.db-cache-clean-wait = 202021-10-05T12:57:32,905 INFO [main] 
> org.apache.atlas.ApplicationProperties - Property (set to default) 
> atlas.graph.cache.db-cache-size = 0.52021-10-05T12:57:32,905 INFO [main] 
> org.apache.atlas.ApplicationProperties - Property (set to default) 
> atlas.graph.cache.tx-cache-size = 150002021-10-05T12:57:32,905 INFO [main] 
> org.apache.atlas.ApplicationProperties - Property (set to default) 
> atlas.graph.cache.tx-dirty-size = 120Enter username for atlas :- adminEnter 
> password for atlas :- Exception in thread "main" 
> java.lang.NoClassDefFoundError: javax/ws/rs/core/Link$Builder at 
> java.lang.Class.getDeclaredConstructors0(Native Method) at 
> java.lang.Class.privateGetDeclaredConstructors(Class.java:2671) at 
> java.lang.Class.getConstructor0(Class.java:3075) at 
> java.lang.Class.newInstance(Class.java:412) at 
> javax.ws.rs.ext.FactoryFinder.newInstance(FactoryFinder.java:65) at 
> javax.ws.rs.ext.FactoryFinder.find(FactoryFinder.java:117) at 
> javax.ws.rs.ext.RuntimeDelegate.findDelegate(RuntimeDelegate.java:105) at 
> javax.ws.rs.ext.RuntimeDelegate.getInstance(RuntimeDelegate.java:91) at 
> javax.ws.rs.core.MediaType.(MediaType.java:44) at 
> com.sun.jersey.core.header.MediaTypes.(MediaTypes.java:65) at 
> com.sun.jersey.core.spi.factory.MessageBodyFactory.initReaders(MessageBodyFactory.java:182)
>  at 
> com.sun.jersey.core.spi.factory.MessageBodyFactory.initReaders(MessageBodyFactory.java:175)
>  at 
> com.sun.jersey.core.spi.factory.MessageBodyFactory.init(MessageBodyFactory.java:162)
> 

[jira] [Resolved] (ATLAS-2388) Disabled Unit Test: InMemoryJAASConfigurationTest.java:31

2022-11-09 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-2388.
-
Fix Version/s: 3.0.0
   2.3.0
   (was: 1.0.0)
   Resolution: Fixed

> Disabled Unit Test: InMemoryJAASConfigurationTest.java:31
> -
>
> Key: ATLAS-2388
> URL: https://issues.apache.org/jira/browse/ATLAS-2388
> Project: Atlas
>  Issue Type: Bug
>Reporter: Graham Wallis
>Priority: Blocker
> Fix For: 3.0.0, 2.3.0
>
>
> The above test needs to be reviewed and should probably be re-enabled



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Resolved] (ATLAS-4709) Python client version 0.0.12

2022-11-08 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-4709.
-
Fix Version/s: 3.0.0
   2.3.0
   Resolution: Fixed

> Python client version 0.0.12
> 
>
> Key: ATLAS-4709
> URL: https://issues.apache.org/jira/browse/ATLAS-4709
> Project: Atlas
>  Issue Type: Task
>  Components: atlas-intg
>    Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Following fixes were added since Apache Atlas Python client release - 0.0.11 :
>  * ATLAS-4205: fixes in Python sample-app and Python client
>  * ATLAS-4318 : fix python client api url concatenation
>  * ATLAS-4318 : Fixed bug in entity.py
>  * ATLAS-4590: fix minor problem in list_attributes_to_params()
>  * ATLAS-4676: do not expect response body to be a valid JSON
>  * ATLAS-4704: Remove os.path.join causing incorrect windows path
> This Jira tracks release of Apache Atlas Python client version 0.0.12



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (ATLAS-4709) Python client version 0.0.12

2022-11-08 Thread Madhan Neethiraj (Jira)
Madhan Neethiraj created ATLAS-4709:
---

 Summary: Python client version 0.0.12
 Key: ATLAS-4709
 URL: https://issues.apache.org/jira/browse/ATLAS-4709
 Project: Atlas
  Issue Type: Task
  Components: atlas-intg
Reporter: Madhan Neethiraj
Assignee: Madhan Neethiraj


Following fixes were added since Apache Atlas Python client release - 0.0.11 :
 * ATLAS-4205: fixes in Python sample-app and Python client
 * ATLAS-4318 : fix python client api url concatenation
 * ATLAS-4318 : Fixed bug in entity.py
 * ATLAS-4590: fix minor problem in list_attributes_to_params()
 * ATLAS-4676: do not expect response body to be a valid JSON
 * ATLAS-4704: Remove os.path.join causing incorrect windows path

This Jira tracks release of Apache Atlas Python client version 0.0.12



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (ATLAS-4707) update UI component version for scss-tokenizer, node-sass and loader-utils

2022-11-08 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-4707:

Summary: update UI component version for scss-tokenizer, node-sass and 
loader-utils  (was: update UI component version for scss-tokenizer and 
node-sass )

> update UI component version for scss-tokenizer, node-sass and loader-utils
> --
>
> Key: ATLAS-4707
> URL: https://issues.apache.org/jira/browse/ATLAS-4707
> Project: Atlas
>  Issue Type: Improvement
>  Components: atlas-webui
>    Reporter: Madhan Neethiraj
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> This Jira tracks version updates in following pull requests:
>  - [pull request #202|https://github.com/apache/atlas/pull/202]
>  - [pull request #201|https://github.com/apache/atlas/pull/201]
>  - [pull request #205|https://github.com/apache/atlas/pull/205]
>  - [pull request #206|https://github.com/apache/atlas/pull/206]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (ATLAS-4707) update UI component version for scss-tokenizer and node-sass

2022-11-08 Thread Madhan Neethiraj (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-4707:

Description: 
This Jira tracks version updates in following pull requests:
 - [pull request #202|https://github.com/apache/atlas/pull/202]
 - [pull request #201|https://github.com/apache/atlas/pull/201]
 - [pull request #205|https://github.com/apache/atlas/pull/205]
 - [pull request #206|https://github.com/apache/atlas/pull/206]

  was:
This Jira tracks version updates in following pull requests:
- [pull request #202|https://github.com/apache/atlas/pull/202]
- [pull request #201|https://github.com/apache/atlas/pull/201]


> update UI component version for scss-tokenizer and node-sass 
> -
>
> Key: ATLAS-4707
> URL: https://issues.apache.org/jira/browse/ATLAS-4707
> Project: Atlas
>  Issue Type: Improvement
>  Components: atlas-webui
>    Reporter: Madhan Neethiraj
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> This Jira tracks version updates in following pull requests:
>  - [pull request #202|https://github.com/apache/atlas/pull/202]
>  - [pull request #201|https://github.com/apache/atlas/pull/201]
>  - [pull request #205|https://github.com/apache/atlas/pull/205]
>  - [pull request #206|https://github.com/apache/atlas/pull/206]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Review Request 74197: ATLAS-4707: updated loader-utils version from 1.4.0 to 1.4.1

2022-11-08 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74197/
---

Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nikhil Bonte, Nixon 
Rodrigues, Pinal Shah, Prasad Pawar, Radhika Kundam, Sarath Subramanian, and 
Sidharth Mishra.


Bugs: ATLAS-4707
https://issues.apache.org/jira/browse/ATLAS-4707


Repository: atlas


Description
---

Patch from following pull requests:
- https://github.com/apache/atlas/pull/205
- https://github.com/apache/atlas/pull/206


Diffs
-

  dashboardv2/public/js/external_lib/atlas-lineage/package-lock.json 890350198 
  dashboardv3/public/js/external_lib/atlas-lineage/package-lock.json 890350198 


Diff: https://reviews.apache.org/r/74197/diff/1/


Testing
---

- sanity testing of Atlas UI


Thanks,

Madhan Neethiraj



Review Request 74193: ATLAS-4707: updated versions of scss-tokenizer 0.2.3 to 0.4.3, node-sass 4.13.1 to 7.0.3

2022-11-07 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74193/
---

Review request for atlas, Ashutosh Mestry, Jayendra Parab, keval bhatt, Nikhil 
Bonte, Nixon Rodrigues, Pinal Shah, Prasad Pawar, Radhika Kundam, Sarath 
Subramanian, and Sidharth Mishra.


Bugs: ATLAS-4707
https://issues.apache.org/jira/browse/ATLAS-4707


Repository: atlas


Description
---

Patch from following pull requests:
- https://github.com/apache/atlas/pull/201
- https://github.com/apache/atlas/pull/202


Diffs
-

  dashboardv2/package-lock.json 232376eab 
  dashboardv2/package.json 393d4c319 
  dashboardv3/package-lock.json 35b46349b 
  dashboardv3/package.json 8c0abc697 


Diff: https://reviews.apache.org/r/74193/diff/1/


Testing
---

- sanity testing of Atlas UI


Thanks,

Madhan Neethiraj



[jira] [Created] (ATLAS-4707) update UI component version for scss-tokenizer and node-sass

2022-11-07 Thread Madhan Neethiraj (Jira)
Madhan Neethiraj created ATLAS-4707:
---

 Summary: update UI component version for scss-tokenizer and 
node-sass 
 Key: ATLAS-4707
 URL: https://issues.apache.org/jira/browse/ATLAS-4707
 Project: Atlas
  Issue Type: Improvement
  Components: atlas-webui
Reporter: Madhan Neethiraj


This Jira tracks version updates in following pull requests:
- [pull request #202|https://github.com/apache/atlas/pull/202]
- [pull request #201|https://github.com/apache/atlas/pull/201]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Apache Atlas 2.3.0 release plan

2022-11-07 Thread Madhan Neethiraj
All,

 

I am excited to announce that Apache Atlas community has started on releasing 
Apache Atlas 2.3.0!

 

Please review the following brief plan and let me know if you have any comments:

 

1. 2022-11-30, Wed: tentative release date

2. 2022-11-11, Fri: release branch will be closed for commits. After this date, 
commits will require approval from the release manager.

3. 2022-11-15, Tue, first release candidate out for vote

4. 2022-11-25: tentative vote completion

5. Release will be from branch-2.0 – the same branch used for earlier 2.x 
releases.

 

Looking forward to your help in making 2.3.0 release as planned.

 

Thanks,

Madhan

 



  1   2   3   4   5   6   7   8   9   10   >