Re: Review Request 72627: ATLAS-3867 : Relationship search API should have a provision to fetch custom attributes in search results

2020-06-30 Thread Sarath Subramanian

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72627/#review221106
---


Ship it!




Ship It!

- Sarath Subramanian


On June 30, 2020, 10:26 p.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72627/
> ---
> 
> (Updated June 30, 2020, 10:26 p.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3867
> https://issues.apache.org/jira/browse/ATLAS-3867
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Issue:** Attributes in search result of Relationship api are minimal.
> 
> **WorkAround:** Relationship api request should have provision to specify 
> attributes to be present in search result.
> 
> **Example Request:** 
> /v2/search/relationship?guid=ac9e04cc-f927-4334-af08-c83bc3733f5b=columns=name=ASCENDING=dcProfiledData
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java
>  e64c31522 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  4169e14ef 
>   webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java 076284ec4 
> 
> 
> Diff: https://reviews.apache.org/r/72627/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested
> Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1994
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72631: ATLAS-3871 Add unit tests to path extractor utility for s3, s3 v2, adls path entities

2020-06-30 Thread Sarath Subramanian

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72631/#review221105
---


Ship it!




Ship It!

- Sarath Subramanian


On June 30, 2020, 9:29 p.m., Nikhil Bonte wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72631/
> ---
> 
> (Updated June 30, 2020, 9:29 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
> Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3871
> https://issues.apache.org/jira/browse/ATLAS-3871
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> AtlasPathExtractorUtilTest added in ATLAS-3836 which covers unit tests for 
> ozone & hdfs path entites.
>  
> This will improve AtlasPathExtractorUtilTest.java to cover s3, s3v2 & adls 
> path entities.
> 
> Would add more tests for ozone as well.
> 
> 
> Diffs
> -
> 
>   common/src/test/java/org/apache/atlas/utils/AtlasPathExtractorUtilTest.java 
> 664bfb72b 
> 
> 
> Diff: https://reviews.apache.org/r/72631/diff/2/
> 
> 
> Testing
> ---
> 
> PC build 
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2001/console 
> (Finished: SUCCESS)
> 
> 
> Thanks,
> 
> Nikhil Bonte
> 
>



Re: Review Request 72627: ATLAS-3867 : Relationship search API should have a provision to fetch custom attributes in search results

2020-06-30 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72627/
---

(Updated July 1, 2020, 5:26 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

rebase patch


Bugs: ATLAS-3867
https://issues.apache.org/jira/browse/ATLAS-3867


Repository: atlas


Description
---

**Issue:** Attributes in search result of Relationship api are minimal.

**WorkAround:** Relationship api request should have provision to specify 
attributes to be present in search result.

**Example Request:** 
/v2/search/relationship?guid=ac9e04cc-f927-4334-af08-c83bc3733f5b=columns=name=ASCENDING=dcProfiledData


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java 
e64c31522 
  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
4169e14ef 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java 076284ec4 


Diff: https://reviews.apache.org/r/72627/diff/2/

Changes: https://reviews.apache.org/r/72627/diff/1-2/


Testing
---

Manually tested
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1994


Thanks,

Pinal Shah



Re: Review Request 72625: ATLAS-3866 : Relationship search API for hive storage desc throws error code 500

2020-06-30 Thread Sarath Subramanian

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72625/#review221104
---


Ship it!




Ship It!

- Sarath Subramanian


On June 29, 2020, 3:49 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72625/
> ---
> 
> (Updated June 29, 2020, 3:49 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3866
> https://issues.apache.org/jira/browse/ATLAS-3866
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Issue:** In relationship api v2/relationship, for hive_storagedesc it 
> throws exception
> 
> **Reason:** it is because, sort attribute is assigned 'name' as default, and 
> 'name' attribute is not in the defination of hive_storagedesc.
> 
> **Workaround:** Validate if attribute is present in relationship end def, if 
> not ignore sorting
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  dd4d1b441 
> 
> 
> Diff: https://reviews.apache.org/r/72625/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested
> Precommit : 
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1992/console
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



[jira] [Commented] (ATLAS-3869) Upgrade httpclient from 4.5.3 to 4.5.4

2020-06-30 Thread Nikhil Bonte (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149096#comment-17149096
 ] 

Nikhil Bonte commented on ATLAS-3869:
-

+1 for the patch.

Thanks [~nixon] .

> Upgrade httpclient from 4.5.3 to 4.5.4
> --
>
> Key: ATLAS-3869
> URL: https://issues.apache.org/jira/browse/ATLAS-3869
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>Priority: Major
> Attachments: ATLAS-3869-Upgrade-httpclient-from-4.5.3-to-4.5.4.patch
>
>
> Found below exception where data was not getting added in SOLR from 
> janusgraph.
>  
> {noformat}
> Caused by: java.lang.ClassCastException: [B cannot be cast to java.lang.String
>   at 
> org.apache.http.conn.ssl.DefaultHostnameVerifier.getSubjectAltNames(DefaultHostnameVerifier.java:309)
>   at 
> org.apache.http.conn.ssl.DefaultHostnameVerifier.verify(DefaultHostnameVerifier.java:112)
>   at 
> org.apache.http.conn.ssl.DefaultHostnameVerifier.verify(DefaultHostnameVerifier.java:99)
>   at 
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.verifyHostname(SSLConnectionSocketFactory.java:463)
>   at 
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:397)
>   at 
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355)
>   at 
> org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
>   at 
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359)
>   at 
> org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
>   at 
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
>   at 
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
>   at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
>   at 
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
>   at 
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
>   at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
>   at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:526)
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
>   at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
>   ... 7 more
> 2020-06-29 07:31:25,893 WARN  - [SIGTERM handler:] ~ Unable to close 
> transaction standardjanusgraphtx[0x18d7a16a] (StandardJanusGraph:230)
> java.lang.IllegalArgumentException: The transaction has already been closed
>   at 
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:142)
>   at 
> org.janusgraph.graphdb.transaction.StandardJanusGraphTx.rollback(StandardJanusGraphTx.java:1401)
>   at 
> org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsTransaction$1.doRollback(JanusGraphBlueprintsTransaction.java:190)
>   at 
> org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.rollback(AbstractTransaction.java:114)
>   at 
> org.apache.tinkerpop.gremlin.structure.Transaction$CLOSE_BEHAVIOR$2.accept(Transaction.java:182)
>   at 
> org.apache.tinkerpop.gremlin.structure.Transaction$CLOSE_BEHAVIOR$2.accept(Transaction.java:179)
>   at 
> org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.close(JanusGraphBlueprintsGraph.java:327)
>   at 
> org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsTransaction$1.doClose(JanusGraphBlueprintsTransaction.java:205)
>   at 
> org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.close(AbstractTransaction.java:139)
>   at 
> org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsTransaction.close(JanusGraphBlueprintsTransaction.java:215)
>   at 
> org.janusgraph.graphdb.database.StandardJanusGraph.closeInternal(StandardJanusGraph.java:226)
>   at 
> org.janusgraph.graphdb.database.StandardJanusGraph.access$500(StandardJanusGraph.java:91)
>   at 
> org.janusgraph.graphdb.database.StandardJanusGraph$ShutdownThread.start(StandardJanusGraph.java:818)
>   at 
> java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:102)
>   at 
> java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:46)
>   at java.lang.Shutdown.runHooks(Shutdown.java:123)
>  

Re: Review Request 72631: ATLAS-3871 Add unit tests to path extractor utility for s3, s3 v2, adls path entities

2020-06-30 Thread Nikhil Bonte

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72631/
---

(Updated July 1, 2020, 4:29 a.m.)


Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.


Changes
---

Updated pre-commit build status


Bugs: ATLAS-3871
https://issues.apache.org/jira/browse/ATLAS-3871


Repository: atlas


Description
---

AtlasPathExtractorUtilTest added in ATLAS-3836 which covers unit tests for 
ozone & hdfs path entites.
 
This will improve AtlasPathExtractorUtilTest.java to cover s3, s3v2 & adls path 
entities.

Would add more tests for ozone as well.


Diffs
-

  common/src/test/java/org/apache/atlas/utils/AtlasPathExtractorUtilTest.java 
664bfb72b 


Diff: https://reviews.apache.org/r/72631/diff/2/


Testing (updated)
---

PC build https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2001/console 
(Finished: SUCCESS)


Thanks,

Nikhil Bonte



[jira] [Assigned] (ATLAS-3872) While creating Column entity shows error message

2020-06-30 Thread Mandar Ambawane (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mandar Ambawane reassigned ATLAS-3872:
--

Assignee: Mandar Ambawane

> While creating Column entity shows error message
> 
>
> Key: ATLAS-3872
> URL: https://issues.apache.org/jira/browse/ATLAS-3872
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Durga Kadam
>Assignee: Mandar Ambawane
>Priority: Minor
> Attachments: column_entity_name_field_issue.ogv
>
>
> Description:: While creating Column entity(whose entity-type has been 
> generated from running quick_start.py) if user does not provide the 'name'(by 
> enabling the toggle 'Required All') it shows below error message.
> Error message:: 
> Invalid instance creation/updation parameters passed : Column.name: mandatory 
> attribute value missing in type Asset
> Steps to reproduce issue::
>  # Go to Create  entity
>  # Select 'Column' type
>  # Enter 'QualifiedName(string)' and select the 'Table(Table)' from drop-down
>  # Click on Create buttonIt will show error message
> Note-
>  # If user provides the 'name' which is listed inside the 'Required All' tab 
> it will allow user to save the entity. PFA attached video.
>  # Working fine for Hive_column and Hbase_column as user provides the 
> mandatory name field which is listed outside the 'Required All' tab



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3801) NotificationHookConsumer: Introduce Retry Logic for PermanentLockingException

2020-06-30 Thread Ashutosh Mestry (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Mestry updated ATLAS-3801:
---
Attachment: ATLAS-3801-Notification-hook-processing-concurrent-e.patch

> NotificationHookConsumer: Introduce Retry Logic for PermanentLockingException
> -
>
> Key: ATLAS-3801
> URL: https://issues.apache.org/jira/browse/ATLAS-3801
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: trunk
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
> Fix For: trunk
>
> Attachments: 
> ATLAS-3801-Notification-hook-processing-concurrent-e.patch
>
>
> *Background*
> _NotificationHookConsumer_ when performing graph commit can potentially 
> encounter _JanusGraph's PermanentLockingException_.
> JanusGraph [documentation|https://docs.janusgraph.org/basics/transactions/] 
> details the cases in which this exception occurs. The way past this exception 
> is rollback, pause and replay the data being committed.
> *Solution*
> Introduce the rollback, pause and retry mechanism in 
> _NotificationHookConsumer_.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3870) UI: hive_table entity table not updating queryparams based on filter selected

2020-06-30 Thread Sarath Subramanian (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3870:
--
Component/s: atlas-webui

> UI: hive_table entity table not updating queryparams based on filter selected
> -
>
> Key: ATLAS-3870
> URL: https://issues.apache.org/jira/browse/ATLAS-3870
> Project: Atlas
>  Issue Type: Improvement
>  Components: atlas-webui
>Affects Versions: 2.0.0
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 2.1.0, 3.0.0
>
> Attachments: ATLAS-3870.patch, Screenshot 2020-06-30 at 2.33.11 
> PM.png, Screenshot 2020-06-30 at 2.34.06 PM.png
>
>
> * Go to hive_db entity detail page
>  * Click on Table tab
>  * toggle "Show historical entities" checkbox and check the network for the 
> API
> API always send the same value for
> {code:java}
> excludeDeletedEntities: true
> {code}
> !Screenshot 2020-06-30 at 2.33.11 PM.png|width=374,height=219!
> !Screenshot 2020-06-30 at 2.34.06 PM.png|width=397,height=142!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3870) UI: hive_table entity table not updating queryparams based on filter selected

2020-06-30 Thread Sarath Subramanian (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3870:
--
Labels: ui  (was: )

> UI: hive_table entity table not updating queryparams based on filter selected
> -
>
> Key: ATLAS-3870
> URL: https://issues.apache.org/jira/browse/ATLAS-3870
> Project: Atlas
>  Issue Type: Improvement
>  Components: atlas-webui
>Affects Versions: 2.0.0
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
>  Labels: ui
> Fix For: 2.1.0, 3.0.0
>
> Attachments: ATLAS-3870.patch, Screenshot 2020-06-30 at 2.33.11 
> PM.png, Screenshot 2020-06-30 at 2.34.06 PM.png
>
>
> * Go to hive_db entity detail page
>  * Click on Table tab
>  * toggle "Show historical entities" checkbox and check the network for the 
> API
> API always send the same value for
> {code:java}
> excludeDeletedEntities: true
> {code}
> !Screenshot 2020-06-30 at 2.33.11 PM.png|width=374,height=219!
> !Screenshot 2020-06-30 at 2.34.06 PM.png|width=397,height=142!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3870) UI: hive_table entity table not updating queryparams based on filter selected

2020-06-30 Thread Keval Bhatt (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3870:
---
Attachment: ATLAS-3870.patch

> UI: hive_table entity table not updating queryparams based on filter selected
> -
>
> Key: ATLAS-3870
> URL: https://issues.apache.org/jira/browse/ATLAS-3870
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 2.1.0, 3.0.0
>
> Attachments: ATLAS-3870.patch, Screenshot 2020-06-30 at 2.33.11 
> PM.png, Screenshot 2020-06-30 at 2.34.06 PM.png
>
>
> * Go to hive_db entity detail page
>  * Click on Table tab
>  * toggle "Show historical entities" checkbox and check the network for the 
> API
> API always send the same value for
> {code:java}
> excludeDeletedEntities: true
> {code}
> !Screenshot 2020-06-30 at 2.33.11 PM.png|width=374,height=219!
> !Screenshot 2020-06-30 at 2.34.06 PM.png|width=397,height=142!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (ATLAS-3872) While creating Column entity shows error message

2020-06-30 Thread Durga Kadam (Jira)
Durga Kadam created ATLAS-3872:
--

 Summary: While creating Column entity shows error message
 Key: ATLAS-3872
 URL: https://issues.apache.org/jira/browse/ATLAS-3872
 Project: Atlas
  Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Durga Kadam
 Attachments: column_entity_name_field_issue.ogv

Description:: While creating Column entity(whose entity-type has been generated 
from running quick_start.py) if user does not provide the 'name'(by enabling 
the toggle 'Required All') it shows below error message.

Error message:: 

Invalid instance creation/updation parameters passed : Column.name: mandatory 
attribute value missing in type Asset

Steps to reproduce issue::
 # Go to Create  entity
 # Select 'Column' type
 # Enter 'QualifiedName(string)' and select the 'Table(Table)' from drop-down
 # Click on Create buttonIt will show error message

Note-
 # If user provides the 'name' which is listed inside the 'Required All' tab it 
will allow user to save the entity. PFA attached video.
 # Working fine for Hive_column and Hbase_column as user provides the mandatory 
name field which is listed outside the 'Required All' tab



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: Review Request 72615: ATLAS-3855 Bulk entity tag association and bulk api enhancement

2020-06-30 Thread chaitali

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72615/
---

(Updated June 30, 2020, 2:56 p.m.)


Review request for atlas, Jayendra Parab, Nikhil Bonte, Nixon Rodrigues, and 
Sarath Subramanian.


Bugs: ATLAS-3855
https://issues.apache.org/jira/browse/ATLAS-3855


Repository: atlas


Description
---

API : /api/atlas/v2/entity/bulk/classification & v2/entity/bulk : if some guids 
belong to entities on which user is unauthorized other guids belong to entities 
on which user is authorized This APIs fail with 403 error without returning the 
authorized entities.

1.Unauthorized guids are filtered with this patch for both the APIs.
2.Added ignoreUnauthorisedGuids flag for /bulk/classification API as it doesn't 
return any object
2.Also added unauthorized guids in return object of /bulk in response for 
user's reference.


Diffs
-

  intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 4d8c94894 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java
 7b9455ef3 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
 bf1629cb3 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2Test.java
 b9cbef1b0 
  webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 
00b29e6c8 
  webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 88de8b679 
  webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 
615bc0f1b 


Diff: https://reviews.apache.org/r/72615/diff/4/


Testing
---

Tested with ranger policies:
For /bulk api following policy were applied:
1.Added hive_column entity as entity type.
2.Included admin as user and prohibited access for read entity all this under 
deny policy.

For /bulk/classification following policy were applied:
1.Added hive_column entity as entity type.
2.Gave all the access to admin but kwpt hive_column entity type in excluded 
state.


Thanks,

chaitali



Re: Review Request 72631: ATLAS-3871 Add unit tests to path extractor utility for s3, s3 v2, adls path entities

2020-06-30 Thread Nikhil Bonte

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72631/
---

(Updated June 30, 2020, 1:34 p.m.)


Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.


Changes
---

Added Pre commit build details


Bugs: ATLAS-3871
https://issues.apache.org/jira/browse/ATLAS-3871


Repository: atlas


Description
---

AtlasPathExtractorUtilTest added in ATLAS-3836 which covers unit tests for 
ozone & hdfs path entites.
 
This will improve AtlasPathExtractorUtilTest.java to cover s3, s3v2 & adls path 
entities.

Would add more tests for ozone as well.


Diffs
-

  common/src/test/java/org/apache/atlas/utils/AtlasPathExtractorUtilTest.java 
664bfb72b 


Diff: https://reviews.apache.org/r/72631/diff/2/


Testing (updated)
---

PC build https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2001/console 
(In Progress)


Thanks,

Nikhil Bonte



Re: Review Request 72631: ATLAS-3871 Add unit tests to path extractor utility for s3, s3 v2, adls path entities

2020-06-30 Thread Nikhil Bonte

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72631/
---

(Updated June 30, 2020, 1:30 p.m.)


Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.


Changes
---

Refactorings


Bugs: ATLAS-3871
https://issues.apache.org/jira/browse/ATLAS-3871


Repository: atlas


Description
---

AtlasPathExtractorUtilTest added in ATLAS-3836 which covers unit tests for 
ozone & hdfs path entites.
 
This will improve AtlasPathExtractorUtilTest.java to cover s3, s3v2 & adls path 
entities.

Would add more tests for ozone as well.


Diffs (updated)
-

  common/src/test/java/org/apache/atlas/utils/AtlasPathExtractorUtilTest.java 
664bfb72b 


Diff: https://reviews.apache.org/r/72631/diff/2/

Changes: https://reviews.apache.org/r/72631/diff/1-2/


Testing
---


Thanks,

Nikhil Bonte



Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-06-30 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/
---

(Updated June 30, 2020, 1:09 p.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3782
https://issues.apache.org/jira/browse/ATLAS-3782


Repository: atlas


Description
---

**Issue:**
The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
implemented in SearchProcessors. It would allow a search of entities that do 
not contain in given string in a specified attribute (eg exclude entities from 
search that contain 'temp' in the qualified name). 

**WorkAround:**
JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
inmemory
BasicSearch generates query via three modes
1. Index query -> NOT_CONTAINS will not be supported
2. InMemeory Predicates -> NOT_CONTAINS will be supported, Already handled #123 
SearchProcessor
3. Graph query -> NOT_CONTAINS will not be supported

As in index and graph query wiil not support not_Contains operator, We need to 
apply filter(inMemoryPredicate) after either index/graph query.

To support above, I have modified ClassificationSearchProcessor
+ For both cases index as well as graph, added typeNamePredicate and 
attributePredicate
+ Added these predicate after query
- Removed gremlinQuery block

**Operator Value:**
It can be either "not_contains" or "NOT_CONTAINS"

**Note:**
As part of this jira, not_contains is also added in quick search.


Diffs
-

  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasSolrQueryBuilder.java
 6c06a3cbe 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 9c72cd4a2 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
c9a605355 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
  
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 9b16e919d 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 b7ce97845 


Diff: https://reviews.apache.org/r/72567/diff/4/


Testing (updated)
---

Added testcases
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1999


Thanks,

Pinal Shah



Review Request 72631: ATLAS-3871 Add unit tests to path extractor utility for s3, s3 v2, adls path entities

2020-06-30 Thread Nikhil Bonte

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72631/
---

Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.


Bugs: ATLAS-3871
https://issues.apache.org/jira/browse/ATLAS-3871


Repository: atlas


Description
---

AtlasPathExtractorUtilTest added in ATLAS-3836 which covers unit tests for 
ozone & hdfs path entites.
 
This will improve AtlasPathExtractorUtilTest.java to cover s3, s3v2 & adls path 
entities.

Would add more tests for ozone as well.


Diffs
-

  common/src/test/java/org/apache/atlas/utils/AtlasPathExtractorUtilTest.java 
664bfb72b 


Diff: https://reviews.apache.org/r/72631/diff/1/


Testing
---


Thanks,

Nikhil Bonte



[jira] [Updated] (ATLAS-3871) Add unit tests to path extractor utility for s3, s3 v2, adls path entities

2020-06-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3871:

Description: 
AtlasPathExtractorUtilTest added in ATLAS-3836 which covers unit tests for 
ozone & hdfs path entites.

 

This Jira will improve AtlasPathExtractorUtilTest.java to cover s3, s3v2 & adls 
path entities.

Would add more tests for ozone as well.

  was:
AtlasPathExtractorUtilTest added in ATLAS-3836 which covers ozone & hdfs path 
entites unit tests.

This Jira will improve AtlasPathExtractorUtilTest.java to cover s3, s3v2 & adls 
path entities.

Also will add more tests for ozone path.


> Add unit tests to path extractor utility for s3, s3 v2, adls path entities
> --
>
> Key: ATLAS-3871
> URL: https://issues.apache.org/jira/browse/ATLAS-3871
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
>
> AtlasPathExtractorUtilTest added in ATLAS-3836 which covers unit tests for 
> ozone & hdfs path entites.
>  
> This Jira will improve AtlasPathExtractorUtilTest.java to cover s3, s3v2 & 
> adls path entities.
> Would add more tests for ozone as well.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3871) Add unit tests to path extractor utility for s3, s3 v2, adls path entities

2020-06-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3871:

Description: 
AtlasPathExtractorUtilTest added in ATLAS-3836 which covers ozone & hdfs path 
entites unit tests.

This Jira will improve AtlasPathExtractorUtilTest.java to cover s3, s3v2 & adls 
path entities.

Also will add more tests for ozone path.

  was:
AtlasPathExtractorUtilTest is added in ATLAS-3836 which covers ozone & hdfs 
path entites unit tests.

This Jira will improve AtlasPathExtractorUtilTest.java to cover s3, s3v2 & adls 
path entities.

Also will add more tests for ozone path.


> Add unit tests to path extractor utility for s3, s3 v2, adls path entities
> --
>
> Key: ATLAS-3871
> URL: https://issues.apache.org/jira/browse/ATLAS-3871
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
>
> AtlasPathExtractorUtilTest added in ATLAS-3836 which covers ozone & hdfs path 
> entites unit tests.
> This Jira will improve AtlasPathExtractorUtilTest.java to cover s3, s3v2 & 
> adls path entities.
> Also will add more tests for ozone path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3871) Add unit tests to path extractor utility for s3, s3 v2, adls path entities

2020-06-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3871:

Summary: Add unit tests to path extractor utility for s3, s3 v2, adls path 
entities  (was: Add unit tests for s3, s3 v2, adls path entities in path 
extractor utility)

> Add unit tests to path extractor utility for s3, s3 v2, adls path entities
> --
>
> Key: ATLAS-3871
> URL: https://issues.apache.org/jira/browse/ATLAS-3871
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Nikhil Bonte
>Assignee: Nikhil Bonte
>Priority: Major
>
> AtlasPathExtractorUtilTest is added in ATLAS-3836 which covers ozone & hdfs 
> path entites unit tests.
> This Jira will improve AtlasPathExtractorUtilTest.java to cover s3, s3v2 & 
> adls path entities.
> Also will add more tests for ozone path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (ATLAS-3871) Add unit tests for s3, s3 v2, adls path entities in path extractor utility

2020-06-30 Thread Nikhil Bonte (Jira)
Nikhil Bonte created ATLAS-3871:
---

 Summary: Add unit tests for s3, s3 v2, adls path entities in path 
extractor utility
 Key: ATLAS-3871
 URL: https://issues.apache.org/jira/browse/ATLAS-3871
 Project: Atlas
  Issue Type: Improvement
  Components:  atlas-core
Affects Versions: 2.0.0
Reporter: Nikhil Bonte
Assignee: Nikhil Bonte


AtlasPathExtractorUtilTest is added in ATLAS-3836 which covers ozone & hdfs 
path entites unit tests.

This Jira will improve AtlasPathExtractorUtilTest.java to cover s3, s3v2 & adls 
path entities.

Also will add more tests for ozone path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3869) Upgrade httpclient from 4.5.3 to 4.5.4

2020-06-30 Thread Nixon Rodrigues (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nixon Rodrigues updated ATLAS-3869:
---
Attachment: ATLAS-3869-Upgrade-httpclient-from-4.5.3-to-4.5.4.patch

> Upgrade httpclient from 4.5.3 to 4.5.4
> --
>
> Key: ATLAS-3869
> URL: https://issues.apache.org/jira/browse/ATLAS-3869
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>Priority: Major
> Attachments: ATLAS-3869-Upgrade-httpclient-from-4.5.3-to-4.5.4.patch
>
>
> Found below exception where data was not getting added in SOLR from 
> janusgraph.
>  
> {noformat}
> Caused by: java.lang.ClassCastException: [B cannot be cast to java.lang.String
>   at 
> org.apache.http.conn.ssl.DefaultHostnameVerifier.getSubjectAltNames(DefaultHostnameVerifier.java:309)
>   at 
> org.apache.http.conn.ssl.DefaultHostnameVerifier.verify(DefaultHostnameVerifier.java:112)
>   at 
> org.apache.http.conn.ssl.DefaultHostnameVerifier.verify(DefaultHostnameVerifier.java:99)
>   at 
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.verifyHostname(SSLConnectionSocketFactory.java:463)
>   at 
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:397)
>   at 
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355)
>   at 
> org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
>   at 
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359)
>   at 
> org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
>   at 
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
>   at 
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
>   at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
>   at 
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
>   at 
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
>   at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
>   at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:526)
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
>   at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
>   ... 7 more
> 2020-06-29 07:31:25,893 WARN  - [SIGTERM handler:] ~ Unable to close 
> transaction standardjanusgraphtx[0x18d7a16a] (StandardJanusGraph:230)
> java.lang.IllegalArgumentException: The transaction has already been closed
>   at 
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:142)
>   at 
> org.janusgraph.graphdb.transaction.StandardJanusGraphTx.rollback(StandardJanusGraphTx.java:1401)
>   at 
> org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsTransaction$1.doRollback(JanusGraphBlueprintsTransaction.java:190)
>   at 
> org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.rollback(AbstractTransaction.java:114)
>   at 
> org.apache.tinkerpop.gremlin.structure.Transaction$CLOSE_BEHAVIOR$2.accept(Transaction.java:182)
>   at 
> org.apache.tinkerpop.gremlin.structure.Transaction$CLOSE_BEHAVIOR$2.accept(Transaction.java:179)
>   at 
> org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.close(JanusGraphBlueprintsGraph.java:327)
>   at 
> org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsTransaction$1.doClose(JanusGraphBlueprintsTransaction.java:205)
>   at 
> org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.close(AbstractTransaction.java:139)
>   at 
> org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsTransaction.close(JanusGraphBlueprintsTransaction.java:215)
>   at 
> org.janusgraph.graphdb.database.StandardJanusGraph.closeInternal(StandardJanusGraph.java:226)
>   at 
> org.janusgraph.graphdb.database.StandardJanusGraph.access$500(StandardJanusGraph.java:91)
>   at 
> org.janusgraph.graphdb.database.StandardJanusGraph$ShutdownThread.start(StandardJanusGraph.java:818)
>   at 
> java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:102)
>   at 
> java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:46)
>   at java.lang.Shutdown.runHooks(Shutdown.java:123)
>  

[jira] [Commented] (ATLAS-3488) Update with Filebased auth password with ShaPasswordEncoder with salt.

2020-06-30 Thread yangchengkai (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148549#comment-17148549
 ] 

yangchengkai commented on ATLAS-3488:
-

[~nixon] 
Thanks for apply.
Our company use CDH 6.2.1 to management hadoop cluster, so the hive version of 
CDH 6.2.1 is hive 2.1.1.
Anyway I did not change the hive version to 2.1.1 in my pom cause many Chinese 
blogs tell me not to do so.
I decide to use Atlas 2.0.0 version, because many blogs use 2.0.0 to solve the 
CDH compatible problems.
Again thanks for you help.

> Update with Filebased auth password with ShaPasswordEncoder with salt.
> --
>
> Key: ATLAS-3488
> URL: https://issues.apache.org/jira/browse/ATLAS-3488
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: ATLAS-3488.07.patch, ATLAS-3488.6.patch
>
>
> Current password is encoded with {{sha256 method for filebased 
> authentication.}}
>  
> {{Update the password with }}{{sha256}}{{ with salt encoding which is more 
> secured.}}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (ATLAS-3488) Update with Filebased auth password with ShaPasswordEncoder with salt.

2020-06-30 Thread Nixon Rodrigues (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148508#comment-17148508
 ] 

Nixon Rodrigues commented on ATLAS-3488:


[~18810276...@163.com]

"org.apache.hadoop.hive.metastore.api.Database.getCatalogName()"ERROR?

 This is issue is due to mismatch in hive version, Atlas expects hive with 
version 3.1.0.

 

from the logs from provided. 
{noformat}
17:47:41.269 [main] DEBUG org.apache.atlas.AtlasBaseClient - Call : GET 
api/atlas/v2/entity/uniqueAttribute/type/
17:47:41.269 [main] DEBUG org.apache.atlas.AtlasBaseClient - Content-type : 
application/json; charset=UTF-8
17:47:41.269 [main] DEBUG org.apache.atlas.AtlasBaseClient - Accept : 
application/json
17:47:41.283 [main] DEBUG org.apache.atlas.AtlasBaseClient - HTTP Status : 404
{noformat}
 its giving 404 since database do not exist in Atlas and authentication has 
passed. if there was authentication issue then it will give 401.

  

> Update with Filebased auth password with ShaPasswordEncoder with salt.
> --
>
> Key: ATLAS-3488
> URL: https://issues.apache.org/jira/browse/ATLAS-3488
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: ATLAS-3488.07.patch, ATLAS-3488.6.patch
>
>
> Current password is encoded with {{sha256 method for filebased 
> authentication.}}
>  
> {{Update the password with }}{{sha256}}{{ with salt encoding which is more 
> secured.}}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-06-30 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/
---

(Updated June 30, 2020, 10:04 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3782
https://issues.apache.org/jira/browse/ATLAS-3782


Repository: atlas


Description (updated)
---

**Issue:**
The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
implemented in SearchProcessors. It would allow a search of entities that do 
not contain in given string in a specified attribute (eg exclude entities from 
search that contain 'temp' in the qualified name). 

**WorkAround:**
JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
inmemory
BasicSearch generates query via three modes
1. Index query -> NOT_CONTAINS will not be supported
2. InMemeory Predicates -> NOT_CONTAINS will be supported, Already handled #123 
SearchProcessor
3. Graph query -> NOT_CONTAINS will not be supported

As in index and graph query wiil not support not_Contains operator, We need to 
apply filter(inMemoryPredicate) after either index/graph query.

To support above, I have modified ClassificationSearchProcessor
+ For both cases index as well as graph, added typeNamePredicate and 
attributePredicate
+ Added these predicate after query
- Removed gremlinQuery block

**Operator Value:**
It can be either "not_contains" or "NOT_CONTAINS"

**Note:**
As part of this jira, not_contains is also added in quick search.


Diffs
-

  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasSolrQueryBuilder.java
 6c06a3cbe 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 9c72cd4a2 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
c9a605355 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
  
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 9b16e919d 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 b7ce97845 


Diff: https://reviews.apache.org/r/72567/diff/4/


Testing
---

Added testcases
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1983


Thanks,

Pinal Shah



[jira] [Commented] (ATLAS-3488) Update with Filebased auth password with ShaPasswordEncoder with salt.

2020-06-30 Thread yangchengkai (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148491#comment-17148491
 ] 

yangchengkai commented on ATLAS-3488:
-

[~nixon] 
PS:
Is this the "Invalid salt version" Error caused my 
"org.apache.hadoop.hive.metastore.api.Database.getCatalogName()"ERROR?
I google and baidu around,and did not find the problems like this.

> Update with Filebased auth password with ShaPasswordEncoder with salt.
> --
>
> Key: ATLAS-3488
> URL: https://issues.apache.org/jira/browse/ATLAS-3488
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: ATLAS-3488.07.patch, ATLAS-3488.6.patch
>
>
> Current password is encoded with {{sha256 method for filebased 
> authentication.}}
>  
> {{Update the password with }}{{sha256}}{{ with salt encoding which is more 
> secured.}}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (ATLAS-3488) Update with Filebased auth password with ShaPasswordEncoder with salt.

2020-06-30 Thread yangchengkai (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148487#comment-17148487
 ] 

yangchengkai commented on ATLAS-3488:
-

[~nixon],hi:

I can run quick-start.py successfully, this is my output:
root@master:/usr/local/src/atlas/apache-atlas-2.1.0# bin/quick_start.py 
Enter username for atlas :- admin
Enter password for atlas :- 

Creating sample types: 
Created type [DB]
Created type [Table]
Created type [StorageDesc]
Created type [Column]
Created type [LoadProcess]
Created type [LoadProcessExecution]
Created type [View]
Created type [JdbcAccess]
Created type [ETL]
Created type [Metric]
Created type [PII]
Created type [Fact]
Created type [Dimension]
Created type [Log Data]
Created type [Table_DB]
Created type [View_DB]
Created type [View_Tables]
Created type [Table_Columns]
Created type [Table_StorageDesc]

Creating sample entities: 
Created entity of type [DB], guid: 0809e0f7-b772-4521-bc5e-46192e3c0ef1
Created entity of type [DB], guid: 7a4ba630-2c18-49d9-ba65-d127a2b12a28
Created entity of type [DB], guid: 5effe5a7-7b4c-439a-9b3a-c523bebe2dfc
Created entity of type [Table], guid: 33578c65-ad00-4081-9bab-4fad7dfdc379



but when I run the import_hive.sh,it still output the same error on my console:


17:47:40.926 [main] DEBUG org.apache.hadoop.hive.ql.exec.Utilities - Hive Conf 
not found or Session not initiated, use thread based class loader instead
17:47:40.926 [main] DEBUG org.apache.hadoop.hive.ql.exec.Utilities - Hive Conf 
not found or Session not initiated, use thread based class loader instead
17:47:40.926 [main] DEBUG org.apache.hadoop.hive.ql.exec.Utilities - Hive Conf 
not found or Session not initiated, use thread based class loader instead
17:47:40.926 [main] DEBUG org.apache.hadoop.hive.ql.exec.Utilities - Hive Conf 
not found or Session not initiated, use thread based class loader instead
17:47:40.927 [main] DEBUG org.apache.hadoop.hive.ql.exec.Utilities - Hive Conf 
not found or Session not initiated, use thread based class loader instead
17:47:40.927 [main] DEBUG org.apache.hadoop.hive.ql.exec.Utilities - Hive Conf 
not found or Session not initiated, use thread based class loader instead
17:47:40.927 [main] DEBUG org.apache.hadoop.hive.ql.exec.Utilities - Hive Conf 
not found or Session not initiated, use thread based class loader instead
17:47:41.047 [main] DEBUG org.apache.hadoop.security.SecurityUtil - Setting 
hadoop.security.token.service.use_ip to true
17:47:41.058 [main] DEBUG org.apache.hadoop.security.Groups -  Creating new 
Groups object
17:47:41.063 [main] DEBUG org.apache.hadoop.security.Groups - Group mapping 
impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping; 
cacheTimeout=30; warningDeltaMs=5000
17:47:41.066 [main] DEBUG org.apache.hadoop.security.UserGroupInformation - 
hadoop login
17:47:41.066 [main] DEBUG org.apache.hadoop.security.UserGroupInformation - 
hadoop login commit
17:47:41.067 [main] DEBUG org.apache.hadoop.security.UserGroupInformation - 
using local user:UnixPrincipal: root
17:47:41.067 [main] DEBUG org.apache.hadoop.security.UserGroupInformation - 
Using user: "UnixPrincipal: root" with name root
17:47:41.067 [main] DEBUG org.apache.hadoop.security.UserGroupInformation - 
User entry: "root"
17:47:41.068 [main] DEBUG org.apache.hadoop.security.UserGroupInformation - UGI 
loginUser:root (auth:SIMPLE)
17:47:41.081 [main] INFO  hive.metastore - HMS client filtering is enabled.
17:47:41.081 [main] INFO  hive.metastore - Trying to connect to metastore with 
URI thrift://master:9083
17:47:41.094 [main] INFO  hive.metastore - Opened a connection to metastore, 
current connections: 1
17:47:41.097 [main] DEBUG org.apache.hadoop.security.Groups - GroupCacheLoader 
- load.
17:47:41.114 [main] INFO  hive.metastore - Connected to metastore.
17:47:41.255 [main] INFO  org.apache.atlas.hive.bridge.HiveMetaStoreBridge - 
Importing Hive metadata
17:47:41.260 [main] INFO  org.apache.atlas.hive.bridge.HiveMetaStoreBridge - 
Found 2 databases
17:47:41.264 [main] DEBUG org.apache.atlas.hive.bridge.HiveMetaStoreBridge - 
Searching Atlas for database default
17:47:41.268 [main] DEBUG org.apache.atlas.AtlasBaseClient - 
--
17:47:41.269 [main] DEBUG org.apache.atlas.AtlasBaseClient - Call : GET 
api/atlas/v2/entity/uniqueAttribute/type/
17:47:41.269 [main] DEBUG org.apache.atlas.AtlasBaseClient - Content-type : 
application/json; charset=UTF-8 
17:47:41.269 [main] DEBUG org.apache.atlas.AtlasBaseClient - Accept   : 
application/json 
17:47:41.283 [main] DEBUG org.apache.atlas.AtlasBaseClient - HTTP Status  : 404
Exception in thread "main" java.lang.NoSuchMethodError: 
org.apache.hadoop.hive.metastore.api.Database.getCatalogName()Ljava/lang/String;
at 
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getDatabaseName(HiveMetaStoreBridge.java:577)
at 

Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-06-30 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/
---

(Updated June 30, 2020, 9:54 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

Rebase patch and added not_contains in quicksearch aggregation metrics


Bugs: ATLAS-3782
https://issues.apache.org/jira/browse/ATLAS-3782


Repository: atlas


Description
---

The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
implemented in SearchProcessors. It would allow a search of entities that do 
not contain in given string in a specified attribute (eg exclude entities from 
search that contain 'temp' in the qualified name). 

JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
inmemory
BasicSearch can generates query via three modes
1. Index query -> NOT_CONTAINS will not be supported
2. InMemeory Predicated -> NOT_CONTAINS will be supported, Already handled #123 
SearchProcessor
3. Graph query -> NOT_CONTAINS will not be supported

We need to apply filter(inMemoryPredicate) after either index/graph query.

To support above, i have modified ClassificationSearchProcessor
+ For both cases index as well as graph, added typeNamePredicate and 
attributePredicate
+ Added these predicate after query
- Removed gremlinQuery block


Diffs (updated)
-

  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasSolrQueryBuilder.java
 6c06a3cbe 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 9c72cd4a2 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
c9a605355 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
  
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 9b16e919d 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 b7ce97845 


Diff: https://reviews.apache.org/r/72567/diff/4/

Changes: https://reviews.apache.org/r/72567/diff/3-4/


Testing
---

Added testcases
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1983


Thanks,

Pinal Shah



[jira] [Commented] (ATLAS-3488) Update with Filebased auth password with ShaPasswordEncoder with salt.

2020-06-30 Thread Nixon Rodrigues (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148473#comment-17148473
 ] 

Nixon Rodrigues commented on ATLAS-3488:


[~18810276...@163.com],

I verified quick-start.py which is using same authentication route and its 
working correctly with admin/admin.

can you verify if you are using same users-credentials.properties file provided 
in 2.1.0-rc1  bits.

admin=ADMIN::a4a88c0872bf652bb9ed803ece5fd6e82354838a9bf59ab4babb1dab322154e1

> Update with Filebased auth password with ShaPasswordEncoder with salt.
> --
>
> Key: ATLAS-3488
> URL: https://issues.apache.org/jira/browse/ATLAS-3488
> Project: Atlas
>  Issue Type: Improvement
>Reporter: Nixon Rodrigues
>Assignee: Nixon Rodrigues
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: ATLAS-3488.07.patch, ATLAS-3488.6.patch
>
>
> Current password is encoded with {{sha256 method for filebased 
> authentication.}}
>  
> {{Update the password with }}{{sha256}}{{ with salt encoding which is more 
> secured.}}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3836) Add Apache Ozone support in hive hook

2020-06-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3836:

Description: 
Apache Ozone is the new object store for Hadoop - 
[https://hadoop.apache.org/ozone/]

Apache Atlas needs to add entity types to support creation of Ozone entities. 
Hive hook should also be updated to create lineage between ozone entities and 
hive tables (for EXTERNAL TABLE)

+*Approach :*+
 # Refactored BaseHiveEvent.getPathEntity() -> moved to 
AtlasPathExtractorUtil.java
 # Created PathExtractorContext.java to wrap most arguments.
 # AtlasPathExtractorUtil.getPathEntity() -> accept Path, PathExtractorContext 
-> return AtlasEntityWithExtInfo
 # Added specific condition in AtlasPathExtractorUtil.getPathEntity() to handle 
Ozone path
 -> path starts with "ofs://" or "o3fs://"
 # Added UT around AtlasPathExtractorUtil.getPathEntity() -> 
AtlasPathExtractorUtilTest.java

 

+*Examples :*+
{code:java}
-> CREATE EXTERNAL TABLE sales (id int) row format delimited fields terminated 
by ' ' stored as textfile location 
'o3fs://bucket1.volume1.ozone1/sale1/q1/sales';{code}
|| ||Name||Qualified Name||
|ozone_key|/sale1/q1/sales|o3fs://bucket1.volume1.ozone1/sale1/q1/sales@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 
{code:java}
-> create EXTERNAL table stocks (id int) row format delimited fields terminated 
by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket1.volume1.ozone1/stocks.txt' into table 
stocks;
{code}
|| ||Name||Qualified Name||
|ozone_key|/stocks.txt|o3fs://bucket1.volume1.ozone1/stocks.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 
{code:java}
-> create EXTERNAL table stocks_q1 (id int) row format delimited fields 
terminated by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket15.volume15.ozone1/quarter1/stocks_q1.txt' 
into table stocks_q1;{code}
|| ||Name||Qualified Name||
|ozone_key|/quarter1/stocks_q1.txt|o3fs://bucket1.volume1.ozone1/quarter1/stocks_q1.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 

  was:
Apache Ozone is the new object store for Hadoop - 
[https://hadoop.apache.org/ozone/]

Apache Atlas needs to add entity types to support creation of Ozone entities. 
Hive hook should also be updated to create lineage between ozone entities and 
hive tables (for EXTERNAL TABLE)

+*Approach :*+
 # Refactored BaseHiveEvent.getPathEntity() -> moved to 
AtlasPathExtractorUtil.java
 # Created PathExtractorContext.java to wrap most arguments.
 # AtlasPathExtractorUtil.getPathEntity() -> accept Path, PathExtractorContext 
-> return AtlasEntityWithExtInfo
 # Added specific condition in AtlasPathExtractorUtil.getPathEntity() to handle 
Ozone path
 -> path starts with "ofs://" or "o3fs://"
 # Added UT around AtlasPathExtractorUtil.getPathEntity() -> 
AtlasPathExtractorUtilTest.java

 

+*Examples :*+
{code:java}
-> CREATE EXTERNAL TABLE sales (id int) row format delimited fields terminated 
by ' ' stored as textfile location 
'o3fs://bucket1.volume1.ozone1/sale1/q1/sales';{code}
|| ||Name||Qualified Name||
|ozone_key|/sale1/q1/sales|o3fs://bucket1.volume1.ozone1/sale1/q1/sales@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 
{code:java}
-> create EXTERNAL table stocks (id int) row format delimited fields terminated 
by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket1.volume1.ozone1/stocks.txt' into table 
stocks;
{code}
|| ||Name||Qualified Name||
|ozone_key|/stocks.txt|o3fs://bucket1.volume1.ozone1/stocks.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 
{code:java}
-> create EXTERNAL table stocks_q1 (id int) row format delimited fields 
terminated by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket15.volume15.ozone1/quarter1/stocks_q1.txt' 
into table stocks_q1;{code}
|| ||Name||Qualified Name||
|ozone_key|/quarter1/stocks_q1.txt|o3fs://bucket1.volume1.ozone1/quarter1/stocks_q1.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 


> Add Apache Ozone support in hive hook
> -
>
> Key: ATLAS-3836
> URL: https://issues.apache.org/jira/browse/ATLAS-3836
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Nikhil Bonte
>Priority: Major
>  Labels: hive-hooks, ozone
> Fix For: 2.1.0
>
> Attachments: Hive_table_lineage.png, 
> Hive_table_lineage_load_in_path.png, Ozone_bucket.png, Ozone_key.png, 
> Ozone_volume.png
>
>
> Apache Ozone is the new object store for Hadoop - 
> 

[jira] [Updated] (ATLAS-3836) Add Apache Ozone support in hive hook

2020-06-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3836:

Description: 
Apache Ozone is the new object store for Hadoop - 
[https://hadoop.apache.org/ozone/]

Apache Atlas needs to add entity types to support creation of Ozone entities. 
Hive hook should also be updated to create lineage between ozone entities and 
hive tables (for EXTERNAL TABLE)

+*Approach :*+
 # Refactored BaseHiveEvent.getPathEntity() -> moved to 
AtlasPathExtractorUtil.java
 # Created PathExtractorContext.java to wrap most arguments.
 # AtlasPathExtractorUtil.getPathEntity() -> accept Path, PathExtractorContext 
-> return AtlasEntityWithExtInfo
 # Added specific condition in AtlasPathExtractorUtil.getPathEntity() to handle 
Ozone path
 -> path starts with "ofs://" or "o3fs://"
 # Added UT around AtlasPathExtractorUtil.getPathEntity() -> 
AtlasPathExtractorUtilTest.java

 

+*Examples :*+
{code:java}
-> CREATE EXTERNAL TABLE sales (id int) row format delimited fields terminated 
by ' ' stored as textfile location 
'o3fs://bucket1.volume1.ozone1/sale1/q1/sales';{code}
|| ||Name||Qualified Name||
|ozone_key|/sale1/q1/sales|o3fs://bucket1.volume1.ozone1/sale1/q1/sales@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 
{code:java}
-> create EXTERNAL table stocks (id int) row format delimited fields terminated 
by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket1.volume1.ozone1/stocks.txt' into table 
stocks;
{code}
|| ||Name||Qualified Name||
|ozone_key|/stocks.txt|o3fs://bucket1.volume1.ozone1/stocks.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 
{code:java}
-> create EXTERNAL table stocks_q1 (id int) row format delimited fields 
terminated by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket1.volume1.ozone1/quarter1/stocks_q1.txt' into 
table stocks_q1;{code}
|| ||Name||Qualified Name||
|ozone_key|/quarter1/stocks_q1.txt|o3fs://bucket1.volume1.ozone1/quarter1/stocks_q1.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 

  was:
Apache Ozone is the new object store for Hadoop - 
[https://hadoop.apache.org/ozone/]

Apache Atlas needs to add entity types to support creation of Ozone entities. 
Hive hook should also be updated to create lineage between ozone entities and 
hive tables (for EXTERNAL TABLE)

+*Approach :*+
 # Refactored BaseHiveEvent.getPathEntity() -> moved to 
AtlasPathExtractorUtil.java
 # Created PathExtractorContext.java to wrap most arguments.
 # AtlasPathExtractorUtil.getPathEntity() -> accept Path, PathExtractorContext 
-> return AtlasEntityWithExtInfo
 # Added specific condition in AtlasPathExtractorUtil.getPathEntity() to handle 
Ozone path
 -> path starts with "ofs://" or "o3fs://"
 # Added UT around AtlasPathExtractorUtil.getPathEntity() -> 
AtlasPathExtractorUtilTest.java

 

+*Examples :*+
{code:java}
-> CREATE EXTERNAL TABLE sales (id int) row format delimited fields terminated 
by ' ' stored as textfile location 
'o3fs://bucket1.volume1.ozone1/sale1/q1/sales';{code}
|| ||Name||Qualified Name||
|ozone_key|/sale1/q1/sales|o3fs://bucket1.volume1.ozone1/sale1/q1/sales@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 
{code:java}
-> create EXTERNAL table stocks (id int) row format delimited fields terminated 
by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket1.volume1.ozone1/stocks.txt' into table 
stocks;
{code}
|| ||Name||Qualified Name||
|ozone_key|/stocks.txt|o3fs://bucket1.volume1.ozone1/stocks.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 
{code:java}
-> create EXTERNAL table stocks_q1 (id int) row format delimited fields 
terminated by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket15.volume15.ozone1/quarter1/stocks_q1.txt' 
into table stocks_q1;{code}
|| ||Name||Qualified Name||
|ozone_key|/quarter1/stocks_q1.txt|o3fs://bucket1.volume1.ozone1/quarter1/stocks_q1.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 


> Add Apache Ozone support in hive hook
> -
>
> Key: ATLAS-3836
> URL: https://issues.apache.org/jira/browse/ATLAS-3836
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Nikhil Bonte
>Priority: Major
>  Labels: hive-hooks, ozone
> Fix For: 2.1.0
>
> Attachments: Hive_table_lineage.png, 
> Hive_table_lineage_load_in_path.png, Ozone_bucket.png, Ozone_key.png, 
> Ozone_volume.png
>
>
> Apache Ozone is the new object store for Hadoop - 
> 

[jira] [Updated] (ATLAS-3836) Add Apache Ozone support in hive hook

2020-06-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3836:

Description: 
Apache Ozone is the new object store for Hadoop - 
[https://hadoop.apache.org/ozone/]

Apache Atlas needs to add entity types to support creation of Ozone entities. 
Hive hook should also be updated to create lineage between ozone entities and 
hive tables (for EXTERNAL TABLE)

+*Approach :*+
 # Refactored BaseHiveEvent.getPathEntity() -> moved to 
AtlasPathExtractorUtil.java
 # Created PathExtractorContext.java to wrap most arguments.
 # AtlasPathExtractorUtil.getPathEntity() -> accept Path, PathExtractorContext 
-> return AtlasEntityWithExtInfo
 # Added specific condition in AtlasPathExtractorUtil.getPathEntity() to handle 
Ozone path
 -> path starts with "ofs://" or "o3fs://"
 # Added UT around AtlasPathExtractorUtil.getPathEntity() -> 
AtlasPathExtractorUtilTest.java

 

+*Examples :*+
{code:java}
-> CREATE EXTERNAL TABLE sales (id int) row format delimited fields terminated 
by ' ' stored as textfile location 
'o3fs://bucket1.volume1.ozone1/sale1/q1/sales';{code}
|| ||Name||Qualified Name||
|ozone_key|/sale1/q1/sales|o3fs://bucket1.volume1.ozone1/sale1/q1/sales@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 
{code:java}
-> create EXTERNAL table stocks (id int) row format delimited fields terminated 
by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket1.volume1.ozone1/stocks.txt' into table 
stocks;
{code}
|| ||Name||Qualified Name||
|ozone_key|/stocks.txt|o3fs://bucket1.volume1.ozone1/stocks.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 
{code:java}
-> create EXTERNAL table stocks_q1 (id int) row format delimited fields 
terminated by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket15.volume15.ozone1/quarter1/stocks_q1.txt' 
into table stocks_q1;{code}
|| ||Name||Qualified Name||
|ozone_key|/quarter1/stocks_q1.txt|o3fs://bucket1.volume1.ozone1/quarter1/stocks_q1.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 

  was:
Apache Ozone is the new object store for Hadoop - 
[https://hadoop.apache.org/ozone/]

Apache Atlas needs to add entity types to support creation of Ozone entities. 
Hive hook should also be updated to create lineage between ozone entities and 
hive tables (for EXTERNAL TABLE)

+*Approach :*+
 # Refactored BaseHiveEvent.getPathEntity() -> moved to 
AtlasPathExtractorUtil.java
 # Created PathExtractorContext.java to wrap most arguments.
 # AtlasPathExtractorUtil.getPathEntity() -> accept Path, PathExtractorContext 
-> return AtlasEntityWithExtInfo
 # Added specific condition in AtlasPathExtractorUtil.getPathEntity() to handle 
Ozone path
 -> path starts with "ofs://" or "o3fs://"
 # Added UT around AtlasPathExtractorUtil.getPathEntity() -> 
AtlasPathExtractorUtilTest.java

 

+*Examples :*+
{code:java}
-> CREATE EXTERNAL TABLE sales (id int) row format delimited fields terminated 
by ' ' stored as textfile location 
'o3fs://bucket1.volume1.ozone1/sale1/q1/sales';{code}
|| ||Name||Qualified Name||
|ozone_key|/sale1/q1/sales2|o3fs://bucket1.volume1.ozone1/sale1/q1/sales2@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 
{code:java}
-> create EXTERNAL table stocks (id int) row format delimited fields terminated 
by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket1.volume1.ozone1/stocks.txt' into table 
stocks;
{code}
|| ||Name||Qualified Name||
|ozone_key|/stocks.txt|o3fs://bucket1.volume1.ozone1/stocks.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 
{code:java}
-> create EXTERNAL table stocks_q1 (id int) row format delimited fields 
terminated by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket15.volume15.ozone1/quarter1/stocks_q1.txt' 
into table stocks_q1;{code}
|| ||Name||Qualified Name||
|ozone_key|/quarter1/stocks_q1.txt|o3fs://bucket1.volume1.ozone1/quarter1/stocks_q1.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 


> Add Apache Ozone support in hive hook
> -
>
> Key: ATLAS-3836
> URL: https://issues.apache.org/jira/browse/ATLAS-3836
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Nikhil Bonte
>Priority: Major
>  Labels: hive-hooks, ozone
> Fix For: 2.1.0
>
> Attachments: Hive_table_lineage.png, 
> Hive_table_lineage_load_in_path.png, Ozone_bucket.png, Ozone_key.png, 
> Ozone_volume.png
>
>
> Apache Ozone is the new object store for Hadoop - 
> 

[jira] [Updated] (ATLAS-3836) Add Apache Ozone support in hive hook

2020-06-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3836:

Description: 
Apache Ozone is the new object store for Hadoop - 
[https://hadoop.apache.org/ozone/]

Apache Atlas needs to add entity types to support creation of Ozone entities. 
Hive hook should also be updated to create lineage between ozone entities and 
hive tables (for EXTERNAL TABLE)

+*Approach :*+
 # Refactored BaseHiveEvent.getPathEntity() -> moved to 
AtlasPathExtractorUtil.java
 # Created PathExtractorContext.java to wrap most arguments.
 # AtlasPathExtractorUtil.getPathEntity() -> accept Path, PathExtractorContext 
-> return AtlasEntityWithExtInfo
 # Added specific condition in AtlasPathExtractorUtil.getPathEntity() to handle 
Ozone path
 -> path starts with "ofs://" or "o3fs://"
 # Added UT around AtlasPathExtractorUtil.getPathEntity() -> 
AtlasPathExtractorUtilTest.java

 

+*Examples :*+
{code:java}
-> CREATE EXTERNAL TABLE sales (id int) row format delimited fields terminated 
by ' ' stored as textfile location 
'o3fs://bucket1.volume1.ozone1/sale1/q1/sales';{code}
 

 
|| ||Name||Qualified Name||
|ozone_key|/sale1/q1/sales2|o3fs://bucket1.volume1.ozone1/sale1/q1/sales2@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 
{code:java}
-> create EXTERNAL table stocks (id int) row format delimited fields terminated 
by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket1.volume1.ozone1/stocks.txt' into table 
stocks;
{code}
 
|| ||Name||Qualified Name||
|ozone_key|/stocks.txt|o3fs://bucket1.volume1.ozone1/stocks.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 
{code:java}
-> create EXTERNAL table stocks_q1 (id int) row format delimited fields 
terminated by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket15.volume15.ozone1/quarter1/stocks_q1.txt' 
into table stocks_q1;{code}
|| ||Name||Qualified Name||
|ozone_key|/quarter1/stocks_q1.txt|o3fs://bucket1.volume1.ozone1/quarter1/stocks_q1.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 

  was:
Apache Ozone is the new object store for Hadoop - 
[https://hadoop.apache.org/ozone/]

Apache Atlas needs to add entity types to support creation of Ozone entities. 
Hive hook should also be updated to create lineage between ozone entities and 
hive tables (for EXTERNAL TABLE)


+*Approach :*+
 # Refactored BaseHiveEvent.getPathEntity() -> moved to 
AtlasPathExtractorUtil.java
 # Created PathExtractorContext.java to wrap most arguments.
 # AtlasPathExtractorUtil.getPathEntity() -> accept Path, PathExtractorContext 
-> return AtlasEntityWithExtInfo
 # Added specific condition in AtlasPathExtractorUtil.getPathEntity() to handle 
Ozone path 
 -> path starts with "ofs://" or "o3fs://"
 # Added UT around AtlasPathExtractorUtil.getPathEntity() -> 
AtlasPathExtractorUtilTest.java

 

 

 


> Add Apache Ozone support in hive hook
> -
>
> Key: ATLAS-3836
> URL: https://issues.apache.org/jira/browse/ATLAS-3836
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Nikhil Bonte
>Priority: Major
>  Labels: hive-hooks, ozone
> Fix For: 2.1.0
>
> Attachments: Hive_table_lineage.png, 
> Hive_table_lineage_load_in_path.png, Ozone_bucket.png, Ozone_key.png, 
> Ozone_volume.png
>
>
> Apache Ozone is the new object store for Hadoop - 
> [https://hadoop.apache.org/ozone/]
> Apache Atlas needs to add entity types to support creation of Ozone entities. 
> Hive hook should also be updated to create lineage between ozone entities and 
> hive tables (for EXTERNAL TABLE)
> +*Approach :*+
>  # Refactored BaseHiveEvent.getPathEntity() -> moved to 
> AtlasPathExtractorUtil.java
>  # Created PathExtractorContext.java to wrap most arguments.
>  # AtlasPathExtractorUtil.getPathEntity() -> accept Path, 
> PathExtractorContext -> return AtlasEntityWithExtInfo
>  # Added specific condition in AtlasPathExtractorUtil.getPathEntity() to 
> handle Ozone path
>  -> path starts with "ofs://" or "o3fs://"
>  # Added UT around AtlasPathExtractorUtil.getPathEntity() -> 
> AtlasPathExtractorUtilTest.java
>  
> +*Examples :*+
> {code:java}
> -> CREATE EXTERNAL TABLE sales (id int) row format delimited fields 
> terminated by ' ' stored as textfile location 
> 'o3fs://bucket1.volume1.ozone1/sale1/q1/sales';{code}
>  
>  
> || ||Name||Qualified Name||
> |ozone_key|/sale1/q1/sales2|o3fs://bucket1.volume1.ozone1/sale1/q1/sales2@cm|
> |ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
> |ozone_volume|volume1|o3fs://volume1@cm|
>  
>  
> {code:java}
> -> create EXTERNAL table stocks (id int) row 

[jira] [Updated] (ATLAS-3836) Add Apache Ozone support in hive hook

2020-06-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3836:

Description: 
Apache Ozone is the new object store for Hadoop - 
[https://hadoop.apache.org/ozone/]

Apache Atlas needs to add entity types to support creation of Ozone entities. 
Hive hook should also be updated to create lineage between ozone entities and 
hive tables (for EXTERNAL TABLE)

+*Approach :*+
 # Refactored BaseHiveEvent.getPathEntity() -> moved to 
AtlasPathExtractorUtil.java
 # Created PathExtractorContext.java to wrap most arguments.
 # AtlasPathExtractorUtil.getPathEntity() -> accept Path, PathExtractorContext 
-> return AtlasEntityWithExtInfo
 # Added specific condition in AtlasPathExtractorUtil.getPathEntity() to handle 
Ozone path
 -> path starts with "ofs://" or "o3fs://"
 # Added UT around AtlasPathExtractorUtil.getPathEntity() -> 
AtlasPathExtractorUtilTest.java

 

+*Examples :*+
{code:java}
-> CREATE EXTERNAL TABLE sales (id int) row format delimited fields terminated 
by ' ' stored as textfile location 
'o3fs://bucket1.volume1.ozone1/sale1/q1/sales';{code}
|| ||Name||Qualified Name||
|ozone_key|/sale1/q1/sales2|o3fs://bucket1.volume1.ozone1/sale1/q1/sales2@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 
{code:java}
-> create EXTERNAL table stocks (id int) row format delimited fields terminated 
by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket1.volume1.ozone1/stocks.txt' into table 
stocks;
{code}
|| ||Name||Qualified Name||
|ozone_key|/stocks.txt|o3fs://bucket1.volume1.ozone1/stocks.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 
{code:java}
-> create EXTERNAL table stocks_q1 (id int) row format delimited fields 
terminated by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket15.volume15.ozone1/quarter1/stocks_q1.txt' 
into table stocks_q1;{code}
|| ||Name||Qualified Name||
|ozone_key|/quarter1/stocks_q1.txt|o3fs://bucket1.volume1.ozone1/quarter1/stocks_q1.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 

  was:
Apache Ozone is the new object store for Hadoop - 
[https://hadoop.apache.org/ozone/]

Apache Atlas needs to add entity types to support creation of Ozone entities. 
Hive hook should also be updated to create lineage between ozone entities and 
hive tables (for EXTERNAL TABLE)

+*Approach :*+
 # Refactored BaseHiveEvent.getPathEntity() -> moved to 
AtlasPathExtractorUtil.java
 # Created PathExtractorContext.java to wrap most arguments.
 # AtlasPathExtractorUtil.getPathEntity() -> accept Path, PathExtractorContext 
-> return AtlasEntityWithExtInfo
 # Added specific condition in AtlasPathExtractorUtil.getPathEntity() to handle 
Ozone path
 -> path starts with "ofs://" or "o3fs://"
 # Added UT around AtlasPathExtractorUtil.getPathEntity() -> 
AtlasPathExtractorUtilTest.java

 

+*Examples :*+
{code:java}
-> CREATE EXTERNAL TABLE sales (id int) row format delimited fields terminated 
by ' ' stored as textfile location 
'o3fs://bucket1.volume1.ozone1/sale1/q1/sales';{code}
 

 
|| ||Name||Qualified Name||
|ozone_key|/sale1/q1/sales2|o3fs://bucket1.volume1.ozone1/sale1/q1/sales2@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 
{code:java}
-> create EXTERNAL table stocks (id int) row format delimited fields terminated 
by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket1.volume1.ozone1/stocks.txt' into table 
stocks;
{code}
 
|| ||Name||Qualified Name||
|ozone_key|/stocks.txt|o3fs://bucket1.volume1.ozone1/stocks.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 
{code:java}
-> create EXTERNAL table stocks_q1 (id int) row format delimited fields 
terminated by ' ' stored as textfile;
-> load data inpath 'o3fs://bucket15.volume15.ozone1/quarter1/stocks_q1.txt' 
into table stocks_q1;{code}
|| ||Name||Qualified Name||
|ozone_key|/quarter1/stocks_q1.txt|o3fs://bucket1.volume1.ozone1/quarter1/stocks_q1.txt@cm|
|ozone_bucket|bucket1|o3fs://volume1.bucket1@cm|
|ozone_volume|volume1|o3fs://volume1@cm|

 

 


> Add Apache Ozone support in hive hook
> -
>
> Key: ATLAS-3836
> URL: https://issues.apache.org/jira/browse/ATLAS-3836
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Nikhil Bonte
>Priority: Major
>  Labels: hive-hooks, ozone
> Fix For: 2.1.0
>
> Attachments: Hive_table_lineage.png, 
> Hive_table_lineage_load_in_path.png, Ozone_bucket.png, Ozone_key.png, 
> Ozone_volume.png
>
>
> Apache Ozone is the new object store for Hadoop - 
> 

[jira] [Created] (ATLAS-3870) UI: hive_table entity table not updating queryparams based on filter selected

2020-06-30 Thread Keval Bhatt (Jira)
Keval Bhatt created ATLAS-3870:
--

 Summary: UI: hive_table entity table not updating queryparams 
based on filter selected
 Key: ATLAS-3870
 URL: https://issues.apache.org/jira/browse/ATLAS-3870
 Project: Atlas
  Issue Type: Improvement
Affects Versions: 2.0.0
Reporter: Keval Bhatt
Assignee: Keval Bhatt
 Fix For: 2.1.0, 3.0.0
 Attachments: Screenshot 2020-06-30 at 2.33.11 PM.png, Screenshot 
2020-06-30 at 2.34.06 PM.png

* Go to hive_db entity detail page
 * Click on Table tab
 * toggle "Show historical entities" checkbox and check the network for the API

API always send the same value for
{code:java}
excludeDeletedEntities: true
{code}
!Screenshot 2020-06-30 at 2.33.11 PM.png|width=374,height=219!

!Screenshot 2020-06-30 at 2.34.06 PM.png|width=397,height=142!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3836) Add Apache Ozone support in hive hook

2020-06-30 Thread Nikhil Bonte (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikhil Bonte updated ATLAS-3836:

Description: 
Apache Ozone is the new object store for Hadoop - 
[https://hadoop.apache.org/ozone/]

Apache Atlas needs to add entity types to support creation of Ozone entities. 
Hive hook should also be updated to create lineage between ozone entities and 
hive tables (for EXTERNAL TABLE)


+*Approach :*+
 # Refactored BaseHiveEvent.getPathEntity() -> moved to 
AtlasPathExtractorUtil.java
 # Created PathExtractorContext.java to wrap most arguments.
 # AtlasPathExtractorUtil.getPathEntity() -> accept Path, PathExtractorContext 
-> return AtlasEntityWithExtInfo
 # Added specific condition in AtlasPathExtractorUtil.getPathEntity() to handle 
Ozone path 
 -> path starts with "ofs://" or "o3fs://"
 # Added UT around AtlasPathExtractorUtil.getPathEntity() -> 
AtlasPathExtractorUtilTest.java

 

 

 

  was:
Apache Ozone is the new object store for Hadoop - 
[https://hadoop.apache.org/ozone/]

Apache Atlas needs to add entity types to support creation of Ozone entities. 
Hive hook should also be updated to create lineage between ozone entities and 
hive tables (for EXTERNAL TABLE)


> Add Apache Ozone support in hive hook
> -
>
> Key: ATLAS-3836
> URL: https://issues.apache.org/jira/browse/ATLAS-3836
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Nikhil Bonte
>Priority: Major
>  Labels: hive-hooks, ozone
> Fix For: 2.1.0
>
> Attachments: Hive_table_lineage.png, 
> Hive_table_lineage_load_in_path.png, Ozone_bucket.png, Ozone_key.png, 
> Ozone_volume.png
>
>
> Apache Ozone is the new object store for Hadoop - 
> [https://hadoop.apache.org/ozone/]
> Apache Atlas needs to add entity types to support creation of Ozone entities. 
> Hive hook should also be updated to create lineage between ozone entities and 
> hive tables (for EXTERNAL TABLE)
> +*Approach :*+
>  # Refactored BaseHiveEvent.getPathEntity() -> moved to 
> AtlasPathExtractorUtil.java
>  # Created PathExtractorContext.java to wrap most arguments.
>  # AtlasPathExtractorUtil.getPathEntity() -> accept Path, 
> PathExtractorContext -> return AtlasEntityWithExtInfo
>  # Added specific condition in AtlasPathExtractorUtil.getPathEntity() to 
> handle Ozone path 
>  -> path starts with "ofs://" or "o3fs://"
>  # Added UT around AtlasPathExtractorUtil.getPathEntity() -> 
> AtlasPathExtractorUtilTest.java
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Comment Edited] (ATLAS-3488) Update with Filebased auth password with ShaPasswordEncoder with salt.

2020-06-30 Thread yangchengkai (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148414#comment-17148414
 ] 

yangchengkai edited comment on ATLAS-3488 at 6/30/20, 8:15 AM:
---

Hi,I got this error in logs/application.log when I run ./bin/import_hive.sh:

 

2020-06-30 15:50:24,815 DEBUG - [pool-2-thread-2:] ~ checkPasswordBCrypt() 
(UserDao:159)
2020-06-30 15:50:24,816 DEBUG - [pool-2-thread-2:] ~ checkPasswordBCrypt(): 
failed (UserDao:168)
java.lang.IllegalArgumentException: Invalid salt version
 at org.springframework.security.crypto.bcrypt.BCrypt.hashpw(BCrypt.java:556)
 at org.springframework.security.crypto.bcrypt.BCrypt.checkpw(BCrypt.java:659)
 at org.apache.atlas.web.dao.UserDao.checkPasswordBCrypt(UserDao.java:165)
 at org.apache.atlas.web.dao.UserDao.checkEncrypted(UserDao.java:144)
 at 
org.apache.atlas.web.security.AtlasFileAuthenticationProvider.authenticate(AtlasFileAuthenticationProvider.java:65)
 at 
org.apache.atlas.web.security.AtlasAuthenticationProvider.authenticate(AtlasAuthenticationProvider.java:143)
 at 
org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:174)
 at 
org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:199)
 at 
org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:180)
 at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.apache.atlas.web.filters.AtlasKnoxSSOAuthenticationFilter.doFilter(AtlasKnoxSSOAuthenticationFilter.java:142)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66)
 at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
 at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
 at 
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
 at 
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)
 at 
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)
 at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
 at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
 at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
 at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
 at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
 at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
 at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
 at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
 at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
 at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
 at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
 at org.eclipse.jetty.server.Server.handle(Server.java:534)
 at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
 at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
 at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
 at 

[jira] [Comment Edited] (ATLAS-3488) Update with Filebased auth password with ShaPasswordEncoder with salt.

2020-06-30 Thread yangchengkai (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148414#comment-17148414
 ] 

yangchengkai edited comment on ATLAS-3488 at 6/30/20, 8:15 AM:
---

Hi,I got this error in logs/application.log when I run ./bin/import_hive.sh:

 

2020-06-30 15:50:24,815 DEBUG - [pool-2-thread-2:] ~ checkPasswordBCrypt() 
(UserDao:159)
2020-06-30 15:50:24,816 DEBUG - [pool-2-thread-2:] ~ checkPasswordBCrypt(): 
failed (UserDao:168)
java.lang.IllegalArgumentException: Invalid salt version
 at org.springframework.security.crypto.bcrypt.BCrypt.hashpw(BCrypt.java:556)
 at org.springframework.security.crypto.bcrypt.BCrypt.checkpw(BCrypt.java:659)
 at org.apache.atlas.web.dao.UserDao.checkPasswordBCrypt(UserDao.java:165)
 at org.apache.atlas.web.dao.UserDao.checkEncrypted(UserDao.java:144)
 at 
org.apache.atlas.web.security.AtlasFileAuthenticationProvider.authenticate(AtlasFileAuthenticationProvider.java:65)
 at 
org.apache.atlas.web.security.AtlasAuthenticationProvider.authenticate(AtlasAuthenticationProvider.java:143)
 at 
org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:174)
 at 
org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:199)
 at 
org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:180)
 at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.apache.atlas.web.filters.AtlasKnoxSSOAuthenticationFilter.doFilter(AtlasKnoxSSOAuthenticationFilter.java:142)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66)
 at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
 at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
 at 
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
 at 
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)
 at 
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)
 at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
 at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
 at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
 at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
 at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
 at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
 at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
 at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
 at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
 at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
 at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
 at org.eclipse.jetty.server.Server.handle(Server.java:534)
 at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
 at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
 at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
 at 

[jira] [Commented] (ATLAS-3488) Update with Filebased auth password with ShaPasswordEncoder with salt.

2020-06-30 Thread yangchengkai (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148414#comment-17148414
 ] 

yangchengkai commented on ATLAS-3488:
-

Hi,I got this error in logs/application.log when I run ./bin/import_hive.sh:

 

2020-06-30 15:50:24,815 DEBUG - [pool-2-thread-2:] ~ checkPasswordBCrypt() 
(UserDao:159)
2020-06-30 15:50:24,816 DEBUG - [pool-2-thread-2:] ~ checkPasswordBCrypt(): 
failed (UserDao:168)
java.lang.IllegalArgumentException: Invalid salt version
 at org.springframework.security.crypto.bcrypt.BCrypt.hashpw(BCrypt.java:556)
 at org.springframework.security.crypto.bcrypt.BCrypt.checkpw(BCrypt.java:659)
 at org.apache.atlas.web.dao.UserDao.checkPasswordBCrypt(UserDao.java:165)
 at org.apache.atlas.web.dao.UserDao.checkEncrypted(UserDao.java:144)
 at 
org.apache.atlas.web.security.AtlasFileAuthenticationProvider.authenticate(AtlasFileAuthenticationProvider.java:65)
 at 
org.apache.atlas.web.security.AtlasAuthenticationProvider.authenticate(AtlasAuthenticationProvider.java:143)
 at 
org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:174)
 at 
org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:199)
 at 
org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:180)
 at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.apache.atlas.web.filters.AtlasKnoxSSOAuthenticationFilter.doFilter(AtlasKnoxSSOAuthenticationFilter.java:142)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66)
 at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
 at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
 at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
 at 
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
 at 
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
 at 
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)
 at 
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)
 at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
 at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
 at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
 at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
 at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
 at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
 at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
 at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
 at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
 at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
 at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
 at org.eclipse.jetty.server.Server.handle(Server.java:534)
 at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
 at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
 at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
 at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
 at 

[jira] [Resolved] (ATLAS-3836) Add Apache Ozone support in hive hook

2020-06-30 Thread Sarath Subramanian (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian resolved ATLAS-3836.
---
Resolution: Fixed

> Add Apache Ozone support in hive hook
> -
>
> Key: ATLAS-3836
> URL: https://issues.apache.org/jira/browse/ATLAS-3836
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Nikhil Bonte
>Priority: Major
>  Labels: hive-hooks, ozone
> Fix For: 2.1.0
>
> Attachments: Hive_table_lineage.png, 
> Hive_table_lineage_load_in_path.png, Ozone_bucket.png, Ozone_key.png, 
> Ozone_volume.png
>
>
> Apache Ozone is the new object store for Hadoop - 
> [https://hadoop.apache.org/ozone/]
> Apache Atlas needs to add entity types to support creation of Ozone entities. 
> Hive hook should also be updated to create lineage between ozone entities and 
> hive tables (for EXTERNAL TABLE)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (ATLAS-3836) Add Apache Ozone support in hive hook

2020-06-30 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148384#comment-17148384
 ] 

ASF subversion and git services commented on ATLAS-3836:


Commit f4a1e078006985c21883f7cef00e7cf94750d96e in atlas's branch 
refs/heads/branch-2.0 from Nikhil Bonte
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=f4a1e07 ]

ATLAS-3836 Add Apache Ozone support in hive hook

Signed-off-by: Sarath Subramanian 
(cherry picked from commit 8b50ac0c7d8eba8efa0deb97f1fee68d382f11ad)


> Add Apache Ozone support in hive hook
> -
>
> Key: ATLAS-3836
> URL: https://issues.apache.org/jira/browse/ATLAS-3836
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Nikhil Bonte
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: Hive_table_lineage.png, 
> Hive_table_lineage_load_in_path.png, Ozone_bucket.png, Ozone_key.png, 
> Ozone_volume.png
>
>
> Apache Ozone is the new object store for Hadoop - 
> [https://hadoop.apache.org/ozone/]
> Apache Atlas needs to add entity types to support creation of Ozone entities. 
> Hive hook should also be updated to create lineage between ozone entities and 
> hive tables (for EXTERNAL TABLE)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3836) Add Apache Ozone support in hive hook

2020-06-30 Thread Sarath Subramanian (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3836:
--
Labels: hive-hooks ozone  (was: )

> Add Apache Ozone support in hive hook
> -
>
> Key: ATLAS-3836
> URL: https://issues.apache.org/jira/browse/ATLAS-3836
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Nikhil Bonte
>Priority: Major
>  Labels: hive-hooks, ozone
> Fix For: 2.1.0
>
> Attachments: Hive_table_lineage.png, 
> Hive_table_lineage_load_in_path.png, Ozone_bucket.png, Ozone_key.png, 
> Ozone_volume.png
>
>
> Apache Ozone is the new object store for Hadoop - 
> [https://hadoop.apache.org/ozone/]
> Apache Atlas needs to add entity types to support creation of Ozone entities. 
> Hive hook should also be updated to create lineage between ozone entities and 
> hive tables (for EXTERNAL TABLE)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (ATLAS-3836) Add Apache Ozone support in hive hook

2020-06-30 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148383#comment-17148383
 ] 

ASF subversion and git services commented on ATLAS-3836:


Commit 8b50ac0c7d8eba8efa0deb97f1fee68d382f11ad in atlas's branch 
refs/heads/master from Nikhil Bonte
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=8b50ac0 ]

ATLAS-3836 Add Apache Ozone support in hive hook

Signed-off-by: Sarath Subramanian 


> Add Apache Ozone support in hive hook
> -
>
> Key: ATLAS-3836
> URL: https://issues.apache.org/jira/browse/ATLAS-3836
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Nikhil Bonte
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: Hive_table_lineage.png, 
> Hive_table_lineage_load_in_path.png, Ozone_bucket.png, Ozone_key.png, 
> Ozone_volume.png
>
>
> Apache Ozone is the new object store for Hadoop - 
> [https://hadoop.apache.org/ozone/]
> Apache Atlas needs to add entity types to support creation of Ozone entities. 
> Hive hook should also be updated to create lineage between ozone entities and 
> hive tables (for EXTERNAL TABLE)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (ATLAS-3868) [Regression] removing a term-association doesn't remove classifications propagated from the term

2020-06-30 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148353#comment-17148353
 ] 

ASF subversion and git services commented on ATLAS-3868:


Commit 84da9a389068635ddfd5827c70dc6d476a9aea6a in atlas's branch 
refs/heads/branch-2.0 from Sarath Subramanian
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=84da9a3 ]

ATLAS-3868: [Regression] removing a term-association doesn't remove 
classifications propagated from the term

Signed-off-by: Sarath Subramanian 
(cherry picked from commit ce95c6299986eae212f6861ab423f32aa8f7f7d0)


> [Regression] removing a term-association doesn't remove classifications 
> propagated from the term
> 
>
> Key: ATLAS-3868
> URL: https://issues.apache.org/jira/browse/ATLAS-3868
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
>  Labels: glossary, tagpropagation, terms
> Fix For: 2.1.0
>
>
> # Create a business term – *term1*
>  # Associate classification *PII* to *term1*
>  # Associate *term1* to an entity
>  # Entity will now have PII classification propagated from the term
>  # Now remove *term1* association with the entity
>  # Entity still shows *PII* classification, propagated from the term => this 
> is a regression
>  
> This regression is caused by ATLAS-3863



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (ATLAS-3868) [Regression] removing a term-association doesn't remove classifications propagated from the term

2020-06-30 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148351#comment-17148351
 ] 

ASF subversion and git services commented on ATLAS-3868:


Commit ce95c6299986eae212f6861ab423f32aa8f7f7d0 in atlas's branch 
refs/heads/master from Sarath Subramanian
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=ce95c62 ]

ATLAS-3868: [Regression] removing a term-association doesn't remove 
classifications propagated from the term

Signed-off-by: Sarath Subramanian 


> [Regression] removing a term-association doesn't remove classifications 
> propagated from the term
> 
>
> Key: ATLAS-3868
> URL: https://issues.apache.org/jira/browse/ATLAS-3868
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
>  Labels: glossary, tagpropagation, terms
> Fix For: 2.1.0
>
>
> # Create a business term – *term1*
>  # Associate classification *PII* to *term1*
>  # Associate *term1* to an entity
>  # Entity will now have PII classification propagated from the term
>  # Now remove *term1* association with the entity
>  # Entity still shows *PII* classification, propagated from the term => this 
> is a regression
>  
> This regression is caused by ATLAS-3863



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (ATLAS-3869) Upgrade httpclient from 4.5.3 to 4.5.4

2020-06-30 Thread Nixon Rodrigues (Jira)
Nixon Rodrigues created ATLAS-3869:
--

 Summary: Upgrade httpclient from 4.5.3 to 4.5.4
 Key: ATLAS-3869
 URL: https://issues.apache.org/jira/browse/ATLAS-3869
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Reporter: Nixon Rodrigues
Assignee: Nixon Rodrigues


Found below exception where data was not getting added in SOLR from janusgraph.

 
{noformat}
Caused by: java.lang.ClassCastException: [B cannot be cast to java.lang.String
at 
org.apache.http.conn.ssl.DefaultHostnameVerifier.getSubjectAltNames(DefaultHostnameVerifier.java:309)
at 
org.apache.http.conn.ssl.DefaultHostnameVerifier.verify(DefaultHostnameVerifier.java:112)
at 
org.apache.http.conn.ssl.DefaultHostnameVerifier.verify(DefaultHostnameVerifier.java:99)
at 
org.apache.http.conn.ssl.SSLConnectionSocketFactory.verifyHostname(SSLConnectionSocketFactory.java:463)
at 
org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:397)
at 
org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355)
at 
org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
at 
org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359)
at 
org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
at 
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
at 
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at 
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at 
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:526)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
... 7 more
2020-06-29 07:31:25,893 WARN  - [SIGTERM handler:] ~ Unable to close 
transaction standardjanusgraphtx[0x18d7a16a] (StandardJanusGraph:230)
java.lang.IllegalArgumentException: The transaction has already been closed
at 
com.google.common.base.Preconditions.checkArgument(Preconditions.java:142)
at 
org.janusgraph.graphdb.transaction.StandardJanusGraphTx.rollback(StandardJanusGraphTx.java:1401)
at 
org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsTransaction$1.doRollback(JanusGraphBlueprintsTransaction.java:190)
at 
org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.rollback(AbstractTransaction.java:114)
at 
org.apache.tinkerpop.gremlin.structure.Transaction$CLOSE_BEHAVIOR$2.accept(Transaction.java:182)
at 
org.apache.tinkerpop.gremlin.structure.Transaction$CLOSE_BEHAVIOR$2.accept(Transaction.java:179)
at 
org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.close(JanusGraphBlueprintsGraph.java:327)
at 
org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsTransaction$1.doClose(JanusGraphBlueprintsTransaction.java:205)
at 
org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.close(AbstractTransaction.java:139)
at 
org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsTransaction.close(JanusGraphBlueprintsTransaction.java:215)
at 
org.janusgraph.graphdb.database.StandardJanusGraph.closeInternal(StandardJanusGraph.java:226)
at 
org.janusgraph.graphdb.database.StandardJanusGraph.access$500(StandardJanusGraph.java:91)
at 
org.janusgraph.graphdb.database.StandardJanusGraph$ShutdownThread.start(StandardJanusGraph.java:818)
at 
java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:102)
at 
java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:46)
at java.lang.Shutdown.runHooks(Shutdown.java:123)
at java.lang.Shutdown.sequence(Shutdown.java:167)
at java.lang.Shutdown.exit(Shutdown.java:212)
at java.lang.Terminator$1.handle(Terminator.java:52)
at sun.misc.Signal$1.run(Signal.java:212)
at java.lang.Thread.run(Thread.java:745)
{noformat}
its due to HTTPCLIENT-1836 

To fix above issue:- Upgrade httpclient from 4.5.3 to 4.5.4.

 



--
This message was sent by Atlassian Jira

Re: Review Request 72630: ATLAS-3868: [Regression] removing a term-association doesn't remove classifications propagated from the term

2020-06-30 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72630/#review221096
---


Ship it!




Ship It!

- Madhan Neethiraj


On June 30, 2020, 5:52 a.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72630/
> ---
> 
> (Updated June 30, 2020, 5:52 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
> Nikhil Bonte, Nixon Rodrigues, Pinal Shah, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3868
> https://issues.apache.org/jira/browse/ATLAS-3868
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Issue:
> ==
> removing a term-association doesn't remove classifications propagated from 
> the term
> 
> Cause:
> ==
> Regression caused by https://issues.apache.org/jira/browse/ATLAS-3863
> 
> Solution:
> =
> Re-evaluate tag propagation only when relationship (edge) is force deleted or 
> for internal types.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/HardDeleteHandlerV1.java
>  72dd632f4 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1.java
>  59e7cf864 
> 
> 
> Diff: https://reviews.apache.org/r/72630/diff/2/
> 
> 
> Testing
> ---
> 
> Manually validated - term dissassociation removes all propagated 
> classifications
> 
> Precommit:
> ==
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1998/console
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>