Re: Review Request 61274: ATLAS-1944 - Fix for ConcurrentModificationException Exception in HookConsumer (KafkaConsumer) while stopping Atlas server

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61274/#review182029
---


Ship it!




Ship It!

- Madhan Neethiraj


On Aug. 2, 2017, 8:48 a.m., Nixon Rodrigues wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61274/
> ---
> 
> (Updated Aug. 2, 2017, 8:48 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Madhan Neethiraj, and 
> Sarath Subramanian.
> 
> 
> Bugs: ATLAS-1944
> https://issues.apache.org/jira/browse/ATLAS-1944
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Background:- NotificationHookConsumer was throwing 
> ConcurrentModificationException exception while shutting down the consumer 
> thread. The stop method of thread was calling the kafkacosumer.stop method 
> while in run method of HookConsumer which is busy in cosuming the kafka 
> message by polling kafka server. This simaltanous action of stoping and 
> polling causes ConcurrentModificationException.
> 
> Fix:- 
> 
> * Added try finally block in run method which wraps the while loop and when 
> loop ends the finally blocks calls the consumer close method is called which 
> ensures that close is called after consuming is ended. 
> 
> * Replaced the ShutdownThread interface which was delaying the shutdown 
> process of consumer with Runnable interface used earlier.
> 
> 
> Reference : - 
> https://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html
> 
> 
> Diffs
> -
> 
>   notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaConsumer.java 
> d431176 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationConsumer.java
>  0bd75e1 
>   
> notification/src/test/java/org/apache/atlas/notification/AbstractNotificationConsumerTest.java
>  bcee00c 
>   
> webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
>  51276d3 
> 
> 
> Diff: https://reviews.apache.org/r/61274/diff/2/
> 
> 
> Testing
> ---
> 
> Tested the Atlas kafkaconsumer from hive hook and shutdown of atlas server to 
> check if there is issue while shutdown process.
> Unit test and integration test are passing.
> 
> 
> Thanks,
> 
> Nixon Rodrigues
> 
>

Re: Review Request 61389: ATLAS-2017: Import API: Request Parameter Made Optional

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61389/#review182065
---


Ship it!




Ship It!

- Madhan Neethiraj


On Aug. 2, 2017, 10:26 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61389/
> ---
> 
> (Updated Aug. 2, 2017, 10:26 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2017
> https://issues.apache.org/jira/browse/ATLAS-2017
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Implementation**
> REST Import: Added _DefaultValue_. Removed empty check.
> _ImportService.run_: Added overload to not have _AtlasRequest_ argument.
> _ImportServiceTest_: Used the new method.
> Documentation: Updated to reflect the paramter.
> 
> 
> Diffs
> -
> 
>   docs/src/site/twiki/Import-API.twiki 00ac9c4 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java
>  bdeed7d 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
>  b96197b 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ZipFileResourceTestUtils.java
>  3ccbd8e 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
> fe9111b 
> 
> 
> Diff: https://reviews.apache.org/r/61389/diff/1/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> Updated to include new method.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

[ANNOUNCE] New Committer: Mandy Chessell

[Madhan Neethiraj]

Atlas community,

 

Apache Atlas PMC has asked Mandy Chessell to become a committer and we are 
pleased to announce that she has accepted.

 

Welcome Mandy, great to have you on board.

 

On behalf of Apache Atlas PMC,

 

Madhan

 

 

[ANNOUNCE] New Committer: Ashutosh Mestry

[Madhan Neethiraj]

Atlas community,

 

Apache Atlas PMC has asked Ashutosh Mestry to become a committer and we are 
pleased to announce that he has accepted.

 

Welcome Ashutosh, great to have you on board.

 

On behalf of Apache Atlas PMC,

 

Madhan

 

 

[ANNOUNCE] New Committer: David Radley

[Madhan Neethiraj]

Atlas community,

 

Apache Atlas PMC has asked David Radley to become a committer and we are 
pleased to announce that he has accepted.

 

Welcome David, great to have you on board.

 

On behalf of Apache Atlas PMC,

 

Madhan

 

 

Re: Review Request 61589: [ATLAS-2040]: Relationship with many-to-many cardinality gives incorrect relationship attribute value

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61589/#review182661
---




intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java
Lines 30 (patched)
<https://reviews.apache.org/r/61589/#comment258640>

Consider replacing org.jboss.netty.util.internal.StringUtil usage with 
org.apache.commons.lang.StringUtils.



repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java
Line 1352 (original), 1359 (patched)
<https://reviews.apache.org/r/61589/#comment258641>

Shouldn't this also handle AtlasRelationshipEdgeDirection.BOTH?


- Madhan Neethiraj


On Aug. 10, 2017, 11:43 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61589/
> ---
> 
> (Updated Aug. 10, 2017, 11:43 p.m.)
> 
> 
> Review request for atlas, David Radley and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2040
> https://issues.apache.org/jira/browse/ATLAS-2040
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Relationship with many-to-many cardinality gives incorrect relationship 
> attribute value when same attribute name and same type specified at both ends
> When same attribute name and same type is mentioned in both ends of 
> relationship definition. Creating new relationships fails to retireve the 
> right value.
> For e.g. employee-friends type: same attribute name 'friends' in type 
> 'Employee' specified at both ends
> Add more unit tests to test these cases
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 
> 934dffce 
>   intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 277d0fa2 
>   intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 98be2b8d 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 1ec5a72e 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  42713763 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  b8fd70e5 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  667c61bd 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreHardDeleteV1Test.java
>  2c311403 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreSoftDeleteV1Test.java
>  33ef8c0f 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
>  a35647dc 
> 
> 
> Diff: https://reviews.apache.org/r/61589/diff/1/
> 
> 
> Testing
> ---
> 
> * added new  UTs to test many-to-many relationships on same attribute 
> name/same type
> * added new  UTs to test one-to-one relationships on same attribute name/same 
> type
> * mvn clean install -DskipITs=true suceeded without any failures
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 61552: ATLAS-2038 : Unit Test cases for AtlasAuthentication and Authorization Filter

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61552/#review182810
---


Ship it!




Ship It!

- Madhan Neethiraj


On Aug. 11, 2017, 4:25 p.m., Nixon Rodrigues wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61552/
> ---
> 
> (Updated Aug. 11, 2017, 4:25 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Madhan Neethiraj, and 
> Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2038
> https://issues.apache.org/jira/browse/ATLAS-2038
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch enable the Unittest from this testclass and includes unit test 
> cases for testing Authentication and Authorization Filters which can fail if 
> any regression is introduced in this Filters and authentication/authorization 
> process. ( eg bugs ATLAS-2009).
> 
> 
> Diffs
> -
> 
>   
> webapp/src/test/java/org/apache/atlas/web/filters/AtlasAuthenticationSimpleFilterIT.java
>  PRE-CREATION 
>   
> webapp/src/test/java/org/apache/atlas/web/filters/AtlasAuthenticationSimpleFilterTest.java
>  389eefe 
> 
> 
> Diff: https://reviews.apache.org/r/61552/diff/2/
> 
> 
> Testing
> ---
> 
> Excecuted unit and Integration test from mvn clean install
> 
> 
> Thanks,
> 
> Nixon Rodrigues
> 
>

Re: Review Request 61667: ATLAS-2044: In-memory filtering after index query

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61667/#review183093
---


Ship it!




Ship It!

- Madhan Neethiraj


On Aug. 17, 2017, 1:19 a.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61667/
> ---
> 
> (Updated Aug. 17, 2017, 1:19 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2044
> https://issues.apache.org/jira/browse/ATLAS-2044
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> In-memory filtering is needed to weed out any false positive results from 
> index query. False positive matches happen when the search string is present 
> "as is" after a special token which is used by the indexer (solr, 
> elasticsearch) to tokenize the string (during indexing as well as querying)
> 
> eg.
> 
> name = /a/b/c
> 
> search string "a", "b", "c" will match this name every time under equality, 
> startsWith, endsWith check as it's present immediately after the special 
> token.
> 
> The patch adds extra level of filtering on the the result set obtained from 
> the index query.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  74197ca8 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
>  9cd83fb4 
>   
> repository/src/main/java/org/apache/atlas/discovery/FullTextSearchProcessor.java
>  d556bf1a 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> b209ecb4 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/61667/diff/5/
> 
> 
> Testing
> ---
> 
> REST and UI testing no longer show the false positive matches.
> 
> mvn clean package -Pdist (in progress)
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Review Request 61708: ATLAS-2052: updated packaing to exclude junit jar

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61708/
---

Review request for atlas.


Bugs: ATLAS-2052
https://issues.apache.org/jira/browse/ATLAS-2052


Repository: atlas


Description
---

updated packaing to exclude junit jar


Diffs
-

  pom.xml 8b9eee6f 
  webapp/pom.xml 42981e06 


Diff: https://reviews.apache.org/r/61708/diff/1/


Testing
---

Verified that atlas.war doesn't include junit libraries


Thanks,

Madhan Neethiraj

Re: Review Request 60792: ATLAS-1945 : UI : Moving all plugins to fix release version

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60792/#review180294
---


Ship it!




Ship It!

- Madhan Neethiraj


On July 12, 2017, 6:50 a.m., keval bhatt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60792/
> ---
> 
> (Updated July 12, 2017, 6:50 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Madhan Neethiraj, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-1945
> https://issues.apache.org/jira/browse/ATLAS-1945
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> If npm version will be updated (plugin updated) then UI functionality might 
> get a break so it's better to have all plugin version to fix release version.
> 
> 
> Diffs
> -
> 
>   dashboardv2/package.json 59889d3 
> 
> 
> Diff: https://reviews.apache.org/r/60792/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> keval bhatt
> 
>

Re: Review Request 60851: [ATLAS-1949]: Fix coverity scan issues and IT failure due to ATLAS-1907

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60851/#review180463
---


Ship it!




Ship It!

- Madhan Neethiraj


On July 13, 2017, 8:12 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60851/
> ---
> 
> (Updated July 13, 2017, 8:12 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-1949
> https://issues.apache.org/jira/browse/ATLAS-1949
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Fix coverity scan issues and IT failure due to ATLAS-1907
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java fd1b004f 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> c5e32d85 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  68f83709 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  e919d1ce 
> 
> 
> Diff: https://reviews.apache.org/r/60851/diff/1/
> 
> 
> Testing
> ---
> 
> 1. mvn clean package suceeded.
> 
> 2. Failed IT tests now succeeds:
>org.apache.atlas.web.integration.EntityJerseyResourceIT.testCompleteUpdate
>org.apache.atlas.web.integration.EntityJerseyResourceIT.testPartialUpdate
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Testing API...

[Madhan Neethiraj]

“/api/v2/search” is not the correct URI. Can you try  with 
“api/atlas/v2/search”?

Madhan



On 7/17/17, 11:51 AM, "Jean Georges Perrin"  wrote:

Hi,

I installed and run Atlas on my Mac. Everything seems to be working fine in 
the UI, but I cannot get the API to work... I am sure it is really a small 
thing, but...

curl --user admin:admin -v http://localhost:21000/api/v2/search/basic 


returns:

*   Trying ::1...
* TCP_NODELAY set
* Connection failed
* connect to ::1 port 21000 failed: Connection refused
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 21000 (#0)
* Server auth using Basic with user 'admin'
> GET /api/v2/search/basic HTTP/1.1
> Host: localhost:21000
> Authorization: Basic YWRtaW46YWRtaW4=
> User-Agent: curl/7.54.1
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Date: Mon, 17 Jul 2017 18:42:49 GMT
< Set-Cookie: ATLASSESSIONID=14z2v5j9416lcibimeswa5it;Path=/;HttpOnly
< X-Frame-Options: DENY
< Content-Type: text/html; charset=ISO-8859-1
< Cache-Control: must-revalidate,no-cache,no-store
< Content-Length: 303
< Server: Jetty(9.2.12.v20150709)
<



Error 404 Not Found

HTTP ERROR 404
Problem accessing /api/v2/search/basic. Reason:
Not FoundPowered by 
Jetty://



* Connection #0 to host localhost left intact

Thanks!

jg

Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for all base models and use this definitions when creating edges for entities

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/#review180183
---


Ship it!




Ship It!

- Madhan Neethiraj


On July 11, 2017, 7:40 a.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60578/
> ---
> 
> (Updated July 11, 2017, 7:40 a.m.)
> 
> 
> Review request for atlas, David Radley and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-1907
> https://issues.apache.org/jira/browse/ATLAS-1907
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Create relationshipDefs for all out of the box models. Use relationshipDef 
> information when creating edges for composite attributes in entities.
> 
> 
> Diffs
> -
> 
>   addons/models/0010-base_model.json 7f64d85e 
>   addons/models/0030-hive_model.json b359ea82 
>   addons/models/0050-falcon_model.json b7398ef4 
>   addons/models/0060-hbase_model.json 42fe00c4 
>   addons/models/0080-storm_model.json 095936e0 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java e8afed1d 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 
> 0e277b19 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 
> 8d2e7ec5 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasStruct.java 
> 7c8ae2d7 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a 
>   intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 
> 49a94931 
>   intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50 
>   intg/src/main/java/org/apache/atlas/type/AtlasType.java 86072fe8 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 329dd7a4 
>   
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java
>  4530f51b 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
>  7b3f1e6e 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
>  86212338 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  ebf6a201 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  a5b5730f 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60578/diff/6/
> 
> 
> Testing
> ---
> 
> mvn clean package succeeded. Validated hive_table creation and lineage 
> rendering.
> 
> UTs/ITs in progress.
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Draft Board Report for Apache Atlas: July-2017

[Madhan Neethiraj]

Hemanth,

Here is the query used to get resolved JIRA count: 
https://issues.apache.org/jira/issues/?jql=project%20%3D%20ATLAS%20AND%20resolutiondate%20%3E%202017-05-29%20AND%20resolutiondate%20%3C%3D%202017-06-30

Thanks,
Madhan



On 7/10/17, 10:12 AM, "Hemanth Yamijala" <hyamij...@hortonworks.com> wrote:

Hi Madhan,

I see the number of issues Resolved / Closed as 20 rather than 32. My JQL 
is:

project = ATLAS AND status in (Resolved, Closed) AND created >= 2017-05-29 
AND created <= 2017-06-30

Could you please check once? Other than that, this looks good to me. +1

Thanks
hemanth
____
From: Madhan Neethiraj <mad...@apache.org>
Sent: Monday, July 10, 2017 8:21 PM
To: dev@atlas.apache.org
Subject: Draft Board Report for Apache Atlas: July-2017

Atlas team,



Please review the draft board report below and send your feedback/comments, 
if any. The report is due on Wednesday this week, July-12.



Thanks,

Madhan









## Description:

  Apache Atlas is a scalable and extensible set of core foundational 
governance

  services that enables enterprises to effectively and efficiently meet 
their

  compliance requirements within Hadoop and allows integration with the

  complete enterprise data ecosystem



## Issues:

  There are no issues requiring board attention at this time.



## Activity:

  - Apache Atlas graduated as a Top Level Project on 6/21

  - Migrating from Incubator infra to TLP infra (INFRA-14486)

  - Working on core enhancements like relationships as first class type

  - Working on significant improvements to search capabilities

  - Working on new features like Virtual Data Connector, Automated Discovery

  - Jira: +59 (created) -32(resolved) between 5/29 and 6/30

  - Git: 28 commits between 5/29 and 6/30



## Health report:

  - Added 5 new contributors between 5/29 and 6/30

  - A lot of interest in adding new features to Atlas from the community,

like: Virtual Data Connector, Automated Discovery, Business User UIs



## PMC changes:

  - Currently 33 PMC members

  - No new PMC members added since graduation on 6/21



## Committer base changes:

  - Currently 33 committers

  - No new committers added since graduation on 6/21



## Releases:

  No new releases in June 2017



## Mailing list activity:

  - dev@atlas.apache.org:

579 emails sent to list (in the month of June 2017)

  - u...@atlas.apache.org:

  5 emails sent to list (in the month of June 2017)

  - comm...@atlas.apache.org:

113 emails sent to list (in the month of June 2017)





## JIRA activity:

  - 59 JIRA tickets created between 5/29 and 6/30

  - 32 JIRA tickets closed/resolved between 5/29 and 6/30

Re: Assign as contributor

[Madhan Neethiraj]

Chandana,

Done. Welcome to Apache Atlas community.

Madhan

On 7/7/17, 6:18 PM, "Chandana Mirashi"  wrote:

Hi,
I would like to contribute to Atlas. Can someone please add me so that 
I can assign myself JIRA to work on?
 
Regards,
Chandana Mirashi

Re: Review Request 60852: ATLAS-1947: Improvements for ATLAS-1880 and some change to AtlasSearchResult

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60852/#review180602
---


Ship it!




Ship It!

- Madhan Neethiraj


On July 14, 2017, 10:49 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60852/
> ---
> 
> (Updated July 14, 2017, 10:49 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-1947
> https://issues.apache.org/jira/browse/ATLAS-1947
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> The search result will now include the following
> 
> 1. guid -> referredEntity
> 2. Any valid attribute found in the search filter will be included for each 
> entity in the result
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/discovery/AtlasSearchResult.java 
> 9513dcb1 
>   intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java 
> 30855dcf 
>   
> repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java
>  030a9577 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  5068fa5f 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/discovery/FullTextSearchProcessor.java
>  PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/discovery/GremlinStep.java 
> 1056b3ef 
>   repository/src/main/java/org/apache/atlas/discovery/SearchContext.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/discovery/SearchPipeline.java 
> 0f91b2d1 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/discovery/SolrStep.java 6a5dd5ae 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  4ac00a32 
>   repository/src/main/java/org/apache/atlas/util/SearchTracker.java 15a8c203 
>   repository/src/test/java/org/apache/atlas/TestModules.java d0da0303 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
> 1a9f57a3 
>   webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java dde300e2 
> 
> 
> Diff: https://reviews.apache.org/r/60852/diff/3/
> 
> 
> Testing
> ---
> 
> mvn clean package runs successfully
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 60863: ATLAS-1952 : UI : Render Search table using referredEntity.

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60863/#review180604
---


Ship it!




Ship It!

- Madhan Neethiraj


On July 14, 2017, 12:34 p.m., keval bhatt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60863/
> ---
> 
> (Updated July 14, 2017, 12:34 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, Nixon Rodrigues, and 
> Sarath Subramanian.
> 
> 
> Bugs: ATLAS-1952
> https://issues.apache.org/jira/browse/ATLAS-1952
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> UI becomes more chatty when user select dynamic columns example 
> Column,Table,db so ATLAS-1947(from server side) will fix this issue by adding 
> referredEntity attribute in search result.
> 
> UI will use this referredEntity and render the table.
> 
> 
> Diffs
> -
> 
>   dashboardv2/public/js/collection/VSearchList.js 6291838 
>   dashboardv2/public/js/utils/CommonViewFunction.js a04fb3f 
>   dashboardv2/public/js/utils/Utils.js 9071df3 
>   dashboardv2/public/js/views/entity/EntityDetailTableLayoutView.js 87c5dcd 
>   dashboardv2/public/js/views/search/SearchResultLayoutView.js dba27fa 
> 
> 
> Diff: https://reviews.apache.org/r/60863/diff/1/
> 
> 
> Testing
> ---
> 
> Tested and checked in network no extra call is made while selecting dynamic 
> columns.
> 
> 
> Thanks,
> 
> keval bhatt
> 
>

Review Request 60940: ATLAS-1961: Basic search improvement in use of Solr index for attribute filtering

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60940/
---

Review request for atlas.


Bugs: ATLAS-1961
https://issues.apache.org/jira/browse/ATLAS-1961


Repository: atlas


Description
---

Basic search improvement in use of Solr index for attribute filtering


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
605cb15d 
  repository/src/main/java/org/apache/atlas/discovery/SearchContext.java 
55a07f3d 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
14818094 


Diff: https://reviews.apache.org/r/60940/diff/1/


Testing
---

Validated that the search completes much quicker with this update.


Thanks,

Madhan Neethiraj

Re: Review Request 60922: ATLAS-1925: Misc bugfixes for ATLAS-1880

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60922/#review180763
---


Fix it, then Ship it!




Ship It!


repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
Lines 479 (patched)
<https://reviews.apache.org/r/60922/#comment255987>

keep this inside a finally {} block, to ensure that search-context gets 
removed when search method exits.


- Madhan Neethiraj


On July 17, 2017, 9:37 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60922/
> ---
> 
> (Updated July 17, 2017, 9:37 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-1925
> https://issues.apache.org/jira/browse/ATLAS-1925
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Resolves ATLAS-1917, 1922, 1926
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  a4538bda 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> 1a2d997b 
> 
> 
> Diff: https://reviews.apache.org/r/60922/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean package executes successfully
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 60742: ATLAS-1938 : UI - Search using entity and trait attributes - Refresh Button fires the search query (Basic/DSL) again.

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60742/#review180030
---


Ship it!




Ship It!

- Madhan Neethiraj


On July 10, 2017, 11:59 a.m., Kalyani Kashikar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60742/
> ---
> 
> (Updated July 10, 2017, 11:59 a.m.)
> 
> 
> Review request for atlas, keval bhatt, Madhan Neethiraj, and Nixon Rodrigues.
> 
> 
> Bugs: ATLAS-1938
> https://issues.apache.org/jira/browse/ATLAS-1938
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> On clicking of Refresh Button, makes the following 2 queries :
>  1./api/atlas/v2/types/typedefs/headers
>  2.Search query
> Before the new search feature in place, only query no.1 was made.
> In Advanced search tab , when refresh button is clicked without any type or 
> query , following invalid query is made in addition to query no.1
> http://localhost:21000/api/atlas/v2/search?
> which throws 500 Internal server error.
> 
> 
> Diffs
> -
> 
>   dashboardv2/public/js/views/search/SearchLayoutView.js 7ac394c 
> 
> 
> Diff: https://reviews.apache.org/r/60742/diff/1/
> 
> 
> Testing
> ---
> 
> Verified when browser URL contains type/tag/query in basic or advance search.
>  1./api/atlas/v2/types/typedefs/headers
>  2.Search query 
> Otherwise on click of refresh button only one api is called i.e 
> /api/atlas/v2/types/typedefs/headers.
> 
> 
> Thanks,
> 
> Kalyani Kashikar
> 
>

Re: Review Request 60940: ATLAS-1961: Basic search improvement in use of Solr index for attribute filtering

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60940/
---

(Updated July 19, 2017, 12:02 a.m.)


Review request for atlas.


Changes
---

Updates to improve performance by:
- adding type and classification pre-filter in fulltext search
- having entity processor handle classification filter as well, when no 
classificaiton attribute filtering is specified


Bugs: ATLAS-1961
https://issues.apache.org/jira/browse/ATLAS-1961


Repository: atlas


Description
---

Basic search improvement in use of Solr index for attribute filtering


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
3204ecf2 
  
repository/src/main/java/org/apache/atlas/discovery/FullTextSearchProcessor.java
 4ddd6425 
  repository/src/main/java/org/apache/atlas/discovery/SearchContext.java 
8dd7667e 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
ff0bd2ee 


Diff: https://reviews.apache.org/r/60940/diff/2/

Changes: https://reviews.apache.org/r/60940/diff/1-2/


Testing
---

Validated that the search completes much quicker with this update.


Thanks,

Madhan Neethiraj

Re: Review Request 60971: ATLAS-1914 : UI - Search using entity and trait attributes : "Clear" button doesn't clear the filter applied to a type

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60971/#review180948
---


Ship it!




Ship It!

- Madhan Neethiraj


On July 19, 2017, 2:22 p.m., keval bhatt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60971/
> ---
> 
> (Updated July 19, 2017, 2:22 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, Nixon Rodrigues, and 
> Sarath Subramanian.
> 
> 
> Bugs: ATLAS-1914
> https://issues.apache.org/jira/browse/ATLAS-1914
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch handles following use cases.
> 
> * Clicking on clear button will clear all filters for type and tag.
> * Prevent attribute popup to be closed, if values are empty.
> * Allow to close popup if all attributes are cleared.
> * If tag is deleted then hash will be flushed.
> 
> 
> Diffs
> -
> 
>   dashboardv2/public/js/router/Router.js 8a1d241 
>   dashboardv2/public/js/views/business_catalog/SideNavLayoutView.js 013c72e 
>   dashboardv2/public/js/views/search/QueryBuilderView.js 468a2d7 
>   dashboardv2/public/js/views/search/SearchLayoutView.js cb8844d 
>   dashboardv2/public/js/views/tag/TagLayoutView.js b8a1cd1 
> 
> 
> Diff: https://reviews.apache.org/r/60971/diff/1/
> 
> 
> Testing
> ---
> 
> All above scenario are tested.
> 
> 
> Thanks,
> 
> keval bhatt
> 
>

Re: Review Request 60970: ATLAS-1901: Tolerate no propogatetags on Relationshipdef

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60970/#review180940
---




repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipDefStoreV1.java
Lines 484 (patched)
<https://reviews.apache.org/r/60970/#comment256292>

@David - makes sense. I will commit the patch shortly. Thanks!


- Madhan Neethiraj


On July 19, 2017, 2:26 p.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60970/
> ---
> 
> (Updated July 19, 2017, 2:26 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-1901: Tolerate no propogatetags on Relationshipdef
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipDefStoreV1.java
>  ddf0af9abc5f07c355043e7aff54bfaae1323814 
> 
> 
> Diff: https://reviews.apache.org/r/60970/diff/1/
> 
> 
> Testing
> ---
> 
> Ran unit tests.
> I looked into creating a unit test - but this would have involved mocking 
> Vertex or the equivalent whcih seemed overkill for this small change.
> Created a relationshipDef without propogatetags set. Without the fix it 
> errors - with the fix it works producing a relationshipDef with propogateTags 
> = NONE.
> 
> 
> Thanks,
> 
> David Radley
> 
>

Re: Review Request 60938: [ATLAS-1959]: Enhance relationship attributes to support different cardinality mappings

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60938/#review180899
---




repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
Line 466 (original), 468 (patched)
<https://reviews.apache.org/r/60938/#comment256223>

for relationshipAttributes, it is not neecessary to retrieve complete 
entity (and store it in entityExtInfo.referredEntities) - it could be expensive 
to retrieve and transmit when large number of entities are referenced (for 
example hive_db.tables referring to 1000s of tables). Storing AtlasObjectId  
(or List) is enough.

Same comment for line #474 below.


- Madhan Neethiraj


On July 18, 2017, 6:25 a.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60938/
> ---
> 
> (Updated July 18, 2017, 6:25 a.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-1959
> https://issues.apache.org/jira/browse/ATLAS-1959
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> * Improve relationship model to support create/update operations and the 
> following cardinalities (previously supported using inverseReference):
> 1 to 1
> 1 to many
> many to 1
> many to many
> 
> 
> * Change legacyLabel flag in AtlasRelationshipEndDef to boolean flag.
> * Add unit tests for the above cases.
> 
> 
> Diffs
> -
> 
>   addons/models/0010-base_model.json 303f3796 
>   addons/models/0030-hive_model.json a795f0f3 
>   addons/models/0050-falcon_model.json 7755fa86 
>   addons/models/0060-hbase_model.json 1d264df4 
>   addons/models/0080-storm_model.json 25360ff0 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 
> 68da6af1 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 
> 2de9bdf0 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipDef.java 
> fc820d49 
>   
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipEndDef.java
>  f80ea895 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java e94dd190 
>   intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 
> 841b66f7 
>   intg/src/main/java/org/apache/atlas/type/AtlasStructType.java f97d7674 
>   intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 
> PRE-CREATION 
>   intg/src/test/java/org/apache/atlas/TestUtilsV2.java 9774583d 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 6f6d74bc 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java
>  12e8bb1f 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
>  cd9a47ad 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
>  3ff6fbef 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  d4fdc257 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  157f8cd2 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  f4257be7 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
>  de8e7ef3 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ZipFileResourceTestUtils.java
>  d9017319 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreHardDeleteV1Test.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreSoftDeleteV1Test.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
>  67702231 
> 
> 
> Diff: https://reviews.apache.org/r/60938/diff/2/
> 
> 
> Testing
> ---
> 
> added unit test - AtlasRelationshipStoreV1Test
> 
> mvn clean package - succeeded with no errors.
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 61007: [ATLAS-1979]: Update storm model relationship category and fix for UT and Coverity scan issues

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61007/#review181089
---


Ship it!




Ship It!

- Madhan Neethiraj


On July 20, 2017, 9:16 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61007/
> ---
> 
> (Updated July 20, 2017, 9:16 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-1979
> https://issues.apache.org/jira/browse/ATLAS-1979
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Update storm_topology_nodes relationshipCategory from ASSOCIATION to 
> AGGREGATION
> Fix the following UT failures:
> _Failed tests: 
> AtlasRelationshipStoreHardDeleteV1Test>AtlasRelationshipStoreV1Test.testDepartmentEmployeeEntitiesUsingRelationship:172
>  expected: but was:<[]>
> AtlasRelationshipStoreSoftDeleteV1Test>AtlasRelationshipStoreV1Test.testDepartmentEmployeeEntitiesUsingRelationship:172
>  expected: but was:<[]>_
> Fix coverity scan issues flagged in last report related to ATLAS-1959
> 
> 
> Diffs
> -
> 
>   addons/models/0080-storm_model.json b008c7ab 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  1282be5e 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java
>  c5eda370 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
>  31efe868 
> 
> 
> Diff: https://reviews.apache.org/r/61007/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean package succeeded
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 60980: ATLAS-1961: Basic search improvement in use of Solr index for attribute filtering (# 3)

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60980/
---

(Updated July 21, 2017, 1:39 a.m.)


Review request for atlas.


Changes
---

updates to handle spaces in tag-names, while searcing in Solr index


Bugs: ATLAS-1961
https://issues.apache.org/jira/browse/ATLAS-1961


Repository: atlas


Description
---

- fixed NPE when search includes a free-text query & a classification that has 
large number of sub-types
- fixed entity search in Solr to include state filter


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
a3525c93 
  
repository/src/main/java/org/apache/atlas/discovery/FullTextSearchProcessor.java
 1b19a0e2 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
7950127d 


Diff: https://reviews.apache.org/r/60980/diff/4/

Changes: https://reviews.apache.org/r/60980/diff/3-4/


Testing
---

- validated that the fix addressed the error cases listed above


Thanks,

Madhan Neethiraj

Re: GIT repo move

[Madhan Neethiraj]

Team Atlas,

Graduation related move was completed earlier today. Here are the new location 
details:
 - Git repo URL:https://git-wip-us.apache.org/repos/asf/atlas.git
 - Git repo mirror: https://github.com/apache/atlas
 - Releases:https://dist.apache.org/repos/dist/release/atlas
 - Web site:http:/atlas.apache.org
 - Mailing lists:
   - Dev: dev@atlas.apache.org
   - Users:   us...@atlas.apache.org
   - Commits: comm...@atlas.apache.org

Git mirror may not be accessible immediately; until them please continue to use 
https://github.com/apache/incubator-atlas.

I will shortly update incubator references in documents and website. Please let 
me know if you have any questions.

Thanks,
Madhan

On 7/1/17, 11:00 PM, "Madhan Neethiraj" <mneethi...@hortonworks.com on behalf 
of mad...@apache.org> wrote:

INFRA-14486 is the JIRA to track infrastructure changes related to Atlas 
moving to a TLP. I will keep this list updated of the progress.

Thanks,
Madhan

On 7/1/17, 10:43 AM, "Madhan Neethiraj" <mad...@apache.org> wrote:

Team Atlas,

 

Congratulations on graduation to a Top Level Project!

 

As we move out of incubation to a TLP, our git repo will move from 
https://git-wip-us.apache.org/repos/asf/incubator-atlas.git to 
https://git-wip-us.apache.org/repos/asf/atlas.git. I will be filing an INFRA 
ticket for this move shortly. I will send a note once the ticket is filed; you 
can track the progress via this ticket. I will send another note after the move 
is complete.

 

If you have any questions/comments, please reply to this thread.

 

Thanks,

Madhan

 

 

Re: Review Request 61046: ATLAS-1960: Export, Import & ImportFile APIs EndPoints Handled in HA PASSIVE State

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61046/#review181161
---


Ship it!




Ship It!

- Madhan Neethiraj


On July 21, 2017, 9:45 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61046/
> ---
> 
> (Updated July 21, 2017, 9:45 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Nixon Rodrigues.
> 
> 
> Bugs: ATLAS-1960
> https://issues.apache.org/jira/browse/ATLAS-1960
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Background**
> Export & Import APIs did not generate correct error in HA-mode on a passive 
> node. 
> 
> **Root Cause**
> The _ActiveServerFilter_ handles redirection based when HA-mode is active. 
> There is a special treatment given to _admin_ URIs, where _admin_ URIs are 
> allowed to execute even on PASSIVE node.
> 
> **Solution**
> Since Export, Import APIs are on _AdminResource_, the special treatmean to 
> _admin_ URIs has been extended to exclude execution of these API end points.
> 
> **Additional Comment**
> We may have to add few more method endpoints to this list, as those may 
> encounter similar issue. Potentially, _metrics_ & _activeSearches_ may have 
> this same problem.
> 
> 
> Diffs
> -
> 
>   webapp/src/main/java/org/apache/atlas/web/filters/ActiveServerFilter.java 
> 36f2688e 
>   
> webapp/src/test/java/org/apache/atlas/web/filters/ActiveServerFilterTest.java 
> a8d11103 
> 
> 
> Diff: https://reviews.apache.org/r/61046/diff/1/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> - Updated unit tests.
> - Executed existing tests.
> 
> **Function tests**
> On local VM by changing application property and using debugger to step thru.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 61085: ATLAS-1983: Add relationship cardinality validation

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61085/#review181253
---




repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
Lines 128 (patched)
<https://reviews.apache.org/r/61085/#comment256778>

consider using vertex.getEdges(direction, edgeLabel), instead of iterating 
through edges to find the one with the expected label.


- Madhan Neethiraj


On July 24, 2017, 5:18 p.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61085/
> ---
> 
> (Updated July 24, 2017, 5:18 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-1983: Add relationship cardinality validation
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
> b24f99f6f9337aa10f40e9f10024fe5a345c3540 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipDef.java 
> c17e875d10169753b76fcdb483e2ca85195104b2 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
>  49e08a070a803a36253b502666947ab92ffe39d3 
> 
> 
> Diff: https://reviews.apache.org/r/61085/diff/1/
> 
> 
> Testing
> ---
> 
> ran Junits 
> Successfully added one relationship, then added a second which resulted in 
> the error.
> 
> 
> Thanks,
> 
> David Radley
> 
>

Review Request 61073: ATLAS-1982: Update references to "incubator" in website

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61073/
---

Review request for atlas.


Bugs: ATLAS-1982
https://issues.apache.org/jira/browse/ATLAS-1982


Repository: atlas


Description
---

Now that Atlas has graduated to a top-level project, updated website to remove 
incubator related references (git repo, emails, URLs).


Diffs
-

  DISCLAIMER.txt f577155d 
  README.txt 9716eb98 
  dashboardv2/package.json bcf5fe77 
  dashboardv2/pom.xml 5a2057d6 
  dashboardv2/public/js/templates/business_catalog/BusinessCatalogHeader.html 
ae172ccb 
  dashboardv2/public/js/templates/search/AdvancedSearchInfo_tmpl.html 9b8e1bc5 
  dashboardv2/public/js/templates/site/header.html 558dd678 
  docs/pom.xml 018904d6 
  docs/src/site/site.xml bf7c0c9c 
  docs/src/site/twiki/InstallationSteps.twiki 4869f9f9 
  docs/src/site/twiki/index.twiki 82d27c05 
  pom.xml ebc07e21 
  webapp/pom.xml 465d2a54 


Diff: https://reviews.apache.org/r/61073/diff/1/


Testing
---

Built the site and verified that incubator references are removed.


Thanks,

Madhan Neethiraj

Review Request 60980: ATLAS-1961: Basic search improvement in use of Solr index for attribute filtering (# 3)

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60980/
---

Review request for atlas.


Bugs: ATLAS-1961
https://issues.apache.org/jira/browse/ATLAS-1961


Repository: atlas


Description
---

- fixed NPE when search includes a free-text query & a classification that has 
large number of sub-types
- fixed entity search in Solr to include state filter


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 77b2c7c2 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
50376ef1 
  
repository/src/main/java/org/apache/atlas/discovery/FullTextSearchProcessor.java
 83368c2b 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
596b43bc 


Diff: https://reviews.apache.org/r/60980/diff/1/


Testing
---

- validated that the fix addressed the error cases listed above


Thanks,

Madhan Neethiraj

Re: Review Request 60938: [ATLAS-1959]: Enhance relationship attributes to support different cardinality mappings

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60938/#review181009
---


Ship it!




Ship It!

- Madhan Neethiraj


On July 19, 2017, 7:27 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60938/
> ---
> 
> (Updated July 19, 2017, 7:27 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-1959
> https://issues.apache.org/jira/browse/ATLAS-1959
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> * Improve relationship model to support create/update operations and the 
> following cardinalities (previously supported using inverseReference):
> 1 to 1
> 1 to many
> many to 1
> many to many
> 
> 
> * Change legacyLabel flag in AtlasRelationshipEndDef to boolean flag.
> * Add unit tests for the above cases.
> 
> 
> Diffs
> -
> 
>   addons/models/0010-base_model.json 303f3796 
>   addons/models/0030-hive_model.json a795f0f3 
>   addons/models/0050-falcon_model.json 7755fa86 
>   addons/models/0060-hbase_model.json 1d264df4 
>   addons/models/0080-storm_model.json 25360ff0 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 
> 68da6af1 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 
> 2de9bdf0 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipDef.java 
> fc820d49 
>   
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipEndDef.java
>  f80ea895 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java e94dd190 
>   intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 
> 841b66f7 
>   intg/src/main/java/org/apache/atlas/type/AtlasStructType.java f97d7674 
>   intg/src/test/java/org/apache/atlas/TestRelationshipUtilsV2.java 
> PRE-CREATION 
>   intg/src/test/java/org/apache/atlas/TestUtilsV2.java 9774583d 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 6f6d74bc 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java
>  12e8bb1f 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
>  cd9a47ad 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
>  3ff6fbef 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  d4fdc257 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  157f8cd2 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  f4257be7 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
>  404225c2 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ZipFileResourceTestUtils.java
>  d9017319 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreHardDeleteV1Test.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreSoftDeleteV1Test.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
>  67702231 
> 
> 
> Diff: https://reviews.apache.org/r/60938/diff/4/
> 
> 
> Testing
> ---
> 
> added unit test - AtlasRelationshipStoreV1Test
> 
> mvn clean package - succeeded with no errors.
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 60980: ATLAS-1961: Basic search improvement in use of Solr index for attribute filtering (# 3)

[Madhan Neethiraj]

> On July 19, 2017, 7:42 p.m., Apoorv Naik wrote:
> > repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java
> > Line 181 (original), 181 (patched)
> > <https://reviews.apache.org/r/60980/diff/1/?file=1779714#file1779714line181>
> >
> > Do it make more sense to pass a boolean flag here to control the 
> > addition of the state to the query ?
> > 
> > EntityProcessor will pass true whereas ClassificationProcessor will 
> > pass false

I choose to read from context, as it is already available in SearchProcessor - 
avoid the need to pass additional parameter.


- Madhan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60980/#review180967
-------


On July 19, 2017, 9:59 p.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60980/
> ---
> 
> (Updated July 19, 2017, 9:59 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-1961
> https://issues.apache.org/jira/browse/ATLAS-1961
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> - fixed NPE when search includes a free-text query & a classification that 
> has large number of sub-types
> - fixed entity search in Solr to include state filter
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  77b2c7c2 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
>  50376ef1 
>   
> repository/src/main/java/org/apache/atlas/discovery/FullTextSearchProcessor.java
>  83368c2b 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> 596b43bc 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
>  cd9a47ad 
> 
> 
> Diff: https://reviews.apache.org/r/60980/diff/2/
> 
> 
> Testing
> ---
> 
> - validated that the fix addressed the error cases listed above
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Relationship attributes

[Madhan Neethiraj]

Current name of ‘relationshipAttributes’ makes sense looking from an entity 
point-of-view – it distinguishes regular-attributes of an entity from 
attributes injected by relationships. However, given that relationships can 
themselves might have attributes, it can be confusing.

I was going to suggest ‘relatedEntities’; but ‘relationships’ seems to be good 
choice.

+1 for naming the field as ‘relationships’.

Thanks,
Madhan


On 7/24/17, 12:53 AM, "David Radley" <david_rad...@uk.ibm.com> wrote:

Hi Sarath,
Great, personally for the Entity's current relationshipAttributes, I 
prefer relationships as it is simpler - is there a reason you need 
attribute in the name?
 all the best, David. 



From:   Sarath Subramanian <sar...@apache.org>
To: dev@atlas.apache.org
Cc: Madhan Neethiraj <mad...@apache.org>
Date:   24/07/2017 07:09
Subject:Re: Relationship attributes



Hi David,

I agree with using the term 'relationship attributes' for attributes of
relationship, I suggest we use "relatedAttributes" for relationship
attributes of entity.


Thanks,
Sarath Subramanian

On Sun, Jul 23, 2017 at 2:22 AM, David Radley <david_rad...@uk.ibm.com>
wrote:

> Hi Madhan,
> When I see the phrase 'relationship attributes', I am never quite sure
> whether we are referring to the attributes of an entity that relate to
> another entity or the attributes of the relationship instance itself. I
> think the phrase ' relationship attributes' more naturally fits as the
> attributes of the relationship itself; we are using it in the other 
sense.
>
> I suggest we change the relationshipAttributes in the entity to
> relationships (if you really want attributes in the name we could call 
it
> relatingAttributes)- and use the term 'relationship attributes' purely 
for
> the attributes of the relationship itself. What do you think?
> all the best, David.
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 
3AU
>



Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Re: Review Request 60970: ATLAS-1901: Tolerate no propogatetags on Relationshipdef

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60970/#review180928
---


Fix it, then Ship it!





repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipDefStoreV1.java
Lines 484 (patched)
<https://reviews.apache.org/r/60970/#comment256274>

It will be good to handle this while reading from vertex as well - in 
toRelationshipDef(vertex) below, so that null value in the store gets mapped to 
NONE.


- Madhan Neethiraj


On July 19, 2017, 2:26 p.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60970/
> ---
> 
> (Updated July 19, 2017, 2:26 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-1901: Tolerate no propogatetags on Relationshipdef
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipDefStoreV1.java
>  ddf0af9abc5f07c355043e7aff54bfaae1323814 
> 
> 
> Diff: https://reviews.apache.org/r/60970/diff/1/
> 
> 
> Testing
> ---
> 
> Ran unit tests.
> I looked into creating a unit test - but this would have involved mocking 
> Vertex or the equivalent whcih seemed overkill for this small change.
> Created a relationshipDef without propogatetags set. Without the fix it 
> errors - with the fix it works producing a relationshipDef with propogateTags 
> = NONE.
> 
> 
> Thanks,
> 
> David Radley
> 
>

Re: Review Request 60980: ATLAS-1961: Basic search improvement in use of Solr index for attribute filtering (# 3)

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60980/
---

(Updated July 19, 2017, 9:59 p.m.)


Review request for atlas.


Changes
---

Updates to reuse temp lists during search iterations


Bugs: ATLAS-1961
https://issues.apache.org/jira/browse/ATLAS-1961


Repository: atlas


Description
---

- fixed NPE when search includes a free-text query & a classification that has 
large number of sub-types
- fixed entity search in Solr to include state filter


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 77b2c7c2 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
50376ef1 
  
repository/src/main/java/org/apache/atlas/discovery/FullTextSearchProcessor.java
 83368c2b 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
596b43bc 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
 cd9a47ad 


Diff: https://reviews.apache.org/r/60980/diff/2/

Changes: https://reviews.apache.org/r/60980/diff/1-2/


Testing
---

- validated that the fix addressed the error cases listed above


Thanks,

Madhan Neethiraj

Review Request 61040: ATLAS-1981: Cache escaped type-query string to avoid repeated computation

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61040/
---

Review request for atlas.


Bugs: ATLAS-1981
https://issues.apache.org/jira/browse/ATLAS-1981


Repository: atlas


Description
---

Updated search to use cached type-query string to improve performance


Diffs
-

  common/src/main/java/org/apache/atlas/repository/Constants.java e8621cf3 
  intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 
c1c3add4 
  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java e3005ee1 
  intg/src/main/java/org/apache/atlas/type/AtlasStructType.java b390a97c 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 281422ba 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 745f9d7b 
  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
dbe11a69 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
efcfb7fe 
  
repository/src/main/java/org/apache/atlas/discovery/FullTextSearchProcessor.java
 0d1c39b7 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
28621658 


Diff: https://reviews.apache.org/r/61040/diff/1/


Testing
---

verified that the search returns correct results after this enhancement


Thanks,

Madhan Neethiraj

Re: Review Request 60159: ATLAS-1880: Search using entity/trait attributes

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60159/#review179336
---


Ship it!




Ship It!

- Madhan Neethiraj


On June 30, 2017, 6:27 a.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60159/
> ---
> 
> (Updated June 30, 2017, 6:27 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Sarath 
> Subramanian, and Suma Shivaprasad.
> 
> 
> Bugs: ATLAS-1880
> https://issues.apache.org/jira/browse/ATLAS-1880
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> See JIRA
> 
> 
> Sample Request structure
> 
> ```json
> {
>   "typeName": "hive_table",
>   "excludeDeletedEntities": false,
>   "limit": 25,
>   "offset": 0,
>   "entityFilters": {
>   "condition": "AND",
>   "criterion": [{
>   "attributeName": "retention",
>   "operator": "=",
>   "attributeValue": "10"
>   }, {
>   "condition": "OR",
>   "criterion": [{
>   "attributeName": "createTime",
>   "operator": ">",
>   "attributeValue": "11"
>   }, {
>   "attributeName": "lastAccessTime",
>   "operator": "<=",
>   "attributeValue": "22"
>   }]
>   }]
>   }
> }
> ```
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/atlas/AtlasClientV2.java 61413428 
>   common/src/main/java/org/apache/atlas/repository/Constants.java ac022528 
>   distro/src/conf/atlas-log4j.xml e6c0d9f6 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphQuery.java
>  841edf71 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasIndexQuery.java
>  1ff9d5ed 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertexQuery.java
>  53f490f5 
>   
> graphdb/common/src/main/java/org/apache/atlas/repository/graphdb/titan/query/NativeTitanGraphQuery.java
>  0211ff05 
>   
> graphdb/common/src/main/java/org/apache/atlas/repository/graphdb/titan/query/TitanGraphQuery.java
>  0077a21f 
>   
> graphdb/common/src/main/java/org/apache/atlas/repository/graphdb/titan/query/expr/AndCondition.java
>  68f0eb28 
>   
> graphdb/common/src/main/java/org/apache/atlas/repository/graphdb/titan/query/expr/HasPredicate.java
>  24e4f5bc 
>   
> graphdb/titan0/src/main/java/com/thinkaurelius/titan/graphdb/query/graph/GraphCentricQueryBuilder.java
>  54ff7cb7 
>   
> graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0IndexQuery.java
>  1ed1734f 
>   
> graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0VertexQuery.java
>  bd8b897f 
>   
> graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/query/NativeTitan0GraphQuery.java
>  7ec6ffeb 
>   
> graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1IndexQuery.java
>  4073dd2a 
>   
> graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1VertexQuery.java
>  4452bcdd 
>   
> graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/query/NativeTitan1GraphQuery.java
>  1ca900d8 
>   intg/src/main/java/org/apache/atlas/model/discovery/AtlasSearchResult.java 
> a402c628 
>   intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java 
> 21b64273 
>   
> repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java
>  923a198b 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  128cdbf9 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java 
> b07091a0 
>   repository/src/main/java/org/apache/atlas/discovery/GremlinStep.java 
> PRE-C

Re: Review Request 60457: ATLS-1899

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60457/#review179467
---


Ship it!




Ship It!

- Madhan Neethiraj


On June 27, 2017, 9:04 a.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60457/
> ---
> 
> (Updated June 27, 2017, 9:04 a.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> fixup the logic around edge creation for relationshipDefs
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipDefStoreV1.java
>  194c5461505b9400cf30d6bc4abebc2c9325c001 
> 
> 
> Diff: https://reviews.apache.org/r/60457/diff/1/
> 
> 
> Testing
> ---
> 
> ran junits
> created a relationshipDef with different types and names 
> created a relationshipDef with ends contianing the same names and types. 
> 
> checked the trace to see that it was as expected in both cases.
> 
> 
> Thanks,
> 
> David Radley
> 
>

GIT repo move

[Madhan Neethiraj]

Team Atlas,

 

Congratulations on graduation to a Top Level Project!

 

As we move out of incubation to a TLP, our git repo will move from 
https://git-wip-us.apache.org/repos/asf/incubator-atlas.git to 
https://git-wip-us.apache.org/repos/asf/atlas.git. I will be filing an INFRA 
ticket for this move shortly. I will send a note once the ticket is filed; you 
can track the progress via this ticket. I will send another note after the move 
is complete.

 

If you have any questions/comments, please reply to this thread.

 

Thanks,

Madhan

 

 

Re: Review Request 61191: [ATLAS-1999]: Use AtlasRelatedObjectId to display relationshipAttribute values during entity retrieval

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61191/#review181861
---


Fix it, then Ship it!





intg/src/main/java/org/apache/atlas/model/instance/AtlasRelatedObjectId.java
Lines 55 (patched)
<https://reviews.apache.org/r/61191/#comment257639>

Constructors at line #55 and #69 shouldn't be needed. Consider removing 
them.



intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
Line 269 (original), 270 (patched)
<https://reviews.apache.org/r/61191/#comment258244>

I think it will be intutive to replace:
  return isValidAttributeValue(obj)

with the following:
  return super.isValidValue(obj) && validateRelationshipAttributes(obj)



intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
Lines 496 (patched)
<https://reviews.apache.org/r/61191/#comment258245>

Consider renaming this method as "validateRelationshipAttributes()"



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
Lines 718 (patched)
<https://reviews.apache.org/r/61191/#comment258239>

Consider using AtlasRelatedObjectId constructor which takes relationship 
guid and attributes as well.


- Madhan Neethiraj


On July 27, 2017, 7:49 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61191/
> ---
> 
> (Updated July 27, 2017, 7:49 p.m.)
> 
> 
> Review request for atlas, David Radley and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-1999
> https://issues.apache.org/jira/browse/ATLAS-1999
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> * Currently relationship attributes uses AtlasObjectId format during entity 
> retrieval. Change this to use AtlasRelatedObjectId to contain additional 
> information for use in search and UI
> * validate and normalize relationship attribute values supplied during entity 
> creation
> 
> 
> Diffs
> -
> 
>   
> intg/src/main/java/org/apache/atlas/model/instance/AtlasRelatedObjectId.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java f89c5568 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> c47a89e6 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
>  49e08a07 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  31fc8377 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
>  263ad5be 
> 
> 
> Diff: https://reviews.apache.org/r/61191/diff/4/
> 
> 
> Testing
> ---
> 
> * tested using POSTMAN rest client - valid relationship attribute values in 
> AtlasRelatedObjectId format is retrieved
> * mvn clean package - succeeded
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 61518: ATLAS-2033: Changed Pretty JSON Generation to Ugly for v1 Entity Serializers

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61518/#review182471
---


Ship it!




Ship It!

- Madhan Neethiraj


On Aug. 9, 2017, 4:43 a.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61518/
> ---
> 
> (Updated Aug. 9, 2017, 4:43 a.m.)
> 
> 
> Review request for atlas and Nixon Rodrigues.
> 
> 
> Bugs: ATLAS-2033
> https://issues.apache.org/jira/browse/ATLAS-2033
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Implementation**
> 
> Changed _writePretty_ to _write_ for GSON serializer for v1 entities.
> 
> 
> Diffs
> -
> 
>   
> typesystem/src/main/scala/org/apache/atlas/typesystem/json/InstanceSerialization.scala
>  3354d7c 
> 
> 
> Diff: https://reviews.apache.org/r/61518/diff/1/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> Existing suite.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 61195: ATLAS-1995: Performance Improvement Using IndexQuery for Attribute-based Entity Lookup

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61195/#review181625
---


Ship it!




Ship It!

- Madhan Neethiraj


On July 27, 2017, 10:34 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61195/
> ---
> 
> (Updated July 27, 2017, 10:34 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-1995
> https://issues.apache.org/jira/browse/ATLAS-1995
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> See [ATLAS-1995](https://issues.apache.org/jira/browse/ATLAS-1995) for 
> details.
> 
> **Implementation**
> The _AtlasGraphUtilsV1.findByUniqueAttributes_ now determines if the 
> attribute specified in the request is indexed. If yes, then index query will 
> be used to fetch the entity. If not, the older graph query mechanism will be 
> utilized.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
>  948d9dd 
>   
> webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
>  51276d3 
> 
> 
> Diff: https://reviews.apache.org/r/61195/diff/1/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> None.
> 
> **Functional tests**
> Verified the performance improvement using the _AtlasPerfTracer_.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Draft Board Report for Apache Atlas: August-2017

[Madhan Neethiraj]

Atlas team,

 

Please review the draft board report below and send your feedback/comments, if 
any. The report is due on Wednesday this week, August-9.

 

Thanks,

Madhan

 

 

 

 

## Description:

  Apache Atlas is a scalable and extensible set of core foundational governance

  services that enables enterprises to effectively and efficiently meet their

  compliance requirements within Hadoop and allows integration with the

  complete enterprise data ecosystem

 

## Issues:

  There are no issues requiring board attention at this time.

 

## Activity:

  - Migration from incubator infra to TLP infra (INFRA-14486) is now complete

  - Planning to release 0.8.1, the first release after graduation, in August

  - Continuing work on core enhancements like relationships as first class type

  - Working on new features like Virtual Data Connector, Automated Discovery

 

## Health report:

  - Added 1 new committer in last month

  - Invited 2 more for committer role in last month. Yet to receive acceptance

    from them, as both are on vacation

  - Added 6 new contributors in last month

  - A lot of interest in adding new features to Atlas from the community,

like: Virtual Data Connector, Automated Discovery, Business User UIs

 

## PMC changes:

  - Currently 33 PMC members

  - No new PMC members added in last month

  - Last PMC member addition was on 6/21/2017

 

## Committer base changes:

  - Currently 34 committers

  - 1 new committer added in last month

  - Last committer addition was Ashutosh Mestry, on 7/29/2017

 

## Releases:

 0.8.1 release work is in progress

 0.8-incubating   was released on 03/16/2017

 0.7.1-incubating was released on 01/26/2017

 0.7-incubating   was released on 07/09/2016

 0.6-incubating   was released on 12/31/2015

 0.5-incubating   was released on 07/11/2015

 

 

Re: Review Request 61747: [ATLAS-2059]: Update test-patch.sh script for PreCommit-ATLAS-Build jenkins job

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61747/#review183245
---


Ship it!




Ship It!

- Madhan Neethiraj


On Aug. 18, 2017, 6:11 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61747/
> ---
> 
> (Updated Aug. 18, 2017, 6:11 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2059
> https://issues.apache.org/jira/browse/ATLAS-2059
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Update test-patch.sh script for PreCommit-ATLAS-Build jenkins job and fix for 
> one failed IT
> 
> 
> Diffs
> -
> 
>   dev-support/test-patch.sh 9e63575d 
>   webapp/src/test/java/org/apache/atlas/examples/QuickStartV2IT.java 773a514d 
>   webapp/src/test/java/org/apache/atlas/web/integration/BaseResourceIT.java 
> 496185fe 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/EntityJerseyResourceIT.java
>  b527583a 
> 
> 
> Diff: https://reviews.apache.org/r/61747/diff/1/
> 
> 
> Testing
> ---
> 
> validated in jenkins test job - 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/
> 
> We can test patches using jenkins job (Build with parameters) using:
> * Review Board ID
> * JIRA ID
> * Local Atlas Patch
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 61721: [ATLAS-2054]: Add flag to exclude attribute mapping in FullTextMapper

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61721/#review183156
---




repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapperV2.java
Line 127 (original), 143 (patched)
<https://reviews.apache.org/r/61721/#comment259181>

excludeAttributes - has attributes for the entity-type. It should be for 
classification.getTypeName() here. Please review.


- Madhan Neethiraj


On Aug. 17, 2017, 9:02 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61721/
> ---
> 
> (Updated Aug. 17, 2017, 9:02 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2054
> https://issues.apache.org/jira/browse/ATLAS-2054
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Add property in atlas-application.properties to exclude listed attributes to 
> be excluded when mapping fulltext
> 
> e.g.
> atlas.search.fulltext.type.hive_table.attributes.exclude=profileData
> atlas.search.fulltext.type.hive_column.attributes.exclude=profileData
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapperV2.java
>  1ed12bc0 
> 
> 
> Diff: https://reviews.apache.org/r/61721/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean package suceeded.
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 61085: ATLAS-1983: Add relationship cardinality validation

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61085/#review183043
---




repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
Lines 128 (patched)
<https://reviews.apache.org/r/61085/#comment259037>

David,

You are effectively looking for edges with label 
'AtlasRelationshipDef.RELATIONSHIP_PREFIX + relationshipType'.

So instead of retrieving all edges and looking for label prefix 
'AtlasRelationshipDef.RELATIONSHIP_PREFIX', followed by a substring() and 
equals(relationshipType) - wouldn't it be simpler to retrieve edges with the 
following call:

vertex.getEdges(AtlasEdgeDirection.BOTH, 
AtlasRelationshipDef.RELATIONSHIP_PREFIX + relationshipType)

Madhan


- Madhan Neethiraj


On July 24, 2017, 5:18 p.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61085/
> ---
> 
> (Updated July 24, 2017, 5:18 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-1983: Add relationship cardinality validation
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
> b24f99f6f9337aa10f40e9f10024fe5a345c3540 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipDef.java 
> c17e875d10169753b76fcdb483e2ca85195104b2 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
>  49e08a070a803a36253b502666947ab92ffe39d3 
> 
> 
> Diff: https://reviews.apache.org/r/61085/diff/1/
> 
> 
> Testing
> ---
> 
> ran Junits 
> Successfully added one relationship, then added a second which resulted in 
> the error.
> 
> 
> Thanks,
> 
> David Radley
> 
>

Re: Review Request 61764: ATLAS-2061: Entity, Tag filter fetches empty results when applied on an enum attribute.

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61764/#review183282
---


Ship it!




Ship It!

- Madhan Neethiraj


On Aug. 18, 2017, 11:26 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61764/
> ---
> 
> (Updated Aug. 18, 2017, 11:26 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2061
> https://issues.apache.org/jira/browse/ATLAS-2061
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Entity,Tag filter fetches empty results when applied on an enum attribute.
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> 15564fc0 
> 
> 
> Diff: https://reviews.apache.org/r/61764/diff/1/
> 
> 
> Testing
> ---
> 
> UI and REST tests give correct results
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 61526: ATLAS-2029: Restrict entities, classifications can be applied to

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61526/#review183296
---




intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java
Lines 54 (patched)
<https://reviews.apache.org/r/61526/#comment259322>

>> we need to store the entityTypes specified in our supertypes. i.e. our 
parent classificationDefs may specify more entityTypes that we also need to 
allow

This comment doesn't look correct. A classification can have restricted 
entity-types than its super-types.



intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java
Lines 110 (patched)
<https://reviews.apache.org/r/61526/#comment259323>

>> If our classificationDef or any of our parent ClassificaitonDefs have an 
empty list of entityTypes, then we will not restrict the entities applied to 
this Classification.

This doesn't look correct. It should be possible for a classtification to 
be applicable to any entity-type but its sub-type classifications to allow only 
few entity-types.

Here, we should ensure that 'entityTypes' for this classificationDef (if 
specified) doesn't contain any entityType that is not applicable for superType 
classifications.



intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java
Lines 386 (patched)
<https://reviews.apache.org/r/61526/#comment259324>

blocks between #387 and #392 can be folded into a the following statement. 
Please review:

  boolean canApply = entityTypes.isEmpty() || 
entityTypes.contains(entity.getTypeName()) || 
CollectionUtils.containsAny(entityTypes.entityType.getAllSuperTypes());



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
Lines 728 (patched)
<https://reviews.apache.org/r/61526/#comment259325>

Once again, I am not sure why you would retrieve "AtlasEntityWithExtInfo" 
here - as the only information needed here seem to be typeName of the entity 
with the given guid. Populating AtlasEntityWithExtInfo would require reading 
all properties of the entity *and* properites of all entities referenced by 
this entity. For example, if table has 100s of columns, getById() would read 
vertex for the table and vertices for each column in the table. Since all these 
details are not necessary here, I suggested earlier to retrieve only typeName 
for the entity.



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
Lines 742 (patched)
<https://reviews.apache.org/r/61526/#comment259326>

Ensure that references are resolved when typeRegistry updates are made. 
Whenever *NoRefResolve() methods are used to update the registry, ensure to run 
typeRegistry.resolveReferences() at the end of updates. You can see references 
in number of methods in AtlasTransientTypeRegistry - like addTypes(), 
updateTypes().

Calling resolveReferences() outside of AtlasTransientTypeRegistry is asking 
for trouble, as the would modify the type instance which might be used in other 
threads.

@Sarath - it might be worth removing "public" access to 
AtlasType.resolveReferences() to avoid possible misuse like here.



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.java
Lines 429 (patched)
<https://reviews.apache.org/r/61526/#comment259328>

It will help readability a lot if a space can be added after a comma. 
Please review rest of the patch for this and update. I find having a readable 
code to be one of the cheapest way to spot potential issues.



repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
Line 944 (original), 944 (patched)
<https://reviews.apache.org/r/61526/#comment259327>

For better readability:
  "+e.getMessage()" ==> "+ e.getMessage()"


- Madhan Neethiraj


On Aug. 20, 2017, 11:34 a.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61526/
> -------
> 
> (Updated Aug. 20, 2017, 11:34 a.m.)
> 
> 
> Review request for atlas, Graham Wallis, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-2029: Restrict entities, classifications can be applied to
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
> 2503d8ef203cf4efbe15b440257b1da2252b6153 
>   
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasClassificationDef.java 
> eeaf71413a56c08db8170fd3323b8e8245ae44fe 
>   intg/src/main/java/org/apache/at

Re: Review Request 61825: [ATLAS-2079]: Fix IT failures introduced by ATLAS-2062

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61825/#review183571
---


Ship it!




Ship It!

- Madhan Neethiraj


On Aug. 22, 2017, 10:22 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61825/
> ---
> 
> (Updated Aug. 22, 2017, 10:22 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-2079
> https://issues.apache.org/jira/browse/ATLAS-2079
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Fix the following IT failures:
> Failed tests: 
> NotificationHookConsumerIT.testDeleteByQualifiedName:184->BaseResourceIT.waitFor:621
>  » AtlasService
> NotificationHookConsumerIT.testUpdateEntityPartial:127->BaseResourceIT.waitFor:626
>  » 
> EntityJerseyResourceIT.testPartialUpdate:881 expected:<1> but was:<3>
> MetadataDiscoveryJerseyResourceIT.testSearchUsingGremlin:153 » AtlasService 
> Me...
> 
> 
> Diffs
> -
> 
>   
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
>  70784363 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 2cb8e274 
>   pom.xml f127774d 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 1ec5a72e 
>   typesystem/src/test/resources/atlas-application.properties 7967b761 
>   
> webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerIT.java
>  9c5597ef 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/EntityJerseyResourceIT.java
>  660f05e1 
> 
> 
> Diff: https://reviews.apache.org/r/61825/diff/2/
> 
> 
> Testing
> ---
> 
> The ITs works fine for webapp module.
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 61824: [ATLAS-2078]: Updating types using v1 API doesn't route to createUpdateTypesDef in v2 API

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61824/#review183570
---


Ship it!




Ship It!

- Madhan Neethiraj


On Aug. 22, 2017, 10:09 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61824/
> ---
> 
> (Updated Aug. 22, 2017, 10:09 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Madhan Neethiraj, and 
> Nixon Rodrigues.
> 
> 
> Bugs: ATLAS-2078
> https://issues.apache.org/jira/browse/ATLAS-2078
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> When updating types using v1 API, if type doesn't exist we create missing 
> types. Currently this API maps to v2 API's update method and doesn't create 
> missing types.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java e1c5a7f5 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
>  7a0bbca1 
>   webapp/src/main/java/org/apache/atlas/web/resources/TypesResource.java 
> f70593a3 
> 
> 
> Diff: https://reviews.apache.org/r/61824/diff/1/
> 
> 
> Testing
> ---
> 
> validated using POSTMAN REST client.
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 60272: ATLAS-1891 Additional Validation checks for RelationshipDefs

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60272/#review178890
---


Ship it!




Ship It!

- Madhan Neethiraj


On June 26, 2017, 12:29 p.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60272/
> ---
> 
> (Updated June 26, 2017, 12:29 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-1891
> Additional validation checks for relationshipDefs and add edges between 
> entityDefs and relationshipDef vertices 
> 
> - for create check reserved words are not used on the endpoints and type name 
> and relationship attribute names
> - for update do not allow updates to endpoints or the name or deletion of 
> attributes or relationship category
> - for not allow deletion of an entityDef that has a relatiionshipDef
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
> 70480df113db36cb938a8817239fbeae441acdc0 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java 
> 31256683f4c1e3217ef4b2dba83f34c32e025435 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
>  aefd1684fe4a71cd0b45c4600cc5eca872ab0892 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityDefStoreV1.java
>  8046234f17885cb6009edbcf3f9b80333932c358 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
>  00fe94b69d69d261e37766434b0cfd1e56117c52 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipDefStoreV1.java
>  6992e221884db5382b89a29ea8a7efaad2cadcbc 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.java
>  2ba41443d7d5c51ec83848c5f9ca89720f74b63a 
>   repository/src/test/java/org/apache/atlas/TestModules.java 
> d28956deb349c65d11031dbfb3866b91d5fd37e4 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasRelationshipDefStoreV1Test.java
>  PRE-CREATION 
>   
> webapp/src/main/java/org/apache/atlas/examples/CreateTypesFromJsonFileUtil.java
>  82d6f2e3ee3f6376b7830e11900fda4e7275375b 
>   
> webapp/src/main/java/org/apache/atlas/examples/UpdateTypesFromJsonFileUtil.java
>  3b6770658298849cb70e69c373f97fb4d13d1148 
> 
> 
> Diff: https://reviews.apache.org/r/60272/diff/2/
> 
> 
> Testing
> ---
> 
> run junits
> create reltionshipDef
> get relationshipDef
> get relationshipDef/headers
> delete relationshipDef
> update relationshipDef
> create a relationshipDef and then update it such that an attribute will be 
> deleted. This should fail.
> Set the config option and try to create a reltionshipDef with a reserversed 
> word as an attribute name - this fails
> Delete an entityDef that has a relationshipdef - this fails
> Create a relationship with an endpoint name as a reserved word - this fails
> 
> 
> Thanks,
> 
> David Radley
> 
>

Re: Review Request 59799: ATLAS-1766 : Atlas with Kafka SASL_SSL protocol

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59799/#review178894
---


Ship it!




Ship It!

- Madhan Neethiraj


On June 26, 2017, 8:37 a.m., Nixon Rodrigues wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59799/
> ---
> 
> (Updated June 26, 2017, 8:37 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Madhan Neethiraj, and 
> Sarath Subramanian.
> 
> 
> Bugs: ATLAS-1766
> https://issues.apache.org/jira/browse/ATLAS-1766
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch include changes that implements kafka new-consumer api for 
> kafkaConsumer for Atlas Notification System.
> 
> 
> Diffs
> -
> 
>   distro/src/conf/atlas-application.properties 5e59528b 
>   notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaConsumer.java 
> PRE-CREATION 
>   notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaMessage.java 
> PRE-CREATION 
>   notification/src/main/java/org/apache/atlas/kafka/KafkaConsumer.java 
> 16c0eb2c 
>   notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 
> 8bd31fde 
>   
> notification/src/main/java/org/apache/atlas/notification/AbstractMessageDeserializer.java
>  95858278 
>   
> notification/src/main/java/org/apache/atlas/notification/AbstractNotificationConsumer.java
>  d4d78de9 
>   
> notification/src/main/java/org/apache/atlas/notification/NotificationConsumer.java
>  a99cb105 
>   notification/src/test/java/org/apache/atlas/kafka/KafkaConsumerTest.java 
> ad7d93e2 
>   
> notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationMockTest.java
>  2126be69 
>   
> notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationTest.java 
> a8100290 
>   
> notification/src/test/java/org/apache/atlas/notification/AbstractNotificationConsumerTest.java
>  13f2f0ba 
>   typesystem/src/test/resources/atlas-application.properties c4ce5eaa 
>   
> webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
>  2f8245d2 
>   
> webapp/src/test/java/org/apache/atlas/notification/EntityNotificationIT.java 
> ac3b5381 
>   
> webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java
>  18fd2ee2 
>   
> webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerTest.java
>  bdb60a2f 
>   webapp/src/test/java/org/apache/atlas/web/integration/BaseResourceIT.java 
> b59d3ee5 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/EntityJerseyResourceIT.java
>  310b2e3e 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
>  98a7abcf 
> 
> 
> Diff: https://reviews.apache.org/r/59799/diff/5/
> 
> 
> Testing
> ---
> 
> Tested Kafka consumer on local embeded kafka /zookeeper env.
> Tested Atlas Hive hook notifications.
> Excecute Quick Start script.
> 
> Due to change in kafkaconsumer api the current notification related UT/IT's 
> need to changed, i have not included this change in this patch but will 
> included it shortly.
> 
> 
> Thanks,
> 
> Nixon Rodrigues
> 
>

Re: Review Request 58434: Import API: Improved the API to Update Existing Types Attributes

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58434/#review178929
---




repository/src/main/java/org/apache/atlas/repository/impexp/TypeAttributeDifference.java
Lines 42 (patched)
<https://reviews.apache.org/r/58434/#comment253265>

addElements() ==> updateTypes()



repository/src/main/java/org/apache/atlas/repository/impexp/TypeAttributeDifference.java
Lines 90 (patched)
<https://reviews.apache.org/r/58434/#comment253262>

addAttributes() ==> addElements()


- Madhan Neethiraj


On June 22, 2017, 2:51 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58434/
> ---
> 
> (Updated June 22, 2017, 2:51 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-1734
> https://issues.apache.org/jira/browse/ATLAS-1734
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Background**
> Existing version of Import API allows for importing types that are not 
> already present in the system being imported in. This causes import to fail 
> in the cases where the data being imported happens to have the additional 
> attribute.
> 
> **Approach**
> New class _TypeAttributeDifference_ does a attribute diff betweeing 2 types.
> _ImportService.processTypeDefs_ method now includes additional step for 
> attribute updation.
> Metrics have been updated to reflect the updates.
> 
> **CURL**
> 
> The ZIP files mentioned below can be found at: 
> _repository/src/test/resources/_
> 
> 
> Create a file _importOptions.json_ with contents below:
> ```javascript
> { 
>   "options": {
> "updateTypeDefinition": true
> }
> }
> ```
> 
> Use the following curl to perform an import with default attributes.
> 
> ```javascript
> curl -g -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F data=@salesNewTypeAttrs.zip 
> "http://localhost:21000/api/atlas/admin/import;
> ```
> Note the attributes of the tables. 
> 
> Then perform this import with the new option.
> ```javascript
> curl -g -X POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F request=@importOptions.json -F 
> data=@salesNewTypeAttrs-next.zip 
> "http://localhost:21000/api/atlas/admin/import;
> ```
> **Impact to API Consumer**
> None.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java 
> b19f7097 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java
>  7744c781 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java
>  4ffbb88c 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportTypeDefProcessor.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/impexp/TypeAttributeDifference.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/AtlasImportRequestTest.java
>  0844bf0d 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
>  a6fed628 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTestUtils.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/TypeAttributeDifferenceTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/UniqueListTest.java
>  PRE-CREATION 
>   repository/src/test/resources/salesNewTypeAttrs-next.zip PRE-CREATION 
>   repository/src/test/resources/salesNewTypeAttrs.zip PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/58434/diff/4/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> - _ImportServiceTest_ Updated to handle the new case. New light-weight _zip_ 
> files added test resources to verify the functionality.
> - _TypeAttributeDifferenceTest_ Unit test for _TypeAttributeDifference_ class.
> - Additional unit tests: _UniqueList_.
> - Import-Export APIs verified with test file set.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: [DISCUSS] Restrict AtlasStruct and AtlasClassification attributes to primitive and enum types

[Madhan Neethiraj]

Mandy,

The relationship introduced in ATLAS-1690 requires entity-type be specified for 
each end; effectively injecting an attribute to the entity types specified. 
This doesn’t allow struct or a classification types at relationship ends. In 
addition, allowing structs to hold entity references add complexity in dealing 
with edges, especially considering cases like array/map of structs. None of the 
existing models use object-reference attributes in structs; and we haven’t come 
across such use by any one so far – hence this discuss thread to find existing 
use for structs to hold object references. It will help us understand concrete 
usecases better and perhaps offer an alternate approach to model the same.

Thanks,
Madhan


On 6/28/17, 12:17 AM, "Mandy Chessell"  wrote:

Hello Sarath,
These restrictions will have an impact on the open metadata and governance 
type model - and I expect other models that people have built.  The impact 
of this change is that many of the structs and classifications from the 
open metadata and governance type model will need to be changed to an 
entity plus relationship combo, creating overhead and complexity in the 
model and reducing clarity.  Perhaps it would help to explain why these 
restrictions are needed?

Here are some examples

The restriction that we can only have one instance of a type of 
classification associated with an entity is creating a problem for 
modelling social tagging and assignment of certifications to assets.  As 
such we need to have an array of structs inside these classifications. 
Each cell in the array represents a user's tag or a certification that the 
asset has received.   In the longer term a better solution to this use 
case is of course to allow cardinality on the classification

Structs and classifications often need to include a relationship.  For 
example, if we use a classification to represent the terms and conditions 
that apply to an asset - then ideally we would want a relationship to the 
contract or license type.

I am assuming that maps and arrays considered primitive since that will 
impact the Hive model.

All the best
Mandy
___
Mandy Chessell CBE FREng CEng FBCS
IBM Distinguished Engineer

Master Inventor
Member of the IBM Academy of Technology
Visiting Professor, Department of Computer Science, University of 
Sheffield

Email: mandy_chess...@uk.ibm.com
LinkedIn: http://www.linkedin.com/pub/mandy-chessell/22/897/a49

Assistant: Janet Brooks - jsbrook...@uk.ibm.com



From:   Sarath Subramanian 
To: d...@atlas.incubator.apache.org, u...@atlas.incubator.apache.org
Date:   28/06/2017 02:43
Subject:[DISCUSS] Restrict AtlasStruct and AtlasClassification 
attributes to primitive and enum types



Hello all,



As part of the new relationship design (*ATLAS-1690
)*, we are planning to
restrict AtlasStruct and AtlasClassification types to have attributes of
primitive and enum types only. Attributes that refer to entity/entities
will no longer be supported for AtlasStruct and AtlasClassification types.
Please note that none of the existing out of the box types are impacted by
this change.



If you have any concerns with this change please let us know.





Thanks,

Sarath Subramanian

Re: Review Request 62355: ATLAS-2102 : Atlas UI Improvements: Search results page

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62355/#review185492
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 15, 2017, 2:17 p.m., keval bhatt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62355/
> ---
> 
> (Updated Sept. 15, 2017, 2:17 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Madhan Neethiraj, 
> Mehul Parikh, Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2102, ATLAS-2103, ATLAS-2104, ATLAS-2105, ATLAS-2107, and 
> ATLAS-2108
> https://issues.apache.org/jira/browse/ATLAS-2102
> https://issues.apache.org/jira/browse/ATLAS-2103
> https://issues.apache.org/jira/browse/ATLAS-2104
> https://issues.apache.org/jira/browse/ATLAS-2105
> https://issues.apache.org/jira/browse/ATLAS-2107
> https://issues.apache.org/jira/browse/ATLAS-2108
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Download patch for 
> [branch-0.8](https://issues.apache.org/jira/secure/attachment/12887349/ATLAS-2102-branch-0.8.patch).
> 
> 
> *ATLAS-2102*
> ==
> 
> 1. Previous and Next button should not be present is the number of search 
> results is less than the current page size.
> 2. Search results table shifts visually if the show historical properties is 
> selected (moves right and down by ~20px) needs to have a consistent position 
> in the layout.
> 3. Remove edit entity action button from search Table.
> 4 . Place the pagination button at the bottom.
> 5. Add go to page functionality.
> 6. Add page size drop down.
> 
> *ATLAS-2103*
> ==
> 
> 1. Change the color of notification
> 
> *ATLAS-2104*
> ==
> 
> 1. Missing labels in Edit tag popup: A Tag could have the same Name and 
> Description attribute values. In this case, when editing a tag, the Edit 
> Description modal does not communicate that only "Description" attribute 
> value is editable and "Name" is not editable. Also since the field does not 
> have any label it is confusing as to what attribute is allowed to be modified 
> on edit. It would be nice to prefix the attribute name to the edit dialogs 
> for updating the value (i.e. show "Name :" and "Description :" as prefixes to 
> the field values being edited so it is very clear which field is being edited.
> 2. Background color of Tag Delete popup confirmation can be improved to be a 
> more neutral and lighter color.
> 3. In create tag pop up move "Add Attribute" button to the left side.
> 
> *ATLAS-2105*
> ==
> 
> 1. Renaming search and apply button to search.
> 2. Change background color to white.
> 
> *ATLAS-2107*
> ==
> 
> 1. The current design of the create entity modal dialog can be improved with 
> small changes such as removing dividing lines, justifying the labels and 
> making the dialog box to be have a more consistent look and feel.
> 
> 
> *ATLAS-2108*
> ==
> 
> Currently there is placeholder content in the help popup dialog for Advanced 
> search. Placeholder need to be removed to make the help more useful and the 
> placeholders need to be replaced with relevant content such as from Apache 
> Atlas wiki for DSL search http://atlas.apache.org/Search.html
> 
> 
> Diffs
> -
> 
>   dashboardv2/public/css/scss/__mixin.scss PRE-CREATION 
>   dashboardv2/public/css/scss/breadcrumb.scss dbd17b4 
>   dashboardv2/public/css/scss/business-catlog.scss 8e68fc1 
>   dashboardv2/public/css/scss/form.scss 9989e7c 
>   dashboardv2/public/css/scss/graph.scss 3e55b96 
>   dashboardv2/public/css/scss/loader.scss ff8b890 
>   dashboardv2/public/css/scss/login.scss b917d46 
>   dashboardv2/public/css/scss/main.scss 83c2a5a 
>   dashboardv2/public/css/scss/nav.scss 524e2b8 
>   dashboardv2/public/css/scss/old-style.scss 89e313e 
>   dashboardv2/public/css/scss/override.scss e4e8bf7 
>   dashboardv2/public/css/scss/profile-table.scss 2b0b483 
>   dashboardv2/public/css/scss/search.scss 2bce210 
>   dashboardv2/public/css/scss/style.scss f634c44 
>   dashboardv2/public/css/scss/tab.scss 9b67e2c 
>   dashboardv2/public/css/scss/tag.scss 39bc995 
>   dashboardv2/public/css/scss/theme.scss 5cf8848 
>   dashboardv2/public/css/scss/tree.scss 50396ba 
>   dashboardv2/public/js/router/Router.js a43fbf7 
>   dashboardv2/public/js/templates/audit/AuditTableLayoutView_tmpl.html 
> ca9421d 
>   dashboa

Re: Review Request 62110: ATLAS-2115: Fix Regression on Basic search

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62110/#review185330
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 13, 2017, 7:26 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62110/
> ---
> 
> (Updated Sept. 13, 2017, 7:26 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2115
> https://issues.apache.org/jira/browse/ATLAS-2115
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-1880 introduced regression in call timings for the basic search GET 
> implementation
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java 
> 972c11e3 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  0daab030 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
>  a4a638af 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> d5e39236 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> fc973e6f 
> 
> 
> Diff: https://reviews.apache.org/r/62110/diff/5/
> 
> 
> Testing
> ---
> 
> Tested via scripts and network call timing in Chrome/Firefox.
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 62110: ATLAS-2115: Fix Regression on Basic search

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62110/#review185312
---




repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
Lines 286 (patched)
<https://reviews.apache.org/r/62110/#comment261622>

This comment doesn't look right. Please review and update.



repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
Lines 67 (patched)
<https://reviews.apache.org/r/62110/#comment261624>

Since typeName is a single-value, 'in' predicate seems appropriate. Please 
review.



repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java
Line 325 (original), 400 (patched)
<https://reviews.apache.org/r/62110/#comment261631>

this should be:
 return ((Collection)attrValue).contains(value);



repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java
Lines 526 (patched)
<https://reviews.apache.org/r/62110/#comment261630>

this should be:
 return ((Collection)value).contains(attrVal);


- Madhan Neethiraj


On Sept. 13, 2017, 5:28 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62110/
> ---
> 
> (Updated Sept. 13, 2017, 5:28 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2115
> https://issues.apache.org/jira/browse/ATLAS-2115
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-1880 introduced regression in call timings for the basic search GET 
> implementation
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java 
> 972c11e3 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  0daab030 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
>  a4a638af 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> d5e39236 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> fc973e6f 
> 
> 
> Diff: https://reviews.apache.org/r/62110/diff/4/
> 
> 
> Testing
> ---
> 
> Tested via scripts and network call timing in Chrome/Firefox.
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 62244: ATLAS2058 Add description to attributedefs and relationship enddefs

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62244/#review185420
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 12, 2017, 5:13 p.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62244/
> ---
> 
> (Updated Sept. 12, 2017, 5:13 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS2058 Add description to attributedefs and relationship enddefs
> 
> 
> Diffs
> -
> 
>   
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipEndDef.java
>  01e5ce738e8a5310cee928560364d4a38f5c6e18 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> 3a5c43ac7e6d0a7f683c24d13d6c841de4d66a14 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java
>  c3c42be86d8c9fd282fdc1428c8fd78e338a3cfb 
> 
> 
> Diff: https://reviews.apache.org/r/62244/diff/1/
> 
> 
> Testing
> ---
> 
> Create an entity attribute, relationship attribute and a relationship end 
> with a description and then issue getTypeDefs to check they are there.
> 
> 
> Thanks,
> 
> David Radley
> 
>

Re: Review Request 62039: ATLAS-2100: UserProfile & SavedSearch Implementation

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62039/#review185523
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 16, 2017, 8:52 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62039/
> ---
> 
> (Updated Sept. 16, 2017, 8:52 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, keval bhatt, Madhan Neethiraj, Nixon 
> Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2100
> https://issues.apache.org/jira/browse/ATLAS-2100
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Background**
> - The new search implementation allows for creation of complex queries.
> - This sub-feature allows for logged in user to persist search definitions 
> created during a session.
> - These search definition once saved are available to the user on subsequent 
> login.
> 
> **Implementation**
> Following implementation approaches were attempted:
> - New types defined in the models json.
> - _Data Access Layer_: This is semi-generic approach that needs some code per 
> implementation.
> - _AtlasEntity_ adapters: This implementation leverages existing type system 
> to create new types. The model is a facade over _AtlasEntity_. 
> - Added REST-layer filtering for internal types.
> - Modified _SearchFilter_ more capable. 
> 
> **_New Entities_**
> - _AtlasUserProfile_
> - _AtlasSavedSearch_
> 
> **_UserProfileService_**
> - Interacts with the new entities and adds operations that are exposed by 
> REST APIs.
> 
> **_DiscoveryService_**
> - New methods added.
> 
> **Retrieve list of all Saved Searches**
> ```
> curl -g -X GET -u admin:admin -H "Content-Type: application/json" -H 
> "Cache-Control: no-cache"  "http://localhost:21000/api/atlas/v2/search/saved;
> ```
> 
> **Save Search**
> Save this to _savedSearch.json_:
> ```javascript
> {
> "name": "my_search1",
> "owerUserName": "admin",
> "guid": "61613c37-cab8-4ee9-b3e7-590d0694f6e8",
> "searchParameters": {
> "typeName": "hive_table",
> "excludeDeletedEntities": false,
> "limit": 25,
> "offset": 0,
> "entityFilters": {
> "condition": "AND",
> "criterion": [
> {
> "attributeName": "retention",
> "operator": "=",
> "attributeValue": "10"
> },
> {
> "condition": "OR",
> "criterion": [
> {
> "attributeName": "createTime",
> "operator": ">",
> "attributeValue": "11"
> },
> {
> "attributeName": "lastAccessTime",
> "operator": "<=",
> "attributeValue": "22"
> }
> ]
> }
> ]
> }
> }
> }
> ```
> 
> ```javascript
> curl -g -X POST -u admin:admin -H "Content-Type: application/json" -H 
> "Cache-Control: no-cache"  "http://localhost:21000/api/atlas/v2/search/save; 
> -d @../docs/savedSearch.json
> ```
> 
> **Update Saved Search**
> This will work on existing saved search. Ensure that the parmeter has _guid_ 
> property set.
> 
> ```javascript
> curl -g -X PUT -u admin:admin -H "Content-Type: application/json" -H 
> "Cache-Control: no-cache"  "http://localhost:21000/api/atlas/v2/search/save; 
> -d @../docs/savedSearch.json
> ```
> 
> **Retrieve Saved Search**
> All searches for the logged in user:
> ```javascript
> curl -g -X GET -u admin:admin -H "Content-Type: application/json" -H 
> "Cache-Control: no-cache"  "http://localhost:21000/api/atlas/v2/search/saved;
> ```
> 
> Specific search for logged in user:
> ```javascript

Re: Review Request 62420: ATLAS-2148: Enhancement to Saved Search

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62420/#review185761
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 19, 2017, 11:33 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62420/
> ---
> 
> (Updated Sept. 19, 2017, 11:33 p.m.)
> 
> 
> Review request for atlas, keval bhatt and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2148
> https://issues.apache.org/jira/browse/ATLAS-2148
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Implementation**
> 
> - _DataAccess.save_ now returns entity with GUID set. Logic uses 
> _EntityMutationResponse_ for inferring the GUID.
> - REST APIs have undergone minor changes. CURL calls updated (see below).
> 
> 
> _Please used attached JSON for the examples below._
> 
> **Retrieve List of Saved Searches**
> ```javascript
> curl -g -X GET -u admin:admin -H "Content-Type: application/json" -H 
> "Cache-Control: no-cache"  "http://localhost:21000/api/atlas/v2/search/saved;
> ```
> 
> **Retrieve Specific Saved Searches**
> 
> ```javascript
> curl -g -X GET -u admin:admin -H "Content-Type: application/json" -H 
> "Cache-Control: no-cache"  
> "http://localhost:21000/api/atlas/v2/search/saved/BASIC/my_search1;
> ```
> 
> ```javascript
> curl -g -X GET -u admin:admin -H "Content-Type: application/json" -H 
> "Cache-Control: no-cache"  
> "http://localhost:21000/api/atlas/v2/search/saved/ADVANCED/my_search3;
> ```
> 
> **Save**
> ```javascript
> curl -g -X POST -u admin:admin -H "Content-Type: application/json" -H 
> "Cache-Control: no-cache"  "http://localhost:21000/api/atlas/v2/search/saved; 
> -d @../docs/savedSearch.json
> ```
> 
> **Update**
> ```javascript
> curl -g -X PUT -u admin:admin -H "Content-Type: application/json" -H 
> "Cache-Control: no-cache"  "http://localhost:21000/api/atlas/v2/search/saved; 
> -d @../docs/savedSearch.json
> ```
> 
> **Delete**
> ```javascript
> curl -g -X DELETE -u admin:admin -H "Content-Type: application/json" -H 
> "Cache-Control: no-cache"  
> "http://localhost:21000/api/atlas/v2/search/saved/0d0f75ef-b101-466f-843e-60196b81c98e;
> ```
> 
> 
> Diffs
> -
> 
>   addons/models/0010-base_model.json 8f38f596 
>   intg/src/main/java/org/apache/atlas/model/profile/AtlasUserSavedSearch.java 
> b0698fcf 
>   
> repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java
>  baaee85d 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  ad21ee48 
>   
> repository/src/main/java/org/apache/atlas/repository/ogm/AtlasSavedSearchDTO.java
>  26eee200 
>   repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java 
> bc93cc6e 
>   
> repository/src/main/java/org/apache/atlas/repository/userprofile/UserProfileService.java
>  766d1c7d 
>   
> repository/src/test/java/org/apache/atlas/repository/userprofile/UserProfileServiceTest.java
>  4e832968 
>   webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java ad595c81 
> 
> 
> Diff: https://reviews.apache.org/r/62420/diff/3/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> Updated to reflect new logic.
> 
> **Functional tests**
> From the new UI.
> CURL calls.
> 
> 
> File Attachments
> 
> 
> saved Search
>   
> https://reviews.apache.org/media/uploaded/files/2017/09/19/f9f6ec2d-28f6-487a-8126-aaf6fc8a1028__savedSearch.json
> saved search for update
>   
> https://reviews.apache.org/media/uploaded/files/2017/09/19/2b1d4619-65d8-4435-87cb-9b11878a7420__savedSearch-for-update.json
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 62342: [ATLAS-2136]: Change audit and notification entityChangeListener to start using v2 API structure

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62342/#review185646
---




client/src/main/java/org/apache/atlas/EntityAuditEventV2.java
Lines 29 (patched)
<https://reviews.apache.org/r/62342/#comment261963>

If this class is going to be exposed via REST API, please move to 
org.apache.atlas.model package.



client/src/main/java/org/apache/atlas/EntityAuditEventV2.java
Lines 31 (patched)
<https://reviews.apache.org/r/62342/#comment261968>

Is it necessary to store entire entity here, given that 'details' will 
include all the information to be saved in audit?



notification/src/main/java/org/apache/atlas/notification/entity/EntityNotificationV2.java
Lines 91 (patched)
<https://reviews.apache.org/r/62342/#comment261967>

Please review for the following cases:
- on ENTITY_CREATE, the notification would have to include propagatedTags 
as well
- on ENTITY_DELETE, the notification should include the classifications & 
list of impacted entities (from whom the classification should be removed)
- how are create/delete of edges handled in notificaitons? They would 
impact the propagatedTags for other entities.



repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
Lines 129 (patched)
<https://reviews.apache.org/r/62342/#comment261972>

It might be efficient to create a single event that includes all 
classifications - instead of creating one event for each classfication.

Please review classificationsUpdated() and classificationsDeleted() as well.



repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
Lines 207 (patched)
<https://reviews.apache.org/r/62342/#comment261970>

This may not work, as clearAttributeValues() call above (line #203) would 
infact set all entries in 'attrValues' (line #201). Please review.



repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
Lines 187 (patched)
<https://reviews.apache.org/r/62342/#comment261974>

For large entities, wouldn't this cause failure in HBase save?



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityChangeNotifier.java
Line 94 (original), 102 (patched)
<https://reviews.apache.org/r/62342/#comment261975>

FullTextMapperV2.java uses RequestContext to get the entity from cache, if 
it exists. It will be efficient than loading entities from the store again. 
Please review FullTextMapperV2.getAndCacheEntity(guid)


- Madhan Neethiraj


On Sept. 19, 2017, 3 a.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62342/
> ---
> 
> (Updated Sept. 19, 2017, 3 a.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2136
> https://issues.apache.org/jira/browse/ATLAS-2136
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Currently audit and notification entityChangeListeners is using v1 API. This 
> needs to be migrated to start using v2 APIs. This is needed for notification 
> of tag propagation.
> Also provide a new REST API to get audit events for an entity using v2 
> structure
> 
> 
> Diffs
> -
> 
>   client/src/main/java/org/apache/atlas/EntityAuditEventV2.java PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java d205faf5 
>   
> notification/src/main/java/org/apache/atlas/notification/entity/EntityNotificationV2.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditRepository.java
>  9dc78350 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  5a5a2c10 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/InMemoryEntityAuditRepository.java
>  22d2a810 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/NoopEntityAuditRepository.java
>  c3826019 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java
>  5a0b74e3 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityChangeNotifier.java
>  7b349c46 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
>  a5db81bf 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  f6aa7bbf 
>   
> repository/src

Re: Atlas issues about 0.9 release

[Madhan Neethiraj]

Junheng, Ismaël,


> 1. The Atlas version 0.9 is in development phase, I can see the version
> 0.9-SNAPSHOT published on your website and then where can I get the 0.9
> package?
0.9 version is still under development in the master branch. Only after it is 
released, 0.9 libraries will be available. The latest release version is 0.8.1, 
from branch-0.8. 

>   And when will the 0.9 formal release be published?
Next release from master branch is likely to be in February/March 2018. Before 
this release from master branch, there might be another release from branch-0.8.


> 2. I see the 0.9 version will support Spark Integration, NiFi Integration
> and so on in “cwiki” page, but in the 0.9-incubating issues page, I don’t
> see these features.
>
>   So will the 0.9 version support them?
NiFi integration – likely to be available in the next release from master. It 
is not clear whether Spark integration will make it though.

>Extra question, what are the goals of the Atlas 0.9 release (what will
>be the main points)?
- we plan to move to Hadoop-3,  HBase-2, JanusGraph; probably higher version of 
Solr & Kafka
- features: relationship as a first-class type, tag-propagation, HBase hook, 
search improvements, Open Metadata and Governance items

>We wrote an integration of Atlas that uses the v1 of the API, so we
 >   would also like to know if the v1 API will still be available for 0.9.
Yes. V1 API will be available; but new functionalities like relationships, 
tag-propagation will only be available via V2 APIs.


Apologies for the delayed response to your questions.

Regards,
Madhan

On 8/16/17, 3:17 AM, "Ismaël Mejía"  wrote:

Extra question, what are the goals of the Atlas 0.9 release (what will
be the main points)?

We wrote an integration of Atlas that uses the v1 of the API, so we
would also like to know if the v1 API will still be available for 0.9.

Regards,
Ismaël

On Thu, Aug 10, 2017 at 8:56 AM, Junheng Cui(崔俊珩)  
wrote:
> Dear Atlas development team，
>
>
>
> I am a SW engineer working at Inspur China. Now I am using Atlas for data
> government. But I hava some questions, may I ask you about that?
>
>
>
> 1. The Atlas version 0.9 is in development phase, I can see the version
> 0.9-SNAPSHOT published on your website and then where can I get the 0.9
> package?
>
>   And when will the 0.9 formal release be published?
>
>
>
> 2. I see the 0.9 version will support Spark Integration, NiFi Integration
> and so on in “cwiki” page, but in the 0.9-incubating issues page, I don’t
> see these features.
>
>   So will the 0.9 version support them?
>
>
>
> The Atlas is a very useful program for data government, it helps my 
company
> so much.
>
> Thanks a lot!
>
>
>
>
>
> Best regards,
>
> Junheng Cui
>
>
>
> 
>
> Software Development Engineer of R Center of INSPUR SOFTWARE Co., Ltd
>
> Address:1036 Langchao Road, Jinan, Shandong Province, China
>
>

Re: Review Request 62455: ATLAS-2092: Upfront edgelabel creation to avoid concurrency issues

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62455/#review186034
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 22, 2017, 9:46 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62455/
> ---
> 
> (Updated Sept. 22, 2017, 9:46 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2092
> https://issues.apache.org/jira/browse/ATLAS-2092
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> See ATLAS-2092 for the discussion
> 
> 
> Diffs
> -
> 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasEdgeLabel.java
>  PRE-CREATION 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphManagement.java
>  0db46d46 
>   
> graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/GraphDbObjectFactory.java
>  0e8519e7 
>   
> graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0EdgeLabel.java
>  PRE-CREATION 
>   
> graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0GraphManagement.java
>  ec4d6c45 
>   
> graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/GraphDbObjectFactory.java
>  c27bc35e 
>   
> graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1EdgeLabel.java
>  PRE-CREATION 
>   
> graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1GraphManagement.java
>  61dc2980 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  be6a0a06 
>   repository/src/test/java/org/apache/atlas/TestUtils.java 1e439f7d 
>   
> repository/src/test/java/org/apache/atlas/discovery/DataSetLineageServiceTest.java
>  c700a61f 
>   
> repository/src/test/java/org/apache/atlas/lineage/EntityLineageServiceTest.java
>  8a88f322 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  5bde6059 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java
>  f372891e 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexerTest.java
>  ef5d4696 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
>  9b4cbd36 
>   
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
>  d8d8d91b 
>   repository/src/test/scala/org/apache/atlas/query/QueryTestsUtils.scala 
> c844558a 
> 
> 
> Diff: https://reviews.apache.org/r/62455/diff/4/
> 
> 
> Testing
> ---
> 
> mvn clean package -Dberkeley-elasticsearch went through fine
> Build was done 3-4 times to ensure that the concurrency tests passed every 
> time
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 62491: ATLAS-2151: Surrounding brace fix for multiple filters on same attribute

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62491/#review186042
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 23, 2017, 1:11 a.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62491/
> ---
> 
> (Updated Sept. 23, 2017, 1:11 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2151
> https://issues.apache.org/jira/browse/ATLAS-2151
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> While fixing the bugs surronding the index query generation, the case where 
> multiple filters on a single entity attribute are specified was not being 
> handled correctly. This would cause the index to return wrong results or a 
> smaller dataset. 
> 
> level == 0 check in the constructIndexQuery led to this problem.
> 
> The fix is to encapsulate the entity filters index clauses within braces if 
> there are more than one attribute filters.
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> 64a86b93 
> 
> 
> Diff: https://reviews.apache.org/r/62491/diff/2/
> 
> 
> Testing
> ---
> 
> mvn clean package -Pberkeley-elasticsearch works fine
> 
> Tested the REST call with following search parameters
> 
> ```javascript
> 
> {
>   "typeName": "hive_table",
>   "excludeDeletedEntities": true,
>   "classification" : "",
>   "query": "",
>   "limit": 10,
>   "offset": 0,
>   "entityFilters": {
>   "condition": "AND",
>   "criterion": [
>   {
>   "attributeName": "createTime",
>   "operator": "neq",
>   "attributeValue": "1491248917103"
>   },
>   {
> "attributeName": "name",
> "operator": "eq",
> "attributeValue": "testtable_0"
> }
>   ]
>   },
>   "tagFilters": null,
>   "attributes": ["createTime"]
> }
> 
> {
>   "typeName": "hive_table",
>   "excludeDeletedEntities": true,
>   "classification" : "",
>   "query": "",
>   "limit": 10,
>   "offset": 0,
>   "entityFilters": {
>   "condition": "OR",
>   "criterion": [
>   {
>   "attributeName": "name",
>   "operator": "eq",
>   "attributeValue": "testtable_1"
>   },
>   {
> "attributeName": "name",
> "operator": "eq",
> "attributeValue": "testtable_0"
> }
>   ]
>   },
>   "tagFilters": null,
>   "attributes": ["createTime"]
> }
> 
> {
>   "typeName": "hive_table",
>   "excludeDeletedEntities": true,
>   "classification" : "",
>   "query": "",
>   "limit": 10,
>   "offset": 0,
>   "entityFilters": {
>   "attributeName": "createTime",
>   "operator": "neq",
>   "attributeValue": "1491248917103"
>   },  
>   "tagFilters": null,
>   "attributes": ["createTime"]
> }
> ```
> 
> Verified from logs that the index query is enclosed within braces for 
> multi-filter attribute query and not when there's only one attribute (single) 
> filter
> 
> Verified the results from REST calls (as UI doesn't support this use case)
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 62455: ATLAS-2092: Upfront edgelabel creation to avoid concurrency issues

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62455/#review185832
---




repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
Line 445 (original), 445 (patched)
<https://reviews.apache.org/r/62455/#comment262185>

Edge label should be created for arrays whose element is of type CLASS or 
STRUCT.



repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
Line 451 (original), 451 (patched)
<https://reviews.apache.org/r/62455/#comment262186>

Edge label should be created for struct type attributes as well.


- Madhan Neethiraj


On Sept. 20, 2017, 9:55 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62455/
> ---
> 
> (Updated Sept. 20, 2017, 9:55 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2092
> https://issues.apache.org/jira/browse/ATLAS-2092
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> See ATLAS-2092 for the discussion
> 
> 
> Diffs
> -
> 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphManagement.java
>  0db46d46 
>   
> graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0GraphManagement.java
>  ec4d6c45 
>   
> graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1GraphManagement.java
>  61dc2980 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  be6a0a06 
> 
> 
> Diff: https://reviews.apache.org/r/62455/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean package -Dberkeley-elasticsearch went through fine
> Build was done 3-4 times to ensure that the concurrency tests passed every 
> time
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 62366: ATLAS-2140 - removing unnecessary exceptions

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62366/#review185525
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 16, 2017, 2:49 p.m., Zsombor Gegesy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62366/
> ---
> 
> (Updated Sept. 16, 2017, 2:49 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-2140
> https://issues.apache.org/jira/browse/ATLAS-2140
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> In various places in the codebase exceptions are declared unnecessarily 
> broadly or without actually throwing it. Which could cause confusion in the 
> caller, making it harder to handling it, and recovering from this.
> 
> 
> Diffs
> -
> 
>   
> addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/bridge/FalconBridge.java
>  349da66 
>   
> addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/hook/FalconHook.java
>  3b384f8 
>   
> addons/falcon-bridge/src/main/java/org/apache/atlas/falcon/publisher/FalconEventPublisher.java
>  ea81226 
>   
> addons/storm-bridge/src/main/java/org/apache/atlas/storm/hook/StormAtlasHook.java
>  5193832 
>   
> addons/storm-bridge/src/main/java/org/apache/atlas/storm/hook/StormTopologyUtil.java
>  d999a6a 
>   
> plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader/AtlasPluginClassLoader.java
>  d7ec65d 
>   
> plugin-classloader/src/main/java/org/apache/atlas/plugin/classloader/AtlasPluginClassLoaderUtil.java
>  dfdad88 
> 
> 
> Diff: https://reviews.apache.org/r/62366/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Zsombor Gegesy
> 
>

Re: Review Request 62129: ATLAS-2117: Basic search issues due to Titan Solr schema

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62129/#review185373
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 14, 2017, 5:15 a.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62129/
> ---
> 
> (Updated Sept. 14, 2017, 5:15 a.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2117
> https://issues.apache.org/jira/browse/ATLAS-2117
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> See JIRA
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> ddf117fa 
> 
> 
> Diff: https://reviews.apache.org/r/62129/diff/5/
> 
> 
> Testing
> ---
> 
> UI and REST calls working as expected now.
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 62129: ATLAS-2117: Basic search issues due to Titan Solr schema

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62129/#review185363
---




repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java
Lines 232 (patched)
<https://reviews.apache.org/r/62129/#comment261669>

Instead of completely rejecting index-query on detecting NEQ on a string 
attribute, consider treating this as a non-indexed attribute (line #240) - so 
that index-query can handle other attributes and this attribute can be handled 
by graph query or in-memory query.


- Madhan Neethiraj


On Sept. 14, 2017, 2:32 a.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62129/
> ---
> 
> (Updated Sept. 14, 2017, 2:32 a.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2117
> https://issues.apache.org/jira/browse/ATLAS-2117
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> See JIRA
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> ddf117fa 
> 
> 
> Diff: https://reviews.apache.org/r/62129/diff/4/
> 
> 
> Testing
> ---
> 
> UI and REST calls working as expected now.
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Draft Board Report for Apache Atlas: September-2017

[Madhan Neethiraj]

Atlas team,

 

Please review the draft board report below and send your feedback/comments,

if any.

 

Thanks,

Madhan

 

 

 

## Description:

  Apache Atlas is a scalable and extensible set of core foundational governance

  services that enables enterprises to effectively and efficiently meet their

  compliance requirements within Hadoop and allows integration with the

  complete enterprise data ecosystem

 

## Issues:

  There are no issues requiring board attention at this time.

 

## Activity:

  - Released 0.8.1, the first release after graduation

  - Completed backend work on relationships as first class type

  - Working on new features like tag-propagation, business catalog,

    Open Metadata & Governance, Virtual Data Connector,

    Automated Discovery

 

## Health report:

  - Added 1 new committer in last month

  - No new contributors added in last month

 

## PMC changes:

  - Currently 33 PMC members

  - No new PMC members added in last month

  - Last PMC member addition was on 6/21/2017

 

## Committer base changes:

  - Currently 36 committers

  - 1 new committer added in last month, on 8/15/2017

  - Last addition to committer role was Mandy Chessell

 

## Releases:

 0.8.1    was released on 08/29/2017

 0.8-incubating   was released on 03/16/2017

 0.7.1-incubating was released on 01/26/2017

 0.7-incubating   was released on 07/09/2016

 0.6-incubating   was released on 12/31/2015

 0.5-incubating   was released on 07/11/2015

 

 

 

 

Re: Add me as a JIRA contributor in Apache Atlas

[Madhan Neethiraj]

Tejas,

Done. Welcome to Apache Atlas community!

Madhan



On 9/21/17, 12:13 AM, "Tejas Rawool"  wrote:

Hi Atlas dev,

Please add me as a JIRA contributor for Atlas

My Userid is "tejas_rawool".

Thanks

Re: Review Request 62491: ATLAS-2151: Surrounding brace fix for multiple filters on same attribute

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62491/#review185942
---




repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java
Line 353 (original), 354 (patched)
<https://reviews.apache.org/r/62491/#comment262282>

For outer most level, braces are not needed when conditon is AND. Consider 
updating this line to:

boolean needSurroundingBraces = level != 0 || (condition == Condition.OR && 
criteria.getCriterion().size() > 0);


- Madhan Neethiraj


On Sept. 22, 2017, 4:35 a.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62491/
> ---
> 
> (Updated Sept. 22, 2017, 4:35 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2151
> https://issues.apache.org/jira/browse/ATLAS-2151
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> While fixing the bugs surronding the index query generation, the case where 
> multiple filters on a single entity attribute are specified was not being 
> handled correctly. This would cause the index to return wrong results or a 
> smaller dataset. 
> 
> level == 0 check in the constructIndexQuery led to this problem.
> 
> The fix is to encapsulate the entity filters index clauses within braces if 
> there are more than one attribute filters.
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> d5e39236 
> 
> 
> Diff: https://reviews.apache.org/r/62491/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean package -Pberkeley-elasticsearch works fine
> 
> Tested the REST call with following search parameters
> 
> ```javascript
> 
> {
>   "typeName": "hive_table",
>   "excludeDeletedEntities": true,
>   "classification" : "",
>   "query": "",
>   "limit": 10,
>   "offset": 0,
>   "entityFilters": {
>   "condition": "AND",
>   "criterion": [
>   {
>   "attributeName": "createTime",
>   "operator": "neq",
>   "attributeValue": "1491248917103"
>   },
>   {
> "attributeName": "name",
> "operator": "eq",
> "attributeValue": "testtable_0"
> }
>   ]
>   },
>   "tagFilters": null,
>   "attributes": ["createTime"]
> }
> 
> {
>   "typeName": "hive_table",
>   "excludeDeletedEntities": true,
>   "classification" : "",
>   "query": "",
>   "limit": 10,
>   "offset": 0,
>   "entityFilters": {
>   "condition": "OR",
>   "criterion": [
>   {
>   "attributeName": "name",
>   "operator": "eq",
>   "attributeValue": "testtable_1"
>   },
>   {
> "attributeName": "name",
> "operator": "eq",
> "attributeValue": "testtable_0"
> }
>   ]
>   },
>   "tagFilters": null,
>   "attributes": ["createTime"]
> }
> 
> {
>   "typeName": "hive_table",
>   "excludeDeletedEntities": true,
>   "classification" : "",
>   "query": "",
>   "limit": 10,
>   "offset": 0,
>   "entityFilters": {
>   "attributeName": "createTime",
>   "operator": "neq",
>   "attributeValue": "1491248917103"
>   },  
>   "tagFilters": null,
>   "attributes": ["createTime"]
> }
> ```
> 
> Verified from logs that the index query is enclosed within braces for 
> multi-filter attribute query and not when there's only one attribute (single) 
> filter
> 
> Verified the results from REST calls (as UI doesn't support this use case)
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 62455: ATLAS-2092: Upfront edgelabel creation to avoid concurrency issues

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62455/#review185996
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 22, 2017, 3:44 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62455/
> ---
> 
> (Updated Sept. 22, 2017, 3:44 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2092
> https://issues.apache.org/jira/browse/ATLAS-2092
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> See ATLAS-2092 for the discussion
> 
> 
> Diffs
> -
> 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphManagement.java
>  0db46d46 
>   
> graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0GraphManagement.java
>  ec4d6c45 
>   
> graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1GraphManagement.java
>  61dc2980 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  be6a0a06 
>   
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
>  d8d8d91b 
> 
> 
> Diff: https://reviews.apache.org/r/62455/diff/3/
> 
> 
> Testing
> ---
> 
> mvn clean package -Dberkeley-elasticsearch went through fine
> Build was done 3-4 times to ensure that the concurrency tests passed every 
> time
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 62457: ATLAS-2157: HiveHook error while processing temporary tables

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62457/#review185918
---




addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java
Line 552 (original), 550 (patched)
<https://reviews.apache.org/r/62457/#comment262254>

This seems to be necessary only for 'case DATABASE' above. I will move this 
block to line #533.


- Madhan Neethiraj


On Sept. 21, 2017, 5:39 p.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62457/
> ---
> 
> (Updated Sept. 21, 2017, 5:39 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Sarath Subramanian, 
> and Suma Shivaprasad.
> 
> 
> Bugs: ATLAS-2157
> https://issues.apache.org/jira/browse/ATLAS-2157
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> As reported in user list, executing Hive SQL that use temprorary tables (for 
> example insert statement) causes Atlas hook to report the following failure. 
> The fix is to ignore this error, as it doesn't result in loss of 
> functionality.
> 
> 
> 17/09/20 09:36:42 [Atlas Logger 1]: ERROR metadata.Hive: Table 
> values__tmp__table__1 not found: default.values__tmp__table__1 table not found
> 17/09/20 09:36:43 [Atlas Logger 1]: ERROR hook.HiveHook: Atlas hook failed 
> due to error
> java.lang.reflect.UndeclaredThrowableException
>         at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1884)
>         at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:195)
>         at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.atlas.hook.AtlasHookException: 
> HiveHook.registerProcess() failed.
>         at 
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:701)
>         at org.apache.atlas.hive.hook.HiveHook.collect(HiveHook.java:268)
>         at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:83)
>         at org.apache.atlas.hive.hook.HiveHook$2$1.run(HiveHook.java:198)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:422)
>         at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
>         ... 6 more
> Caused by: org.apache.atlas.hook.AtlasHookException: 
> HiveHook.processHiveEntity() failed.
>         at 
> org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:731)
>         at 
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:668)
>         ... 12 more
> Caused by: org.apache.atlas.hook.AtlasHookException: 
> HiveHook.createOrUpdateEntities() failed.
>         at 
> org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:597)
>         at 
> org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:711)
>         ... 13 more
> Caused by: org.apache.atlas.hook.AtlasHookException: 
> HiveHook.createOrUpdateEntities() failed.
>         at 
> org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:589)
>         at 
> org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:595)
>         ... 14 more
> Caused by: org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table 
> not found values__tmp__table__1
>         at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1216)
>         at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1186)
>         at 
> org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:568)
>         ... 15 more
> 
> 
> Diffs
> -
> 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
> 7dc2e2f1 
> 
> 
> Diff: https://reviews.apache.org/r/62457/diff/1/
> 
> 
> Testing
> ---
> 
> Verified that the error is not reported in the following cases:
> - CREATE TEMPORARY TABLE tmp1(c1 string);
> - insert into tmp1 values('1');
> 
> Also verified that lineage details are recorded correctly - for both table 
> and column level lineages:
> - CREATE TABLE t1(c1 STRING, c2 STRING);
> - CREATE TABLE s1(c1 STRING, c2 STRING);
> - CREATE VIEW v_t1_s1 AS SELECT (t.c1 + s.c1) c1, (t.c2 + s.c2) c2 FROM t1 t, 
> s1 s;
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Review Request 62457: ATLAS-2157: HiveHook error while processing temporary tables

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62457/
---

(Updated Sept. 21, 2017, 7:47 p.m.)


Review request for atlas, Apoorv Naik, Ashutosh Mestry, Sarath Subramanian, and 
Suma Shivaprasad.


Changes
---

Addressed review comments.


Bugs: ATLAS-2157
https://issues.apache.org/jira/browse/ATLAS-2157


Repository: atlas


Description
---

As reported in user list, executing Hive SQL that use temprorary tables (for 
example insert statement) causes Atlas hook to report the following failure. 
The fix is to ignore this error, as it doesn't result in loss of functionality.


17/09/20 09:36:42 [Atlas Logger 1]: ERROR metadata.Hive: Table 
values__tmp__table__1 not found: default.values__tmp__table__1 table not found
17/09/20 09:36:43 [Atlas Logger 1]: ERROR hook.HiveHook: Atlas hook failed due 
to error
java.lang.reflect.UndeclaredThrowableException
        at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1884)
        at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:195)
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.registerProcess() 
failed.
        at 
org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:701)
        at org.apache.atlas.hive.hook.HiveHook.collect(HiveHook.java:268)
        at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:83)
        at org.apache.atlas.hive.hook.HiveHook$2$1.run(HiveHook.java:198)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
        ... 6 more
Caused by: org.apache.atlas.hook.AtlasHookException: 
HiveHook.processHiveEntity() failed.
        at 
org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:731)
        at 
org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:668)
        ... 12 more
Caused by: org.apache.atlas.hook.AtlasHookException: 
HiveHook.createOrUpdateEntities() failed.
        at 
org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:597)
        at 
org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:711)
        ... 13 more
Caused by: org.apache.atlas.hook.AtlasHookException: 
HiveHook.createOrUpdateEntities() failed.
        at 
org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:589)
        at 
org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:595)
        ... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not 
found values__tmp__table__1
        at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1216)
        at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1186)
        at 
org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:568)
        ... 15 more


Diffs (updated)
-

  addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
7dc2e2f1 


Diff: https://reviews.apache.org/r/62457/diff/2/

Changes: https://reviews.apache.org/r/62457/diff/1-2/


Testing
---

Verified that the error is not reported in the following cases:
- CREATE TEMPORARY TABLE tmp1(c1 string);
- insert into tmp1 values('1');

Also verified that lineage details are recorded correctly - for both table and 
column level lineages:
- CREATE TABLE t1(c1 STRING, c2 STRING);
- CREATE TABLE s1(c1 STRING, c2 STRING);
- CREATE VIEW v_t1_s1 AS SELECT (t.c1 + s.c1) c1, (t.c2 + s.c2) c2 FROM t1 t, 
s1 s;


Thanks,

Madhan Neethiraj

Re: Review Request 62457: ATLAS-2157: HiveHook error while processing temporary tables

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62457/
---

(Updated Sept. 21, 2017, 5:39 p.m.)


Review request for atlas, Apoorv Naik, Ashutosh Mestry, Sarath Subramanian, and 
Suma Shivaprasad.


Summary (updated)
-

ATLAS-2157: HiveHook error while processing temporary tables 


Bugs: ATLAS-2157
https://issues.apache.org/jira/browse/ATLAS-2157


Repository: atlas


Description
---

As reported in user list, executing Hive SQL that use temprorary tables (for 
example insert statement) causes Atlas hook to report the following failure. 
The fix is to ignore this error, as it doesn't result in loss of functionality.


17/09/20 09:36:42 [Atlas Logger 1]: ERROR metadata.Hive: Table 
values__tmp__table__1 not found: default.values__tmp__table__1 table not found
17/09/20 09:36:43 [Atlas Logger 1]: ERROR hook.HiveHook: Atlas hook failed due 
to error
java.lang.reflect.UndeclaredThrowableException
        at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1884)
        at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:195)
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.registerProcess() 
failed.
        at 
org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:701)
        at org.apache.atlas.hive.hook.HiveHook.collect(HiveHook.java:268)
        at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:83)
        at org.apache.atlas.hive.hook.HiveHook$2$1.run(HiveHook.java:198)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
        ... 6 more
Caused by: org.apache.atlas.hook.AtlasHookException: 
HiveHook.processHiveEntity() failed.
        at 
org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:731)
        at 
org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:668)
        ... 12 more
Caused by: org.apache.atlas.hook.AtlasHookException: 
HiveHook.createOrUpdateEntities() failed.
        at 
org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:597)
        at 
org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:711)
        ... 13 more
Caused by: org.apache.atlas.hook.AtlasHookException: 
HiveHook.createOrUpdateEntities() failed.
        at 
org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:589)
        at 
org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:595)
        ... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not 
found values__tmp__table__1
        at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1216)
        at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1186)
        at 
org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:568)
        ... 15 more


Diffs
-

  addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
7dc2e2f1 


Diff: https://reviews.apache.org/r/62457/diff/1/


Testing
---

Verified that the error is not reported in the following cases:
- CREATE TEMPORARY TABLE tmp1(c1 string);
- insert into tmp1 values('1');

Also verified that lineage details are recorded correctly - for both table and 
column level lineages:
- CREATE TABLE t1(c1 STRING, c2 STRING);
- CREATE TABLE s1(c1 STRING, c2 STRING);
- CREATE VIEW v_t1_s1 AS SELECT (t.c1 + s.c1) c1, (t.c2 + s.c2) c2 FROM t1 t, 
s1 s;


Thanks,

Madhan Neethiraj

Re: Review Request 62416: ATLAS-2121: Case sensitivity for tag/type name search with index

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62416/#review185770
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 19, 2017, 7:15 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62416/
> ---
> 
> (Updated Sept. 19, 2017, 7:15 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2121
> https://issues.apache.org/jira/browse/ATLAS-2121
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Currently the index query returns case insensitive results, this patch add 
> the typename / tag name check to the in memory predicate to honor case 
> sensitivity.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  1d4815df 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
>  970cd28e 
> 
> 
> Diff: https://reviews.apache.org/r/62416/diff/2/
> 
> 
> Testing
> ---
> 
> UI and REST calls work as expected.
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 62747: ATLAS-2163 : Save Search Query : Saved search query having deleted types

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62747/#review186986
---


Ship it!




Ship It!

- Madhan Neethiraj


On Oct. 3, 2017, 11:03 a.m., keval bhatt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62747/
> ---
> 
> (Updated Oct. 3, 2017, 11:03 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Madhan Neethiraj, 
> Nixon Rodrigues, Sarath Subramanian, and Sharmadha Sainath.
> 
> 
> Bugs: ATLAS-2163
> https://issues.apache.org/jira/browse/ATLAS-2163
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> 1. Created a tag tag1.
> 2. Fired basic search with classification name = tag1 and saved search as 
> "tag_basic_search"
> 3. Fired DSL search with query = tag1 and saved search as "tag_DSL_search"
> 4. Deleted the tag.
> 5. Selected "tag_DSL_search" - this populated query text box with tag name . 
> Search threw invalid expression - which is expected.
> 6. With Basic search ,on selecting "tag_basic_search" , no information is 
> populated in classification name. Classification name is null in the Basic 
> POST request body.
> 7. Atlas displaying an error notification when selecting the saved search, 
> saying tag name invalid would be user friendly in case of deleted 
> tags/invalid tags . It is not very explicit to the user that the tag is 
> deleted.
> 
> 
> Diffs
> -
> 
>   dashboardv2/public/css/scss/theme.scss 7196702 
>   dashboardv2/public/js/router/Router.js 59d22c9 
>   dashboardv2/public/js/utils/Messages.js feccdc0 
>   dashboardv2/public/js/views/audit/AuditTableLayoutView.js e8483e4 
>   dashboardv2/public/js/views/audit/CreateAuditTableLayoutView.js ebc0021 
>   dashboardv2/public/js/views/search/SearchResultLayoutView.js e8151fd 
>   dashboardv2/public/js/views/search/save/SaveSearchItemView.js 07d8df7 
> 
> 
> Diff: https://reviews.apache.org/r/62747/diff/1/
> 
> 
> Testing
> ---
> 
> If user search for tag/type in basic search using saved search but that 
> type/tag is deleted from backend then UI will show error message. "Invalid 
> Expression or Tag/Type has been deleted".
> 
> 
> Thanks,
> 
> keval bhatt
> 
>

Re: Review Request 62746: ATLAS-2185 : UI to handle is_null, not_null operator for attribute search.

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62746/#review186985
---


Ship it!




Ship It!

- Madhan Neethiraj


On Oct. 3, 2017, 10:56 a.m., keval bhatt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62746/
> ---
> 
> (Updated Oct. 3, 2017, 10:56 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, Nixon Rodrigues, and 
> Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2185
> https://issues.apache.org/jira/browse/ATLAS-2185
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> In Basic search attribute filter allow user to select is_null and not_null 
> operator for all types.
> 
> 
> Diffs
> -
> 
>   dashboardv2/public/js/views/search/QueryBuilderView.js a6a1e3a 
> 
> 
> Diff: https://reviews.apache.org/r/62746/diff/1/
> 
> 
> Testing
> ---
> 
> Tested with API, operators is going correctly.
> 
> 
> Thanks,
> 
> keval bhatt
> 
>

Re: Review Request 62758: ATLAS-2184: IsNull/NotNull bugfixes

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62758/#review187010
---


Ship it!




Ship It!

- Madhan Neethiraj


On Oct. 3, 2017, 9:41 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62758/
> ---
> 
> (Updated Oct. 3, 2017, 9:41 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2184
> https://issues.apache.org/jira/browse/ATLAS-2184
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> handling case when the attrVal is empty (null is already being handled)
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> ba0cd8ae 
> 
> 
> Diff: https://reviews.apache.org/r/62758/diff/1/
> 
> 
> Testing
> ---
> 
> UI/REST calls works as expected, no 500s encountered
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

[ANNOUNCE] Java 8 as minimum requirement for Apache Atlas 1.0.0 (master)

[Madhan Neethiraj]

All,

 

Atlas 1.0.0 (from master branch) will be replacing use of Titan with 
JanusGraph, as Titan is not being actively maintained/enhanced, while 
JanusGraph is. Also, this branch will be using newer version of Hadoop (3.0) 
and HBase (2.0). All these updates require Java 8 and won’t work in earlier 
version of Java. And recent change to use newer version of Jetty (ATLAS-2089) 
also requires use of Java 8.

 

Given above, Apache Atlas (from master branch) would require Java 8 from now on.

 

Apache Atlas from branch-0.8 will continue to use Java 7, as there is no plan 
currently to move to JanusGraph/Hadoop-3/.. in this branch.

 

Madhan

 

 

Re: Review Request 62838: ATLAS-2171 : Error in constructing base path for UI API when accessing Atlas via knox proxy without "/" at the end

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62838/#review187426
---


Ship it!




Ship It!

- Madhan Neethiraj


On Oct. 9, 2017, 2:04 p.m., keval bhatt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62838/
> ---
> 
> (Updated Oct. 9, 2017, 2:04 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sharmadha 
> Sainath.
> 
> 
> Bugs: ATLAS-2171
> https://issues.apache.org/jira/browse/ATLAS-2171
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> 1. Enabled knox proxy for atlas.
> 2.Accessed Atlas UI via proxy using https://knox_gateway:8443/gateway/ui/atlas
> 3. Provided admin username and password and logged into Atlas
> 4. Now, opened another tab in browser with URL: 
> https://knox_gateway:8443/gateway/ui/atlas
> 
> UI is loaded without data because API path is not formed properly.
> 
> If the URL is accessed with "/" at the end, logs in as admin user and all 
> operations work fine.
> Example : https://knox_gateway:8443/gateway/ui/atlas/
> 
> 
> Fix detail.
> 
> Added one regex to form correct API base url.
> 
> Regex condition.
> 
> it is checking if url exnds with slash then exclude it or if url ends with 
> file extention i.e (.html or .jsp) then exclude that file extention with 
> slash.
> 
> Example of regex input = output:
> 
> 1. https://knox_gateway:8443/gateway/ui/atlas/ = 
> https://knox_gateway:8443/gateway/ui/atlas
> 2. https://knox_gateway:8443/gateway/ui/atlas/index.html = 
> https://knox_gateway:8443/gateway/ui/atlas
> 3. https://knox_gateway:8443/gateway/ui/atlas/index.jsp = 
> https://knox_gateway:8443/gateway/ui/atlas
> 4. https://knox_gateway:8443/gateway/ui/atlas = 
> https://knox_gateway:8443/gateway/ui/atlas
> 
> 
> Diffs
> -
> 
>   dashboardv2/public/css/scss/loader.scss d323212 
>   dashboardv2/public/index.html.tpl f13f34a 
>   
> dashboardv2/public/js/templates/business_catalog/SideNavLayoutView_tmpl.html 
> 3f4c11a 
>   dashboardv2/public/js/utils/UrlLinks.js 913df24 
>   dashboardv2/public/js/utils/Utils.js 1de2106 
>   dashboardv2/public/js/views/business_catalog/SideNavLayoutView.js 1a2590c 
>   dashboardv2/public/js/views/graph/LineageLayoutView.js 46f7117 
> 
> 
> Diff: https://reviews.apache.org/r/62838/diff/1/
> 
> 
> Testing
> ---
> 
> Tested with normal url and also proxy url its working as expected.
> 
> 
> Thanks,
> 
> keval bhatt
> 
>

Re: Review Request 62405: ATLAS2031 Allow AtlasClassificationDef entitytypes removal

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62405/#review187582
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 21, 2017, 8:45 a.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62405/
> ---
> 
> (Updated Sept. 21, 2017, 8:45 a.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS2031 Allow AtlasClassificationDef entitytypes removal
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.java
>  603516d49f1e6ce2acd8cb1b45bd89d51cff7e94 
> 
> 
> Diff: https://reviews.apache.org/r/62405/diff/2/
> 
> 
> Testing
> ---
> 
> 1) create a classificationDef with 2 valid entityTypes EntityA and EntityB
> 2) update classification def to remove EntityTypeB
> 3) create an instance of EntityC , try to apply the classificaiton to it . It 
> should fail
> 4) update the classification to have an empty list of classifications.
> 5) Repeat step 3 and it should now work.
> 
> 
> File Attachments
> 
> 
> ATLAS2031.patch
>   
> https://reviews.apache.org/media/uploaded/files/2017/09/21/28a75f1f-3b35-4ffb-a915-010ebf265a6b__ATLAS2031.patch
> 
> 
> Thanks,
> 
> David Radley
> 
>

Re: Review Request 63029: ATLAS-2212 : Regression : Unable to provide negative values directly for tag attributes of data type int.

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63029/#review188140
---


Ship it!




Ship It!

- Madhan Neethiraj


On Oct. 16, 2017, 1:29 p.m., pratik pandey wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63029/
> ---
> 
> (Updated Oct. 16, 2017, 1:29 p.m.)
> 
> 
> Review request for atlas, keval bhatt, Madhan Neethiraj, Nixon Rodrigues, and 
> Sharmadha Sainath.
> 
> 
> Bugs: ATLAS-2212
> https://issues.apache.org/jira/browse/ATLAS-2212
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> 1. Created a tag with int attribute.
> 2. Tried to associate the tag to an entity with tag attribute value as -1. 
> Unable to provide negative value.(i.e) minus sign is not accepted. 
> 3. Able to set negative value for int if value is changed using Up and Down 
> Arrow .
> 4. This behavior is not observed for short,byte,float,double attribute 
> values. Able to provide negative value directly for datatypes other than int.
> 
> 
> Diffs
> -
> 
>   dashboardv2/public/js/utils/Overrides.js 4ffda57 
>   dashboardv2/public/js/views/tag/addTagModalView.js 423a92e 
> 
> 
> Diff: https://reviews.apache.org/r/63029/diff/1/
> 
> 
> Testing
> ---
> 
> I have tested with negative value is working as expected.
> 
> 
> Thanks,
> 
> pratik pandey
> 
>

Review Request 63037: ATLAS-2213: Remove unused attributes from Hive hook

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63037/
---

Review request for atlas.


Bugs: ATLAS-2213
https://issues.apache.org/jira/browse/ATLAS-2213


Repository: atlas


Description
---

Removed API calls to retrieve queryType value, as this field is not unsed in 
Atlas Hive entities.


Diffs
-

  addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
f815773e 


Diff: https://reviews.apache.org/r/63037/diff/1/


Testing
---

Verified that UTs pass successfully.


Thanks,

Madhan Neethiraj

Re: Review Request 62769: ATLAS-2166 - Added validation to prevent kerberos authentication when knox-proxy adds hadoop-auth header to proxied request

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62769/#review187975
---


Ship it!




Ship It!

- Madhan Neethiraj


On Oct. 13, 2017, 12:25 p.m., Nixon Rodrigues wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62769/
> ---
> 
> (Updated Oct. 13, 2017, 12:25 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Madhan Neethiraj, and 
> Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2166
> https://issues.apache.org/jira/browse/ATLAS-2166
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Bug description:-
> On refreshing Atlas page logged in via Knox proxy ,which has ATLASSESSION ID 
> expired (idle for a long time) , logs in as knox user.
> 
> Fix Description :-
> 
> ATLAS-2166 - Added validation to prevent kerberos authentication when 
> knox-proxy adds hadoop-auth header to proxied request
> 
> 
> Diffs
> -
> 
>   
> webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationFilter.java
>  444b094 
> 
> 
> Diff: https://reviews.apache.org/r/62769/diff/3/
> 
> 
> Testing
> ---
> 
> Tested Atlas UI/API  with Atlas and knox Kerberized Env with & without proxy 
> and also with SSO on/off.
> Tested curl with call with --negotiate headers.
> Tested curl with call with hadoop-jwt knox cookie header.
> 
> 
> Thanks,
> 
> Nixon Rodrigues
> 
>

Re: Review Request 62957: ATLAS-2195: Addressing Txn Leak During Startup

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62957/#review188057
---


Ship it!




Ship It!

- Madhan Neethiraj


On Oct. 14, 2017, 1:34 a.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62957/
> ---
> 
> (Updated Oct. 14, 2017, 1:34 a.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2195
> https://issues.apache.org/jira/browse/ATLAS-2195
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Background**
> If entities and types are created simultaneously it was observed that some 
> stale transactions are created. Titan runs an eviction thread every minute to 
> evict these out of transaction cache. That is when the message mentioned in 
> the bug appears.
> 
> **Analysis**
> During startup when built-in types are added to the database, the process did 
> not use _GraphTransactionInterceptor_ approach. This caused stale 
> transactions to appear, which grew over time.
> 
> ManagementSystem class in Titan creates a transaction object in its ctor. The 
> user is expected to close this when done with using it.
> 
> **Analysis for Part 2**
> 
> In our _GraphBackedSearchIndexer.getVertexIndexKeys_ we use this to get index 
> keys. This is invoked for every basic search.
>  
> Interestingly, new object is not created for each basic search. Only 1 
> transaction stays open if user just keeps searching.
>  
> Behavior changes when new tag is created. The count of open transactions 
> keeps incrementing.
> 
> **Instrumentation**
> Used _Titan's_ _StandardTitanGraph_ for adding instrumentation code. This is 
> central point for transaction operations.
> 
> Tracing the creation and co-relating them with destruction helped in 
> narrowing down the area of focus.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  8aa22377 
> 
> 
> Diff: https://reviews.apache.org/r/62957/diff/2/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> Existing.
> 
> **Other**
> Executed the attached bash scripts to create numerous types and entities 
> simultaneously.
> 
> Use this command:
> ```
> bash create-type.sh & bash create-entity.sh &
> ```
> 
> 
> File Attachments
> 
> 
> create type
>   
> https://reviews.apache.org/media/uploaded/files/2017/10/12/755455cc-c08c-4885-9d07-5b5d9e3867e2__create-type.sh
> create entity
>   
> https://reviews.apache.org/media/uploaded/files/2017/10/12/b20ddb13-7300-493d-8f43-5fcaf884c055__create-entity.sh
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 62175: [ATLAS-2119]: Implement update and delete operations for RelationshipREST

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62175/#review188043
---




repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
Line 139 (original), 172 (patched)
<https://reviews.apache.org/r/62175/#comment265117>

Delete of relationships should send out notifications for changes in 
propagated tags - if any.


- Madhan Neethiraj


On Sept. 7, 2017, 10:16 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62175/
> ---
> 
> (Updated Sept. 7, 2017, 10:16 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-2119
> https://issues.apache.org/jira/browse/ATLAS-2119
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> * Implement update and delete relationship in RelationshipREST
> * Add propagateTags information to each relationship instance
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java bf098062 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 
> ec6161c0 
>   notification/pom.xml 9b369403 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 639077dd 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
>  1eb41832 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
>  9b273193 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  b0940f67 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  36cd980f 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
>  94cc5b93 
> 
> 
> Diff: https://reviews.apache.org/r/62175/diff/4/
> 
> 
> Testing
> ---
> 
> validated using POSTMAN Rest client.
> 
> Integration/Unit Tests in progress
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 62175: [ATLAS-2119]: Implement update and delete operations for RelationshipREST

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62175/#review188042
---




notification/pom.xml
Line 1 (original), 1 (patched)
<https://reviews.apache.org/r/62175/#comment265119>

This file has only whitespace changes; please revert.



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
Lines 251 (patched)
<https://reviews.apache.org/r/62175/#comment265118>

tagPropagation is not set in createRelationship() above. Please review.



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
Lines 266 (patched)
<https://reviews.apache.org/r/62175/#comment265120>

Update of relationships should send out notifications for changes in 
propagated tags - if any.



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
Lines 431 (patched)
<https://reviews.apache.org/r/62175/#comment265121>

Why not use tagPropagation setting from the relationShipType?


- Madhan Neethiraj


On Sept. 7, 2017, 10:16 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62175/
> ---
> 
> (Updated Sept. 7, 2017, 10:16 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-2119
> https://issues.apache.org/jira/browse/ATLAS-2119
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> * Implement update and delete relationship in RelationshipREST
> * Add propagateTags information to each relationship instance
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java bf098062 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 
> ec6161c0 
>   notification/pom.xml 9b369403 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 639077dd 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
>  1eb41832 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
>  9b273193 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  b0940f67 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  36cd980f 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
>  94cc5b93 
> 
> 
> Diff: https://reviews.apache.org/r/62175/diff/4/
> 
> 
> Testing
> ---
> 
> validated using POSTMAN Rest client.
> 
> Integration/Unit Tests in progress
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 61200: ATLAS-1996: Log4J Default Configuration Updated

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61200/#review184983
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 8, 2017, 4:28 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61200/
> ---
> 
> (Updated Sept. 8, 2017, 4:28 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-1996
> https://issues.apache.org/jira/browse/ATLAS-1996
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Problem**
> Existing log4J configuration did not include max size parameter. This caused 
> logs to grow making it hard for human log readers.
> 
> Another contributing factor to log size growth is TitanDB's 
> _IndexSerializer_. When this is used, the query conversion from graph query 
> to indexed query is written as info log entry.
> 
> With the recent changes, many more index queries are used, thus contributing 
> to growth of log files.
> 
> **Solution**
> 
> Added maxFileSize parameter.
> Changed _com.thinkaurelius.titan_ log level to _warn_.
> 
> **Additional Info**
> Defaults need to be set in _Ambari_ so that they reflect correctly as part of 
> deployment.
> 
> 
> Diffs
> -
> 
>   addons/hive-bridge/src/main/resources/atlas-hive-import-log4j.xml c1184e5 
>   distro/src/conf/atlas-log4j.xml a0ef6a9 
>   typesystem/src/main/resources/atlas-log4j.xml c30403e 
> 
> 
> Diff: https://reviews.apache.org/r/61200/diff/3/
> 
> 
> Testing
> ---
> 
> **Functional tests**
> Observed configuration changes in the logs generated.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 61997: [ATLAS-2099]: Change branch-0.8 pom version to 0.8.2-SNAPSHOT

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61997/#review184192
---


Ship it!




Ship It!

- Madhan Neethiraj


On Aug. 30, 2017, 6:39 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61997/
> ---
> 
> (Updated Aug. 30, 2017, 6:39 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-2099
> https://issues.apache.org/jira/browse/ATLAS-2099
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Change branch-0.8 pom version to 0.8.2-SNAPSHOT
> 
> 
> Diffs
> -
> 
>   addons/falcon-bridge-shim/pom.xml c59ff090 
>   addons/falcon-bridge/pom.xml 83414241 
>   addons/hdfs-model/pom.xml fb621bb0 
>   addons/hive-bridge-shim/pom.xml a9c6232e 
>   addons/hive-bridge/pom.xml d2ff8e0c 
>   addons/sqoop-bridge-shim/pom.xml f0b52d7a 
>   addons/sqoop-bridge/pom.xml 560a4ce4 
>   addons/storm-bridge-shim/pom.xml 7011cede 
>   addons/storm-bridge/pom.xml f9f7bab3 
>   authorization/pom.xml 2f12ec18 
>   catalog/pom.xml 2a7f3f1a 
>   client/pom.xml 38429902 
>   common/pom.xml 42a8a033 
>   dashboardv2/pom.xml f0ce813e 
>   distro/pom.xml 51eea365 
>   docs/pom.xml b5594731 
>   graphdb/api/pom.xml 2fc3df25 
>   graphdb/common/pom.xml f602394a 
>   graphdb/graphdb-impls/pom.xml eb07b189 
>   graphdb/pom.xml fbe8336f 
>   graphdb/titan0/pom.xml 1bc0fd65 
>   graphdb/titan1/pom.xml 99ecae10 
>   intg/pom.xml d5072863 
>   notification/pom.xml 8744f10f 
>   plugin-classloader/pom.xml 0c6aa2e5 
>   pom.xml c9528deb 
>   repository/pom.xml db7da975 
>   server-api/pom.xml 2ecb7fe6 
>   shaded/hbase-client-shaded/pom.xml e8b2800f 
>   shaded/hbase-server-shaded/pom.xml b5b82900 
>   typesystem/pom.xml 38a7d10b 
>   webapp/pom.xml 59d5f085 
> 
> 
> Diff: https://reviews.apache.org/r/61997/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Re: Review Request 61665: ATLAS-2047: Exception Thrown by Kafka Consumer Ends up Filling Logs Due to Incorrect Handling

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61665/#review184102
---


Fix it, then Ship it!





webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
Lines 246 (patched)
<https://reviews.apache.org/r/61665/#comment260305>

Instead of computing timeSinceLastWait here and passing to 
setWaitDurations(), I would suggest the following:

public void pause(Exception ex) {
  setWaitDuration();
  
  try {
...
Thread.sleep(waitDuration);
...
  } catch(...) {
...
  }
}

private void setWaitDuration() {
  long now   = System.currentTimeMillis();
  long timeSinceLastWait = now - lastWaitAt; // lastWaitAt will be 0 for 
the first time, which will result in "waitDuration = minDuration" in the 
followig if block

  lastWaitAt = now;

  if (timeSinceLastWait > resetInterval) {
waitDuration = minDuration;
  } else if (waitDuration != maxDuration) {
waitDuration += increment;

if (waitDuration > maxDuration) {
  waitDuration = maxDuration;
}
  }
}



webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
Lines 281 (patched)
<https://reviews.apache.org/r/61665/#comment260301>

Use maxWaitDuration from line #127, instead of creating a new one here.



webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
Lines 283 (patched)
<https://reviews.apache.org/r/61665/#comment260302>

500 ==> minWaitDuration


- Madhan Neethiraj


On Aug. 29, 2017, 10:37 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61665/
> ---
> 
> (Updated Aug. 29, 2017, 10:37 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Nixon Rodrigues.
> 
> 
> Bugs: ATLAS-2047
> https://issues.apache.org/jira/browse/ATLAS-2047
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Please refer to 
> [ATLAS-2047](https://issues.apache.org/jira/browse/ATLAS-2047) for background 
> and analysis.
> 
> **Background**
> 
> The _IllegalStateException_ is thrown by _KafkaConsumer.aquire_. This method 
> is called at the beginning of almost every method in this class. The method 
> checks if the consumer is closed, if it is then it throws 
> IllegalStateException.
> 
> Scenario may come about in this way:
> - Shutdown has been initiated. Close on consumer is called.
> - However, the consumer thread is just about to enter another poll cycle.
> - Thus acquire sees that consumer is closed and throws the exception (2nd 
> bullet above).
> 
> Please take a look at this scala code. This is _ShutdownableThread_. The 
> thread does the job of handling all exceptions. Upon exception, it manages 
> the _shutdownLatch_ (from yesterday’s bug fix) and gets out of the 
> _isRunning_ loop.
> ```scala
>   override def run(): Unit = {
> info("Starting ")
> try{
>   while(isRunning.get()){
> doWork()
>   }
> } catch{
>   case e: Throwable =>
> if(isRunning.get())
>   error("Error due to ", e)
> }
> shutdownLatch.countDown()
> info("Stopped ")
>   }
> ```
> 
> **Implementation**
> 
> Special treatment is given to _IllegalStateException_ by implementing pause & 
> retry logic:
> - Modified _LOG_ to _debug_. That way logs are not filled during retry.
> - _HookConsumer_ is more resilient. It handles exceptions resulting from 
> _Kafka_ and entity APIs.
> 
> 
> Diffs
> -
> 
>   
> webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
>  ef64c3b 
>   webapp/src/test/java/org/apache/atlas/notification/AdaptiveWaiterTest.java 
> PRE-CREATION 
>   
> webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerTest.java
>  a6f58e8 
> 
> 
> Diff: https://reviews.apache.org/r/61665/diff/4/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> Updated unit tests to reproduce the scenarios and verify the fix.
> 
> **Functional tests**
> Verified regular notification scenarios.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 61665: ATLAS-2047: Exception Thrown by Kafka Consumer Ends up Filling Logs Due to Incorrect Handling

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61665/#review184093
---




webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
Lines 227 (patched)
<https://reviews.apache.org/r/61665/#comment260095>

Consider marking 'resetInterval' as final.



webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
Lines 257 (patched)
<https://reviews.apache.org/r/61665/#comment260099>

Consider renaming updateDurations() to setWaitDuration() and call from line 
#240.

void setWaitDuration() {
  long now   = System.currentTimeMillis();
  long timeSinceLastWait = now - lastWaitAt;

  if (timeSinceLastWait > resetInterval) {
waitDuration = minDuration; // reset
  } else if (waitDuration != maxDuration) {
waitDuration += increment;

if (waitDuration > maxDuration) {
  waitDuration = maxDuration;
}
  }
  
  lastWaitAt = now;
}



webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
Lines 264 (patched)
<https://reviews.apache.org/r/61665/#comment260094>

Shouldn't waitDuration be set to 'maxDuration' here? Else, the next wait 
will start from minDuration - causing more frequent retries.



webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
Lines 275 (patched)
<https://reviews.apache.org/r/61665/#comment260103>

Consider reading minWaitDuration, maxWaitDuration from configuration, right 
next to where consumerRetryInterval is read currently (line #120).

consumerRetryInterval = 
applicationProperties.getInt(CONSUMER_RETRY_INTERVAL, 500);
minWaitDuration   = applicationProperties.getInt(MIN_RETRY_INTERVAL, 
consumerRetryInterval); // 500 ms  by default
maxWaitDuration   = applicationProperties.getInt(MAX_RETRY_INTERVAL, 
minWaitDuration * 60);  //  30 sec by default



webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
Lines 279 (patched)
<https://reviews.apache.org/r/61665/#comment260104>

'pauseDuration' seems to be used only in tests; considering removing this 
member and update tests to use minWaitDuration.



webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
Line 246 (original), 306 (patched)
<https://reviews.apache.org/r/61665/#comment260105>

Why not call 'adaptiveWaiter.pause()' here as well?


- Madhan Neethiraj


On Aug. 29, 2017, 9:01 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61665/
> ---
> 
> (Updated Aug. 29, 2017, 9:01 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Nixon Rodrigues.
> 
> 
> Bugs: ATLAS-2047
> https://issues.apache.org/jira/browse/ATLAS-2047
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Please refer to 
> [ATLAS-2047](https://issues.apache.org/jira/browse/ATLAS-2047) for background 
> and analysis.
> 
> **Background**
> 
> The _IllegalStateException_ is thrown by _KafkaConsumer.aquire_. This method 
> is called at the beginning of almost every method in this class. The method 
> checks if the consumer is closed, if it is then it throws 
> IllegalStateException.
> 
> Scenario may come about in this way:
> - Shutdown has been initiated. Close on consumer is called.
> - However, the consumer thread is just about to enter another poll cycle.
> - Thus acquire sees that consumer is closed and throws the exception (2nd 
> bullet above).
> 
> Please take a look at this scala code. This is _ShutdownableThread_. The 
> thread does the job of handling all exceptions. Upon exception, it manages 
> the _shutdownLatch_ (from yesterday’s bug fix) and gets out of the 
> _isRunning_ loop.
> ```scala
>   override def run(): Unit = {
> info("Starting ")
> try{
>   while(isRunning.get()){
> doWork()
>   }
> } catch{
>   case e: Throwable =>
> if(isRunning.get())
>   error("Error due to ", e)
> }
> shutdownLatch.countDown()
> info("Stopped ")
>   }
> ```
> 
> **Implementation**
> 
> Special treatment is given to _IllegalStateException_ by implementing pause & 
> retry logic:
> - Modified _LOG_ to _debug_. That way logs are not filled during retry.
> - _HookConsumer_ is more resilient. It handles exceptions resulting from 
> _Kafka_ and entity APIs.
> 
> 
> Diffs
> 

Re: Review Request 59821: ATLAS-1805:Provide an Atlas hook to send Hbase Namespace/Table/column family metadata to Atlas

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59821/#review185093
---




addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java
Lines 76 (patched)
<https://reviews.apache.org/r/59821/#comment261314>

Looks like constants in line #76 - #87 are for attribute names. It will be 
helpful to prefix these names with "ATTR_". Perhaps line #74 as well.



addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java
Lines 132 (patched)
<https://reviews.apache.org/r/59821/#comment261315>

It will be useful to print exception as well:
  LOG.info("Interrupt received in shutdown.", ie);



addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java
Lines 148 (patched)
<https://reviews.apache.org/r/59821/#comment261316>

"try {" at line #148 can move inside 'if' block at line #149.



addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java
Lines 183 (patched)
<https://reviews.apache.org/r/59821/#comment261317>

'operation' is already part of 'hbaseOperationContext'. It will be good to 
not pass this as a separate argument. Same for line #190 and #197 as well.



addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java
Lines 205 (patched)
<https://reviews.apache.org/r/59821/#comment261318>

nameSpaceRef is sent as null from the caller at line #183. It this argument 
is not necessary, please remove.



addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java
Lines 272 (patched)
<https://reviews.apache.org/r/59821/#comment261319>

- namespaceReference and tableReference are sent as null from the caller at 
line #190. It these arguments are not necessary, please remove.
- this method doesn't seem to be used outside this class. In that case, 
consider marking this a private method.



addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java
Lines 332 (patched)
<https://reviews.apache.org/r/59821/#comment261321>

- this method doesn't seem to be used outside this class. In that case, 
consider marking this a private method.



addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java
Lines 350 (patched)
<https://reviews.apache.org/r/59821/#comment261320>

- namespaceReference and tableReference are sent as null from the caller at 
line #197. It these arguments are not necessary, please remove.
- this method doesn't seem to be used outside this class. In that case, 
consider marking this a private method.



addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java
Lines 435 (patched)
<https://reviews.apache.org/r/59821/#comment261322>

to be consistent, avoid space after "(".

Also, I see other places that miss a space after a ",". Please review and 
add a space after a ",".



addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java
Lines 854 (patched)
<https://reviews.apache.org/r/59821/#comment261324>

'user' could be null - from line #838. Please update here to handle this 
case.


- Madhan Neethiraj


On Sept. 10, 2017, 7:04 a.m., Ramesh Mani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59821/
> -------
> 
> (Updated Sept. 10, 2017, 7:04 a.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-1805
> https://issues.apache.org/jira/browse/ATLAS-1805
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-1805: Provide an Atlas hook to send Hbase Namespace/Table/column family 
> metadata to Atlas
> 
> 
> Diffs
> -
> 
>   addons/hbase-bridge-shim/pom.xml PRE-CREATION 
>   
> addons/hbase-bridge-shim/src/main/java/org/apache/atlas/hbase/hook/HBaseAtlasCoprocessor.java
>  PRE-CREATION 
>   addons/hbase-bridge/pom.xml PRE-CREATION 
>   
> addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/bridge/HBaseAtlasHook.java
>  PRE-CREATION 
>   
> addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/hook/HBaseAtlasCoprocessor.java
>  PRE-CREATION 
>   
> addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/hook/HBaseAtlasCoprocessorBase.java
>  PRE-CREATION 
>   
> addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/model/HBaseDataTypes.java
>  PRE-CREATION 
>   
> addons/hbase-bridge/src/main/java/org/apache/atlas/hbase/model/HBaseOperationContext.java
>  PRE-CRE

[ANNOUNCE] New Committer: Sharmadha Sainath

[Madhan Neethiraj]

Atlas community,

 

Apache Atlas PMC has asked Sharmadha Sainath to become a committer and we are 
pleased to announce that she has accepted.

 

Welcome Sharmadha, great to have you on board.

 

On behalf of Apache Atlas PMC,

 

Madhan

 

 

 

Re: Review Request 61200: ATLAS-1996: Log4J Default Configuration Updated

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61200/#review184916
---




addons/hive-bridge/src/main/resources/atlas-hive-import-log4j.xml
Lines 35 (patched)
<https://reviews.apache.org/r/61200/#comment261115>

To limit the number of log files, set maxBackupIndex; I think 20 would be a 
good value - this would result in max space taken by Atlas log files to 2gb (20 
* 100mb).


- Madhan Neethiraj


On Sept. 7, 2017, 5:30 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61200/
> ---
> 
> (Updated Sept. 7, 2017, 5:30 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-1996
> https://issues.apache.org/jira/browse/ATLAS-1996
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Problem**
> Existing log4J configuration did not include max size parameter. This caused 
> logs to grow making it hard for human log readers.
> 
> Another contributing factor to log size growth is TitanDB's 
> _IndexSerializer_. When this is used, the query conversion from graph query 
> to indexed query is written as info log entry.
> 
> With the recent changes, many more index queries are used, thus contributing 
> to growth of log files.
> 
> **Solution**
> 
> Added maxFileSize parameter.
> Changed _com.thinkaurelius.titan_ log level to _warn_.
> 
> **Additional Info**
> Defaults need to be set in _Ambari_ so that they reflect correctly as part of 
> deployment.
> 
> 
> Diffs
> -
> 
>   addons/hive-bridge/src/main/resources/atlas-hive-import-log4j.xml c1184e5 
>   distro/src/conf/atlas-log4j.xml a0ef6a9 
>   typesystem/src/main/resources/atlas-log4j.xml c30403e 
> 
> 
> Diff: https://reviews.apache.org/r/61200/diff/2/
> 
> 
> Testing
> ---
> 
> **Functional tests**
> Observed configuration changes in the logs generated.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 62110: ATLAS-2115: Fix Regression on Basic search

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62110/#review184744
---




repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
Line 91 (original), 92 (patched)
<https://reviews.apache.org/r/62110/#comment260945>

constructInMemoryPredicate() method was setting 'inMemoryPredicate'. Why 
this change? This breaks other callers of this method - like 
EntitySearchProcessor.java line #80. Please revert this change.



repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
Line 98 (original), 99 (patched)
<https://reviews.apache.org/r/62110/#comment260944>

Consider the following renaming for better readability:
  'allGraphQueryWithoutFilters' ==> 'entityGraphQueryWithTagFilter' (this 
query returns entity vertices)
  'allGraphQueryWithFilters'==> 'tagGraphQueryWithAttributesFilter' 
(this query returns classification vertices)



repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
Lines 207 (patched)
<https://reviews.apache.org/r/62110/#comment260943>

this 'if' block is applicable even when indexQuery == null - if the block 
#192 - #198 is executed.

I think this condition should be:
 if (CollectionUtils.isNotEmpty(classificationVertices))


- Madhan Neethiraj


On Sept. 6, 2017, 6:07 a.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62110/
> ---
> 
> (Updated Sept. 6, 2017, 6:07 a.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2115
> https://issues.apache.org/jira/browse/ATLAS-2115
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-1880 introduced regression in call timings for the basic search GET 
> implementation
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  0daab030 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
>  a4a638af 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> d5e39236 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> fc973e6f 
> 
> 
> Diff: https://reviews.apache.org/r/62110/diff/1/
> 
> 
> Testing
> ---
> 
> Tested via scripts and network call timing in Chrome/Firefox.
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 62129: ATLAS-2117: Basic search issues due to Titan Solr schema

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62129/#review184749
---




repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java
Lines 228 (patched)
<https://reviews.apache.org/r/62129/#comment260951>

This is needed only for String type attributes, right?



repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java
Lines 239 (patched)
<https://reviews.apache.org/r/62129/#comment260949>

This was necessary only for NEQ cases, right? For comparators like EQ, I 
think it will be efficient to apply index filter first and apply in-memory 
filter to get rid of incorrect matches.



repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java
Line 325 (original), 326 (patched)
<https://reviews.apache.org/r/62129/#comment260950>

Can attrVal be other type of collection, like Set? If yes, use Collection, 
instead of list here.


- Madhan Neethiraj


On Sept. 6, 2017, 6:45 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62129/
> ---
> 
> (Updated Sept. 6, 2017, 6:45 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2117
> https://issues.apache.org/jira/browse/ATLAS-2117
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> See JIRA
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  0daab030 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> d5e39236 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> fc973e6f 
> 
> 
> Diff: https://reviews.apache.org/r/62129/diff/1/
> 
> 
> Testing
> ---
> 
> UI and REST calls working as expected now.
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 62255: ATLAS-2129: Addressed Case Where Abrupt Shutdown of Atlas During Import Retains Entities Imported Before Shutdown

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62255/#review185256
---


Ship it!




Ship It!

- Madhan Neethiraj


On Sept. 12, 2017, 6:52 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62255/
> ---
> 
> (Updated Sept. 12, 2017, 6:52 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2129
> https://issues.apache.org/jira/browse/ATLAS-2129
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Background**
> Existing implementation of _bulkImport_ was done in single transaction. This 
> can potentially lead to case where abrupt shutdown of Atlas during Import 
> will not retain entities already imported, this renders resume Import 
> cability useless.
> 
> **Implementation**
> - Removed _GraphTransaction_ from _bulkImport_.
> - New method _createOrUpdateForImport_ which is under trasaction scope.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
>  1c168b4 
> 
> 
> Diff: https://reviews.apache.org/r/62255/diff/1/
> 
> 
> Testing
> ---
> 
> **Unit tests**
> No new tests added.
> 
> **Functional tests**
> Import API tested.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 62236: ATLAS-2128 : UI Regression : DSL Results pagination doesn't spawn more than 2 pages

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62236/#review185257
---


Ship it!




@Keval - the fix is good. The patch applies only in branch-0.8; but the review 
indicates its for master. Can you please check? If this fix is needed in master 
as well, can you please create another patch? Thanks!

- Madhan Neethiraj


On Sept. 12, 2017, 6:35 a.m., keval bhatt wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62236/
> ---
> 
> (Updated Sept. 12, 2017, 6:35 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ayub Pathan, Madhan Neethiraj, Nixon 
> Rodrigues, Sarath Subramanian, and Sharmadha Sainath.
> 
> 
> Bugs: ATLAS-2128
> https://issues.apache.org/jira/browse/ATLAS-2128
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> 1. Created 300 hive_tables.
> 2. On Basic Search in UI , fired search typename= hive_table
> 3. hive_table results were fetched across 12 pages - 25 entities in each page 
> - which is the correct behavior.
> 4. On Advanced Search in UI , fired search typename = hive_table
>   a. In the 1st page , 1- 25 results were shown.
>   b. On clicking next , in the 2nd page , 26 - 50 results were shown
>   c. On clicking next , Expected to land in 3rd page with 51 - 75 results , 
> but , 26-50 results were loaded again and the Next Button was disabled.
> 
> Calls made to the Atlas backend seem to be correct. On clicking "Next" each 
> time , correct set of results were fetched. For example , call was made to 
> fetch entities from offset 50 , limit = 25 , but the UI still showed 26-50 
> results and disabled "Next" button.
> 
> 
> Diffs
> -
> 
>   dashboardv2/public/js/views/search/SearchResultLayoutView.js 970d6a3 
> 
> 
> Diff: https://reviews.apache.org/r/62236/diff/1/
> 
> 
> Testing
> ---
> 
> Tested with 200 data next and previous buttons is working as expected.
> 
> 
> Thanks,
> 
> keval bhatt
> 
>