Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-07-24 Thread Nixon Rodrigues

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/#review221358
---


Ship it!




Ship It!

- Nixon Rodrigues


On June 30, 2020, 1:09 p.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72567/
> ---
> 
> (Updated June 30, 2020, 1:09 p.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3782
> https://issues.apache.org/jira/browse/ATLAS-3782
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Issue:**
> The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
> implemented in SearchProcessors. It would allow a search of entities that do 
> not contain in given string in a specified attribute (eg exclude entities 
> from search that contain 'temp' in the qualified name). 
> 
> **WorkAround:**
> JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
> inmemory
> BasicSearch generates query via three modes
> 1. Index query -> NOT_CONTAINS will not be supported
> 2. InMemeory Predicates -> NOT_CONTAINS will be supported, Already handled 
> #123 SearchProcessor
> 3. Graph query -> NOT_CONTAINS will not be supported
> 
> As in index and graph query wiil not support not_Contains operator, We need 
> to apply filter(inMemoryPredicate) after either index/graph query.
> 
> To support above, I have modified ClassificationSearchProcessor
> + For both cases index as well as graph, added typeNamePredicate and 
> attributePredicate
> + Added these predicate after query
> - Removed gremlinQuery block
> 
> **Operator Value:**
> It can be either "not_contains" or "NOT_CONTAINS"
> 
> **Note:**
> As part of this jira, not_contains is also added in quick search.
> 
> 
> Diffs
> -
> 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasSolrQueryBuilder.java
>  6c06a3cbe 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  9c72cd4a2 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> c9a605355 
>   repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
>   
> repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
>  9b16e919d 
>   
> repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
>  b7ce97845 
> 
> 
> Diff: https://reviews.apache.org/r/72567/diff/4/
> 
> 
> Testing
> ---
> 
> Added testcases
> Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1999
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-07-23 Thread Sarath Subramanian

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/#review221344
---


Ship it!




Ship It!

- Sarath Subramanian


On June 30, 2020, 6:09 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72567/
> ---
> 
> (Updated June 30, 2020, 6:09 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3782
> https://issues.apache.org/jira/browse/ATLAS-3782
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Issue:**
> The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
> implemented in SearchProcessors. It would allow a search of entities that do 
> not contain in given string in a specified attribute (eg exclude entities 
> from search that contain 'temp' in the qualified name). 
> 
> **WorkAround:**
> JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
> inmemory
> BasicSearch generates query via three modes
> 1. Index query -> NOT_CONTAINS will not be supported
> 2. InMemeory Predicates -> NOT_CONTAINS will be supported, Already handled 
> #123 SearchProcessor
> 3. Graph query -> NOT_CONTAINS will not be supported
> 
> As in index and graph query wiil not support not_Contains operator, We need 
> to apply filter(inMemoryPredicate) after either index/graph query.
> 
> To support above, I have modified ClassificationSearchProcessor
> + For both cases index as well as graph, added typeNamePredicate and 
> attributePredicate
> + Added these predicate after query
> - Removed gremlinQuery block
> 
> **Operator Value:**
> It can be either "not_contains" or "NOT_CONTAINS"
> 
> **Note:**
> As part of this jira, not_contains is also added in quick search.
> 
> 
> Diffs
> -
> 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasSolrQueryBuilder.java
>  6c06a3cbe 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  9c72cd4a2 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> c9a605355 
>   repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
>   
> repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
>  9b16e919d 
>   
> repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
>  b7ce97845 
> 
> 
> Diff: https://reviews.apache.org/r/72567/diff/4/
> 
> 
> Testing
> ---
> 
> Added testcases
> Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1999
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-06-30 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/
---

(Updated June 30, 2020, 1:09 p.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3782
https://issues.apache.org/jira/browse/ATLAS-3782


Repository: atlas


Description
---

**Issue:**
The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
implemented in SearchProcessors. It would allow a search of entities that do 
not contain in given string in a specified attribute (eg exclude entities from 
search that contain 'temp' in the qualified name). 

**WorkAround:**
JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
inmemory
BasicSearch generates query via three modes
1. Index query -> NOT_CONTAINS will not be supported
2. InMemeory Predicates -> NOT_CONTAINS will be supported, Already handled #123 
SearchProcessor
3. Graph query -> NOT_CONTAINS will not be supported

As in index and graph query wiil not support not_Contains operator, We need to 
apply filter(inMemoryPredicate) after either index/graph query.

To support above, I have modified ClassificationSearchProcessor
+ For both cases index as well as graph, added typeNamePredicate and 
attributePredicate
+ Added these predicate after query
- Removed gremlinQuery block

**Operator Value:**
It can be either "not_contains" or "NOT_CONTAINS"

**Note:**
As part of this jira, not_contains is also added in quick search.


Diffs
-

  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasSolrQueryBuilder.java
 6c06a3cbe 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 9c72cd4a2 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
c9a605355 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
  
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 9b16e919d 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 b7ce97845 


Diff: https://reviews.apache.org/r/72567/diff/4/


Testing (updated)
---

Added testcases
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1999


Thanks,

Pinal Shah



Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-06-30 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/
---

(Updated June 30, 2020, 10:04 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3782
https://issues.apache.org/jira/browse/ATLAS-3782


Repository: atlas


Description (updated)
---

**Issue:**
The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
implemented in SearchProcessors. It would allow a search of entities that do 
not contain in given string in a specified attribute (eg exclude entities from 
search that contain 'temp' in the qualified name). 

**WorkAround:**
JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
inmemory
BasicSearch generates query via three modes
1. Index query -> NOT_CONTAINS will not be supported
2. InMemeory Predicates -> NOT_CONTAINS will be supported, Already handled #123 
SearchProcessor
3. Graph query -> NOT_CONTAINS will not be supported

As in index and graph query wiil not support not_Contains operator, We need to 
apply filter(inMemoryPredicate) after either index/graph query.

To support above, I have modified ClassificationSearchProcessor
+ For both cases index as well as graph, added typeNamePredicate and 
attributePredicate
+ Added these predicate after query
- Removed gremlinQuery block

**Operator Value:**
It can be either "not_contains" or "NOT_CONTAINS"

**Note:**
As part of this jira, not_contains is also added in quick search.


Diffs
-

  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasSolrQueryBuilder.java
 6c06a3cbe 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 9c72cd4a2 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
c9a605355 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
  
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 9b16e919d 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 b7ce97845 


Diff: https://reviews.apache.org/r/72567/diff/4/


Testing
---

Added testcases
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1983


Thanks,

Pinal Shah



Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-06-30 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/
---

(Updated June 30, 2020, 9:54 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

Rebase patch and added not_contains in quicksearch aggregation metrics


Bugs: ATLAS-3782
https://issues.apache.org/jira/browse/ATLAS-3782


Repository: atlas


Description
---

The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
implemented in SearchProcessors. It would allow a search of entities that do 
not contain in given string in a specified attribute (eg exclude entities from 
search that contain 'temp' in the qualified name). 

JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
inmemory
BasicSearch can generates query via three modes
1. Index query -> NOT_CONTAINS will not be supported
2. InMemeory Predicated -> NOT_CONTAINS will be supported, Already handled #123 
SearchProcessor
3. Graph query -> NOT_CONTAINS will not be supported

We need to apply filter(inMemoryPredicate) after either index/graph query.

To support above, i have modified ClassificationSearchProcessor
+ For both cases index as well as graph, added typeNamePredicate and 
attributePredicate
+ Added these predicate after query
- Removed gremlinQuery block


Diffs (updated)
-

  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasSolrQueryBuilder.java
 6c06a3cbe 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 9c72cd4a2 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
c9a605355 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
  
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 9b16e919d 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 b7ce97845 


Diff: https://reviews.apache.org/r/72567/diff/4/

Changes: https://reviews.apache.org/r/72567/diff/3-4/


Testing
---

Added testcases
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1983


Thanks,

Pinal Shah



Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-06-25 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/
---

(Updated June 25, 2020, 9:17 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

commented one testcase, need to look, succesful on local but failed for 
precommit


Bugs: ATLAS-3782
https://issues.apache.org/jira/browse/ATLAS-3782


Repository: atlas


Description
---

The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
implemented in SearchProcessors. It would allow a search of entities that do 
not contain in given string in a specified attribute (eg exclude entities from 
search that contain 'temp' in the qualified name). 

JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
inmemory
BasicSearch can generates query via three modes
1. Index query -> NOT_CONTAINS will not be supported
2. InMemeory Predicated -> NOT_CONTAINS will be supported, Already handled #123 
SearchProcessor
3. Graph query -> NOT_CONTAINS will not be supported

We need to apply filter(inMemoryPredicate) after either index/graph query.

To support above, i have modified ClassificationSearchProcessor
+ For both cases index as well as graph, added typeNamePredicate and 
attributePredicate
+ Added these predicate after query
- Removed gremlinQuery block


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 5dd0d7f97 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
ad48be11b 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
  
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 9b16e919d 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 43f11d1df 


Diff: https://reviews.apache.org/r/72567/diff/3/

Changes: https://reviews.apache.org/r/72567/diff/2-3/


Testing
---

Added testcases
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952


Thanks,

Pinal Shah



Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-06-25 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/
---

(Updated June 25, 2020, 6:57 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

rebase patch, to update indexes from solrConfig.xml and removed one log 
statement


Bugs: ATLAS-3782
https://issues.apache.org/jira/browse/ATLAS-3782


Repository: atlas


Description
---

The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
implemented in SearchProcessors. It would allow a search of entities that do 
not contain in given string in a specified attribute (eg exclude entities from 
search that contain 'temp' in the qualified name). 

JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
inmemory
BasicSearch can generates query via three modes
1. Index query -> NOT_CONTAINS will not be supported
2. InMemeory Predicated -> NOT_CONTAINS will be supported, Already handled #123 
SearchProcessor
3. Graph query -> NOT_CONTAINS will not be supported

We need to apply filter(inMemoryPredicate) after either index/graph query.

To support above, i have modified ClassificationSearchProcessor
+ For both cases index as well as graph, added typeNamePredicate and 
attributePredicate
+ Added these predicate after query
- Removed gremlinQuery block


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 5dd0d7f97 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
ad48be11b 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
  
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 9b16e919d 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 43f11d1df 


Diff: https://reviews.apache.org/r/72567/diff/2/

Changes: https://reviews.apache.org/r/72567/diff/1-2/


Testing
---

Added testcases
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952


Thanks,

Pinal Shah



Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-06-04 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/
---

(Updated June 4, 2020, 12:05 p.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3782
https://issues.apache.org/jira/browse/ATLAS-3782


Repository: atlas


Description
---

The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
implemented in SearchProcessors. It would allow a search of entities that do 
not contain in given string in a specified attribute (eg exclude entities from 
search that contain 'temp' in the qualified name). 

JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
inmemory
BasicSearch can generates query via three modes
1. Index query -> NOT_CONTAINS will not be supported
2. InMemeory Predicated -> NOT_CONTAINS will be supported, Already handled #123 
SearchProcessor
3. Graph query -> NOT_CONTAINS will not be supported

We need to apply filter(inMemoryPredicate) after either index/graph query.

To support above, i have modified ClassificationSearchProcessor
+ For both cases index as well as graph, added typeNamePredicate and 
attributePredicate
+ Added these predicate after query
- Removed gremlinQuery block


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 5dd0d7f97 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
2c5f925c7 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
  
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 9b16e919d 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 43f11d1df 
  test-tools/src/main/resources/solr/core-template/solrconfig.xml ef74a0a45 


Diff: https://reviews.apache.org/r/72567/diff/1/


Testing (updated)
---

Added testcases
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952


Thanks,

Pinal Shah