Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72567/#review221358 --- Ship it! Ship It! - Nixon Rodrigues On June 30, 2020, 1:09 p.m., Pinal Shah wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72567/ > --- > > (Updated June 30, 2020, 1:09 p.m.) > > > Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, > and Sarath Subramanian. > > > Bugs: ATLAS-3782 > https://issues.apache.org/jira/browse/ATLAS-3782 > > > Repository: atlas > > > Description > --- > > **Issue:** > The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and > implemented in SearchProcessors. It would allow a search of entities that do > not contain in given string in a specified attribute (eg exclude entities > from search that contain 'temp' in the qualified name). > > **WorkAround:** > JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in > inmemory > BasicSearch generates query via three modes > 1. Index query -> NOT_CONTAINS will not be supported > 2. InMemeory Predicates -> NOT_CONTAINS will be supported, Already handled > #123 SearchProcessor > 3. Graph query -> NOT_CONTAINS will not be supported > > As in index and graph query wiil not support not_Contains operator, We need > to apply filter(inMemoryPredicate) after either index/graph query. > > To support above, I have modified ClassificationSearchProcessor > + For both cases index as well as graph, added typeNamePredicate and > attributePredicate > + Added these predicate after query > - Removed gremlinQuery block > > **Operator Value:** > It can be either "not_contains" or "NOT_CONTAINS" > > **Note:** > As part of this jira, not_contains is also added in quick search. > > > Diffs > - > > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasSolrQueryBuilder.java > 6c06a3cbe > > repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java > 9c72cd4a2 > repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java > c9a605355 > repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 > > repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java > 9b16e919d > > repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java > b7ce97845 > > > Diff: https://reviews.apache.org/r/72567/diff/4/ > > > Testing > --- > > Added testcases > Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1999 > > > Thanks, > > Pinal Shah > >
Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72567/#review221344 --- Ship it! Ship It! - Sarath Subramanian On June 30, 2020, 6:09 a.m., Pinal Shah wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72567/ > --- > > (Updated June 30, 2020, 6:09 a.m.) > > > Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, > and Sarath Subramanian. > > > Bugs: ATLAS-3782 > https://issues.apache.org/jira/browse/ATLAS-3782 > > > Repository: atlas > > > Description > --- > > **Issue:** > The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and > implemented in SearchProcessors. It would allow a search of entities that do > not contain in given string in a specified attribute (eg exclude entities > from search that contain 'temp' in the qualified name). > > **WorkAround:** > JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in > inmemory > BasicSearch generates query via three modes > 1. Index query -> NOT_CONTAINS will not be supported > 2. InMemeory Predicates -> NOT_CONTAINS will be supported, Already handled > #123 SearchProcessor > 3. Graph query -> NOT_CONTAINS will not be supported > > As in index and graph query wiil not support not_Contains operator, We need > to apply filter(inMemoryPredicate) after either index/graph query. > > To support above, I have modified ClassificationSearchProcessor > + For both cases index as well as graph, added typeNamePredicate and > attributePredicate > + Added these predicate after query > - Removed gremlinQuery block > > **Operator Value:** > It can be either "not_contains" or "NOT_CONTAINS" > > **Note:** > As part of this jira, not_contains is also added in quick search. > > > Diffs > - > > > graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasSolrQueryBuilder.java > 6c06a3cbe > > repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java > 9c72cd4a2 > repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java > c9a605355 > repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 > > repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java > 9b16e919d > > repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java > b7ce97845 > > > Diff: https://reviews.apache.org/r/72567/diff/4/ > > > Testing > --- > > Added testcases > Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1999 > > > Thanks, > > Pinal Shah > >
Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72567/ --- (Updated June 30, 2020, 1:09 p.m.) Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Bugs: ATLAS-3782 https://issues.apache.org/jira/browse/ATLAS-3782 Repository: atlas Description --- **Issue:** The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and implemented in SearchProcessors. It would allow a search of entities that do not contain in given string in a specified attribute (eg exclude entities from search that contain 'temp' in the qualified name). **WorkAround:** JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in inmemory BasicSearch generates query via three modes 1. Index query -> NOT_CONTAINS will not be supported 2. InMemeory Predicates -> NOT_CONTAINS will be supported, Already handled #123 SearchProcessor 3. Graph query -> NOT_CONTAINS will not be supported As in index and graph query wiil not support not_Contains operator, We need to apply filter(inMemoryPredicate) after either index/graph query. To support above, I have modified ClassificationSearchProcessor + For both cases index as well as graph, added typeNamePredicate and attributePredicate + Added these predicate after query - Removed gremlinQuery block **Operator Value:** It can be either "not_contains" or "NOT_CONTAINS" **Note:** As part of this jira, not_contains is also added in quick search. Diffs - graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasSolrQueryBuilder.java 6c06a3cbe repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java 9c72cd4a2 repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java c9a605355 repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java 9b16e919d repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java b7ce97845 Diff: https://reviews.apache.org/r/72567/diff/4/ Testing (updated) --- Added testcases Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1999 Thanks, Pinal Shah
Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72567/ --- (Updated June 30, 2020, 10:04 a.m.) Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Bugs: ATLAS-3782 https://issues.apache.org/jira/browse/ATLAS-3782 Repository: atlas Description (updated) --- **Issue:** The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and implemented in SearchProcessors. It would allow a search of entities that do not contain in given string in a specified attribute (eg exclude entities from search that contain 'temp' in the qualified name). **WorkAround:** JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in inmemory BasicSearch generates query via three modes 1. Index query -> NOT_CONTAINS will not be supported 2. InMemeory Predicates -> NOT_CONTAINS will be supported, Already handled #123 SearchProcessor 3. Graph query -> NOT_CONTAINS will not be supported As in index and graph query wiil not support not_Contains operator, We need to apply filter(inMemoryPredicate) after either index/graph query. To support above, I have modified ClassificationSearchProcessor + For both cases index as well as graph, added typeNamePredicate and attributePredicate + Added these predicate after query - Removed gremlinQuery block **Operator Value:** It can be either "not_contains" or "NOT_CONTAINS" **Note:** As part of this jira, not_contains is also added in quick search. Diffs - graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasSolrQueryBuilder.java 6c06a3cbe repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java 9c72cd4a2 repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java c9a605355 repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java 9b16e919d repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java b7ce97845 Diff: https://reviews.apache.org/r/72567/diff/4/ Testing --- Added testcases Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952 Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1983 Thanks, Pinal Shah
Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72567/ --- (Updated June 30, 2020, 9:54 a.m.) Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Changes --- Rebase patch and added not_contains in quicksearch aggregation metrics Bugs: ATLAS-3782 https://issues.apache.org/jira/browse/ATLAS-3782 Repository: atlas Description --- The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and implemented in SearchProcessors. It would allow a search of entities that do not contain in given string in a specified attribute (eg exclude entities from search that contain 'temp' in the qualified name). JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in inmemory BasicSearch can generates query via three modes 1. Index query -> NOT_CONTAINS will not be supported 2. InMemeory Predicated -> NOT_CONTAINS will be supported, Already handled #123 SearchProcessor 3. Graph query -> NOT_CONTAINS will not be supported We need to apply filter(inMemoryPredicate) after either index/graph query. To support above, i have modified ClassificationSearchProcessor + For both cases index as well as graph, added typeNamePredicate and attributePredicate + Added these predicate after query - Removed gremlinQuery block Diffs (updated) - graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasSolrQueryBuilder.java 6c06a3cbe repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java 9c72cd4a2 repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java c9a605355 repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java 9b16e919d repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java b7ce97845 Diff: https://reviews.apache.org/r/72567/diff/4/ Changes: https://reviews.apache.org/r/72567/diff/3-4/ Testing --- Added testcases Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952 Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1983 Thanks, Pinal Shah
Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72567/ --- (Updated June 25, 2020, 9:17 a.m.) Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Changes --- commented one testcase, need to look, succesful on local but failed for precommit Bugs: ATLAS-3782 https://issues.apache.org/jira/browse/ATLAS-3782 Repository: atlas Description --- The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and implemented in SearchProcessors. It would allow a search of entities that do not contain in given string in a specified attribute (eg exclude entities from search that contain 'temp' in the qualified name). JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in inmemory BasicSearch can generates query via three modes 1. Index query -> NOT_CONTAINS will not be supported 2. InMemeory Predicated -> NOT_CONTAINS will be supported, Already handled #123 SearchProcessor 3. Graph query -> NOT_CONTAINS will not be supported We need to apply filter(inMemoryPredicate) after either index/graph query. To support above, i have modified ClassificationSearchProcessor + For both cases index as well as graph, added typeNamePredicate and attributePredicate + Added these predicate after query - Removed gremlinQuery block Diffs (updated) - repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java 5dd0d7f97 repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java ad48be11b repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java 9b16e919d repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java 43f11d1df Diff: https://reviews.apache.org/r/72567/diff/3/ Changes: https://reviews.apache.org/r/72567/diff/2-3/ Testing --- Added testcases Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952 Thanks, Pinal Shah
Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72567/ --- (Updated June 25, 2020, 6:57 a.m.) Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Changes --- rebase patch, to update indexes from solrConfig.xml and removed one log statement Bugs: ATLAS-3782 https://issues.apache.org/jira/browse/ATLAS-3782 Repository: atlas Description --- The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and implemented in SearchProcessors. It would allow a search of entities that do not contain in given string in a specified attribute (eg exclude entities from search that contain 'temp' in the qualified name). JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in inmemory BasicSearch can generates query via three modes 1. Index query -> NOT_CONTAINS will not be supported 2. InMemeory Predicated -> NOT_CONTAINS will be supported, Already handled #123 SearchProcessor 3. Graph query -> NOT_CONTAINS will not be supported We need to apply filter(inMemoryPredicate) after either index/graph query. To support above, i have modified ClassificationSearchProcessor + For both cases index as well as graph, added typeNamePredicate and attributePredicate + Added these predicate after query - Removed gremlinQuery block Diffs (updated) - repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java 5dd0d7f97 repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java ad48be11b repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java 9b16e919d repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java 43f11d1df Diff: https://reviews.apache.org/r/72567/diff/2/ Changes: https://reviews.apache.org/r/72567/diff/1-2/ Testing --- Added testcases Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952 Thanks, Pinal Shah
Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72567/ --- (Updated June 4, 2020, 12:05 p.m.) Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Bugs: ATLAS-3782 https://issues.apache.org/jira/browse/ATLAS-3782 Repository: atlas Description --- The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and implemented in SearchProcessors. It would allow a search of entities that do not contain in given string in a specified attribute (eg exclude entities from search that contain 'temp' in the qualified name). JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in inmemory BasicSearch can generates query via three modes 1. Index query -> NOT_CONTAINS will not be supported 2. InMemeory Predicated -> NOT_CONTAINS will be supported, Already handled #123 SearchProcessor 3. Graph query -> NOT_CONTAINS will not be supported We need to apply filter(inMemoryPredicate) after either index/graph query. To support above, i have modified ClassificationSearchProcessor + For both cases index as well as graph, added typeNamePredicate and attributePredicate + Added these predicate after query - Removed gremlinQuery block Diffs - repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java 5dd0d7f97 repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 2c5f925c7 repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java 9b16e919d repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java 43f11d1df test-tools/src/main/resources/solr/core-template/solrconfig.xml ef74a0a45 Diff: https://reviews.apache.org/r/72567/diff/1/ Testing (updated) --- Added testcases Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952 Thanks, Pinal Shah
Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72567/ --- Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Bugs: ATLAS-3782 https://issues.apache.org/jira/browse/ATLAS-3782 Repository: atlas Description --- The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and implemented in SearchProcessors. It would allow a search of entities that do not contain in given string in a specified attribute (eg exclude entities from search that contain 'temp' in the qualified name). JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in inmemory BasicSearch can generates query via three modes 1. Index query -> NOT_CONTAINS will not be supported 2. InMemeory Predicated -> NOT_CONTAINS will be supported, Already handled #123 SearchProcessor 3. Graph query -> NOT_CONTAINS will not be supported We need to apply filter(inMemoryPredicate) after either index/graph query. To support above, i have modified ClassificationSearchProcessor + For both cases index as well as graph, added typeNamePredicate and attributePredicate + Added these predicate after query - Removed gremlinQuery block Diffs - repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java 5dd0d7f97 repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 2c5f925c7 repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java 9b16e919d repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java 43f11d1df test-tools/src/main/resources/solr/core-template/solrconfig.xml ef74a0a45 Diff: https://reviews.apache.org/r/72567/diff/1/ Testing --- Added testcases Precommit : Thanks, Pinal Shah
