-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73292/
-----------------------------------------------------------
(Updated May 10, 2021, 4:43 p.m.)
Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj,
and Sarath Subramanian.
Changes
-------
addressed review comments
Bugs: ATLAS-4254
https://issues.apache.org/jira/browse/ATLAS-4254
Repository: atlas
Description
-------
**Current Working of Pagination**
- When user specifies page information i.e offset and limit in the request,
except some condition, offset is reset to 0
- Query is then executed from 0 offset and limit
- Filter out the results obtained from query
- Generate result set by skipping the results from 0 to the given offset
**New Working Structure**
- New Parameter 'marker' (String) has been introduced in SearchParameters which
will have encrypted string as value.
- The marker should be set as '*' for the start of the page
- The response of above request (AtlasSearchResult) will have 'nextMarker'
field which will have encrypted String which is to be passed as value of
'marker' in the subsequent request
- if marker is set then offset should not be greater than 0
**Perfomance details**
Took usecase of ranger tagsync:
- On searching tagged entities starting with offset: 0 and limit: 1000 returned
10000 entities in ~ 21s
- On searching tagged entities starting with marker: * and limit: 1000 returned
10000 entities in ~ 10s
(both the number are taken on the fresh start of the server, so as to avoid
caching affecting the numbers)
- Around 51% improvement
- Improvement is seen as the page size increases.
Diffs (updated)
-----
intg/src/main/java/org/apache/atlas/model/discovery/AtlasSearchResult.java
e1c550ef7
intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java
9d2cd4ffc
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
647ff9ca0
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
a3ab6e38c
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
5dcff3b16
repository/src/main/java/org/apache/atlas/discovery/FreeTextSearchProcessor.java
92152fff5
repository/src/main/java/org/apache/atlas/discovery/FullTextSearchProcessor.java
b37d93a83
repository/src/main/java/org/apache/atlas/discovery/SearchContext.java
aa49121f6
repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java
f9832c3b2
repository/src/main/java/org/apache/atlas/discovery/TermSearchProcessor.java
45a815886
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
027827af2
repository/src/test/java/org/apache/atlas/discovery/ClassificationSearchProcessorTest.java
e1ebbfc70
webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java 4c7b6228d
Diff: https://reviews.apache.org/r/73292/diff/4/
Changes: https://reviews.apache.org/r/73292/diff/3-4/
Testing
-------
MAnual testing
unit testcases
Thanks,
Pinal Shah
