wjp719 commented on PR #780:
URL: https://github.com/apache/lucene/pull/780#issuecomment-1123104829
@jpountz Hi, I modify the logic. The skip interval is only changed when
`updateCounter` is bigger than 256, and the speed of double skip interval is 3
times slower than the speed of divide skip interval.
Based on the new logic, following is the performance tests result of no
index sort in multiple segments and one segment per shard:
#### data set
es rally httpLog
#### main operations
1. use ssd disk
2. no index sort
3. set shard_number as 2
4. only search one index `logs-241998` which is 13GB and 181million docs in
total
#### query result
1. case `asc_sort_timestamp` 6.28% decrease (121.195ms->113.581ms) of query
latency
1. case ` asc-sort-timestamp-after-force-merge-1-seg` 9.36% decrease
(485.229ms->439.778ms) of query latency
2. case `asc-sort-with-after-timestamp-after-force-merge-1-seg` 9.31%
decrease (441.676ms->400.519ms) of query latency
3. other cases also have performance improvement
| Metric |
Task | Baseline | Contender | Diff |
Unit |
|--------------------------------------------------------------:|--------------------------------------------:|------------:|------------:|---------:|-------:|
| 50th percentile service time |
desc_sort_timestamp | 10.0582 | 9.32669 | -0.73154 | ms |
| 90th percentile service time |
desc_sort_timestamp | 10.7014 | 9.61116 | -1.09025 | ms |
| 99th percentile service time |
desc_sort_timestamp | 13.8007 | 9.93615 | -3.86456 | ms |
| 100th percentile service time |
desc_sort_timestamp | 30.3847 | 10.2724 | -20.1123 | ms |
| 50th percentile service time |
asc_sort_timestamp | 118.245 | 111.074 | -7.17049 | ms |
| 90th percentile service time |
asc_sort_timestamp | 121.195 | 113.581 | -7.61345 | ms |
| 99th percentile service time |
asc_sort_timestamp | 126.698 | 114.86 | -11.838 | ms |
| 100th percentile service time |
asc_sort_timestamp | 131.592 | 126.641 | -4.95039 | ms |
| 50th percentile service time |
desc_sort_with_after_timestamp | 71.1903 | 67.4284 | -3.76184 | ms |
| 90th percentile service time |
desc_sort_with_after_timestamp | 72.1165 | 68.5731 | -3.54346 | ms |
| 99th percentile service time |
desc_sort_with_after_timestamp | 81.3781 | 70.8015 | -10.5765 | ms |
| 100th percentile service time |
desc_sort_with_after_timestamp | 82.0475 | 94.6865 | 12.6391 | ms |
| 50th percentile service time |
asc_sort_with_after_timestamp | 92.1529 | 94.0611 | 1.90815 | ms |
| 90th percentile service time |
asc_sort_with_after_timestamp | 96.2921 | 95.5402 | -0.75191 | ms |
| 99th percentile service time |
asc_sort_with_after_timestamp | 118.337 | 114.149 | -4.18805 | ms |
| 100th percentile service time |
asc_sort_with_after_timestamp | 119.652 | 186.802 | 67.1499 | ms |
| 50th percentile service time |
desc-sort-timestamp-after-force-merge-1-seg | 131.23 | 125.879 |
-5.3516 | ms |
| 90th percentile service time |
desc-sort-timestamp-after-force-merge-1-seg | 133.185 | 130.755 |
-2.43072 | ms |
| 99th percentile service time |
desc-sort-timestamp-after-force-merge-1-seg | 154.65 | 140.28 |
-14.37 | ms |
| 100th percentile service time |
desc-sort-timestamp-after-force-merge-1-seg | 158.243 | 151.763 |
-6.47964 | ms |
| 50th percentile service time |
asc-sort-timestamp-after-force-merge-1-seg | 472.373 | 429.798 |
-42.5744 | ms |
| 90th percentile service time |
asc-sort-timestamp-after-force-merge-1-seg | 485.229 | 439.778 |
-45.4507 | ms |
| 99th percentile service time |
asc-sort-timestamp-after-force-merge-1-seg | 504.165 | 455.842 |
-48.3231 | ms |
| 100th percentile service time |
asc-sort-timestamp-after-force-merge-1-seg | 532.754 | 457.975 |
-74.7793 | ms |
| 50th percentile service time |
desc-sort-with-after-timestamp-after-force-merge-1-seg | 84.5181 |
80.2796 | -4.2385 | ms |
| 90th percentile service time |
desc-sort-with-after-timestamp-after-force-merge-1-seg | 85.9541 |
82.4319 | -3.52219 | ms |
| 99th percentile service time |
desc-sort-with-after-timestamp-after-force-merge-1-seg | 116.454 |
96.489 | -19.9651 | ms |
| 100th percentile service time |
desc-sort-with-after-timestamp-after-force-merge-1-seg | 120.934 |
97.8837 | -23.0508 | ms |
| 50th percentile service time |
asc-sort-with-after-timestamp-after-force-merge-1-seg | 428.985 |
392.431 | -36.5541 | ms |
| 90th percentile service time |
asc-sort-with-after-timestamp-after-force-merge-1-seg | 441.676 |
400.519 | -41.1567 | ms |
| 99th percentile service time |
asc-sort-with-after-timestamp-after-force-merge-1-seg | 449.748 |
411.862 | -37.8854 | ms |
| 100th percentile service time |
asc-sort-with-after-timestamp-after-force-merge-1-seg | 450.671 |
424.775 | -25.8964 | ms |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
