[
https://issues.apache.org/jira/browse/LUCENE-5425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13888463#comment-13888463
]
Lei Wang commented on LUCENE-5425:
----------------------------------
I'm not exactly sure how to run the facets benchmarks. I did a run on ant
run-task -Dtask.alg=conf/facets.alg, and changed "SearchSameRdr" Search > : 40
to 40000, to get stable results.
I'm not sure how to read the results also..., but the numbers looks quite
similar between trunk and the docidset on my box.
trunk:
[java] ------------> Report sum by Prefix (Search) and Round (4 about 4
out of 42)
[java] Operation round facets runCnt recsPerRun rec/s
elapsedSec avgUsedMem avgTotalMem
[java] SearchSameRdr_40000 0 true 1 40000 7,155.64
5.59 31,610,096 51,212,288
[java] SearchSameRdr_40000 - 1 false - - 1 - - 40000 - 8,814.46
- - 4.54 - 33,534,008 - 49,209,344
[java] SearchSameRdr_40000 2 true 1 40000 9,088.84
4.40 35,673,136 48,373,760
[java] SearchSameRdr_40000 - 3 false - - 1 - - 40000 - 9,045.68
- - 4.42 - 35,279,544 - 47,661,056
[java]
[java]
[java] ------------> Report sum by Prefix (Populate) and Round (4 about 4
out of 42)
[java] Operation round facets runCnt recsPerRun rec/s
elapsedSec avgUsedMem avgTotalMem
[java] Populate 0 true 1 21578 2,489.96
8.67 31,369,696 51,212,288
[java] Populate - - 1 false - - 1 - - 21578 - 3,973.85 - -
5.43 - 33,272,104 - 49,209,344
[java] Populate 2 true 1 21578 4,216.92
5.12 32,701,392 48,373,760
[java] Populate - - 3 false - - 1 - - 21578 - 4,366.25 - -
4.94 - 35,064,408 - 47,661,056
[java]
[java]
[java] ------------> Report sum by Prefix (MAddDocs) and Round (4 about 4
out of 42)
[java] Operation round facets runCnt recsPerRun rec/s
elapsedSec avgUsedMem avgTotalMem
[java] MAddDocs_Exhaust 0 true 1 21578 3,469.13
6.22 24,536,720 51,212,288
[java] MAddDocs_Exhaust - 1 false - - 1 - - 21578 - 4,845.72 -
- 4.45 - 34,857,920 - 49,209,344
[java] MAddDocs_Exhaust 2 true 1 21578 5,129.07
4.21 29,209,256 48,373,760
[java] MAddDocs_Exhaust - 3 false - - 1 - - 21578 - 5,259.08 -
- 4.10 - 25,845,424 - 47,661,056
With the patch, but I changed the OpenBitSet to FixedBitSet, and use
bits.iterator() to return iterator (It's still an OpenBitSetIterator), the
result:
[java] ------------> Report sum by Prefix (Search) and Round (4 about 4
out of 42)
[java] Operation round facets runCnt recsPerRun rec/s
elapsedSec avgUsedMem avgTotalMem
[java] SearchSameRdr_40000 0 true 1 40000 7,280.67
5.49 25,424,104 51,113,984
[java] SearchSameRdr_40000 - 1 false - - 1 - - 40000 - 8,689.98
- - 4.60 - 31,356,960 - 49,053,696
[java] SearchSameRdr_40000 2 true 1 40000 9,157.51
4.37 38,849,248 47,632,384
[java] SearchSameRdr_40000 - 3 false - - 1 - - 40000 - 9,097.11
- - 4.40 - 39,840,912 - 46,465,024
[java]
[java]
[java] ------------> Report sum by Prefix (Populate) and Round (4 about 4
out of 42)
[java] Operation round facets runCnt recsPerRun rec/s
elapsedSec avgUsedMem avgTotalMem
[java] Populate 0 true 1 21578 2,465.21
8.75 25,187,152 51,113,984
[java] Populate - - 1 false - - 1 - - 21578 - 2,651.19 - -
8.14 - 30,985,904 - 49,053,696
[java] Populate 2 true 1 21578 4,247.64
5.08 38,656,320 47,632,384
[java] Populate - - 3 false - - 1 - - 21578 - 4,298.41 - -
5.02 - 39,355,912 - 46,465,024
[java]
[java]
[java] ------------> Report sum by Prefix (MAddDocs) and Round (4 about 4
out of 42)
[java] Operation round facets runCnt recsPerRun rec/s
elapsedSec avgUsedMem avgTotalMem
[java] MAddDocs_Exhaust 0 true 1 21578 3,404.01
6.34 34,015,968 51,113,984
[java] MAddDocs_Exhaust - 1 false - - 1 - - 21578 - 3,062.88 -
- 7.05 - 30,420,848 - 49,053,696
[java] MAddDocs_Exhaust 2 true 1 21578 5,147.42
4.19 28,833,976 47,632,384
[java] MAddDocs_Exhaust - 3 false - - 1 - - 21578 - 5,129.07 -
- 4.21 - 37,117,288 - 46,465,024
> Make creation of FixedBitSet in FacetsCollector overridable
> -----------------------------------------------------------
>
> Key: LUCENE-5425
> URL: https://issues.apache.org/jira/browse/LUCENE-5425
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/facet
> Affects Versions: 4.6
> Reporter: John Wang
> Attachments: facetscollector.patch, facetscollector.patch
>
>
> In FacetsCollector, creation of bits in MatchingDocs are allocated per query.
> For large indexes where maxDocs are large creating a bitset of maxDoc bits
> will be expensive and would great a lot of garbage.
> Attached patch is to allow for this allocation customizable while maintaining
> current behavior.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
