Hi,
I'm experimenting with making field count faceting ("getFacetFieldCounts()")
multithreaded by defining a Callable for each field to run the
faceting-algorithms, and by running these with the ExecutorService having a
fixed threadpool with number of threads =
Runtime.getRuntime().availableProcessors().
In initial tests, when faceting over multiple fields, it seems like a quite
nice speed improvement for counting the facets. And it is a extremely simple
and small addition to the code.
Test index 9M docs / 9 GB, faceting over single valued fields of type
"string" :
A = Solr 3.1
B = Solr 3.1 with parallel per field faceting
In these tests "B" had the index opened via a Windows Network Share from
"A".
A => facet.method=fc, facet field count 8, avg. 840ms
B => facet.method=fc, facet field count 8, avg. 118ms
A => facet field count 8, avg. 120ms
B => facet.method=enum, facet field count 8, avg. 35ms
A => facet.method=fc, facet field count 4, avg. 470ms
B => facet.method=fc, facet field count 4, avg. 80ms
A => facet.method=enum, facet field count 4, avg. 110ms
B => facet.method=enum, facet.field count 4, avg. 30ms
A => facet.method=fc, facet field count 2, avg. 234ms
B => facet.method=fc, facet field count 2, avg. 60ms
A => facet.method=enum, facet field count 2, avg.80ms
B => facet.method=enum, facet field count 2, avg. 40ms
Tested with Chrome repeating CTRL+F5 randomly...
Now, a couple of questions :
- Is somebody already working on adding multithreading to counting the
facets ?
- Is there a reason to not letting the facet counting methods run in
parallel ? i.e. is this a bad idea ?
I see that there is some multithreading work going into SOLR 4.0 with per
segment single valued faceting, is there a bigger plan already on making the
other faceting methods multithreaded also ?
Thanks,
Janne
