prorated early termination

2019-02-01 Thread Michael Sokolov
I want to propose an optimization to early termination that gives nice speedups for large result sets when searching with multiple threads at the cost of a small (controllable) probability of collecting documents out of order: in benchmarks I see +60-70% QPS for tasks like HighTermDayOfYearSort whe

Re: prorated early termination

2019-02-01 Thread Adrien Grand
Something makes me curious: queries that can leverage sorted indices should be _very_ fast, for instance in your case they only need to look at 500 documents per segment at most (less in practice since we stop collecting as soon as a non-competitive hit is found), so why do you need to parallelize

Re: prorated early termination

2019-02-01 Thread Michael Sokolov
That's a good question. I don't have a very satisfying answer other than to say we saw some improvements, and I would have to dig more now to say why. It may be that in our system we have some additional per-document costs in a custom collector that were saved by this reduction, and that's why we s

Getting list of actual fields in index

2019-02-01 Thread Vladimir Kroz
I'm working on a tool that lists all fields in the index, including explicit list of all dynamic fields. I tried GET `/solr/mycore/schema/fields` Schema API however without any luck. On the other hand there is undocumented API used by Solr UI: GET `/solr/mycore/admin/luke?wt=json` which among other