[
https://issues.apache.org/jira/browse/SOLR-14137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris M. Hostetter updated SOLR-14137:
--------------------------------------
Attachment: hoss_experiement1.tgz
Status: Open (was: Open)
So.... i actually started trying to reproduce Erick's observations yesterday
based on what little info i could find in the ticket/thread about his
methodology – before he posted his github repo.
I'm attaching a tgz containing all the scripts, timing logs and graphs produced
by my experiments (just not the raw 'docs.csv' generated by the scripts, or the
resulting indexes)
My testing was based on:
* comparing 8.4.1 to 7.7.2
* using _default configsets for each
* single core indexes of 1 million docs
** (because i didn't want to sit around waiting to build a 20 million doc
index on my laptop)
* text field built using a the "word list" of 2000 words
** and each doc had 50 random words.
* date field with unique values for each doc, starting with "NOW" and going
backword 0-120seconds using random milliscond deltas
My initial queries were "date boosted" disjunction queries of 10 words (from
the word list) using rows=0.
I compared the timing _from stopwatches run by the query client_ of running
1000 queries in a single thread to running 1000 queries each in 16 concurrent
threads, and didn't see any obvious and/or massive impacts of concurrent
queries in how the query times compared between 7x and 8x – What I did see was
that the mean time of the 7x queries was ~80-85% of the mean time of the 8x
queries. (FWIW: erick's "second-run.png" seems to show 77% relative mean
response time comaring 7.7.1 to 8.3.1)
Based on those observations, I stopped worrying about doing heavy concurrent
threaded query testing, and focused on single thread query testing since it let
me iterate faster.
As far as i can tell, date boosting isn't a significant factor in the relateive
speed of 7x vs 8x queries – date boosted queries slower are then w/o a date
boost, but that relative difference in query time doesn't seem to depend on the
solr version?
*It seems like in general disjunctions may have gotten slower?*
1000 date boosted "match all docs" queries in 7x has a mean request time of 96%
of the same queries in 8x – which seems close enough to suggest it may just be
noise, or something related to jetty level HTTP changes between version...
meanwhile 1000 single term queries _w/o any date boosting_ on 7x clocks in at
80% the mean request time of those same queries in 8x.
1000 date boosted queries containing 100 disjunction clauses (where ~33% aren't
in the word list and won't match any docs) results in 7x having a mean request
time of 82% of the same queries in 8x .... that changes to only 84% when we
remove the date boost – suggesting that date boosting isn't a significant
factor here in relative speed of queries between 7x and 8x.
----
Obviously index size (both in terms of number of docs and number of terms in
the corpus) is going to be a big factor, but I think the most important
question to start with is...
[~erickerickson]: did you do any testing of just basic 1, 10, or 100 clause
disjunction queries _w/o any date boosting_ to see how basic query performance
compared between these versions of solr?
> Boosting by date (and perhaps others) shows a steady decline 6.6->8.3
> ---------------------------------------------------------------------
>
> Key: SOLR-14137
> URL: https://issues.apache.org/jira/browse/SOLR-14137
> Project: Solr
> Issue Type: Improvement
> Reporter: Erick Erickson
> Priority: Major
> Attachments: Screen Shot 2019-12-19 at 2.35.41 PM.png, Screen Shot
> 2019-12-19 at 3.09.37 PM.png, Screen Shot 2019-12-19 at 3.31.16 PM.png,
> hoss_experiement1.tgz, second_run.png
>
>
> Moving a user's list discussion over here.
> {color:#000000}The very short form is that from Solr 6.6.1 to Solr 8.3.1, the
> throughput for date boosting in my tests dropped by 40+%{color}
> {color:#000000}I’ve been hearing about slowdowns in successive Solr releases
> with boost functions, so I dug into it a bit. The test setup is just a
> boost-by-date with an additional big OR clause of 100 random words so I’d be
> sure to hit a bunch of docs. I figured that if there were few hits, the
> signal would be lost in the noise, but I didn’t look at the actual hit
> counts.{color}
> {color:#000000}I saw several Solr JIRAs about this subject, but they were
> slightly different, although quite possibly the same underlying issue. So I
> tried to get this down to a very specific form of a query.{color}
> {color:#000000}I’ve also seen some cases in the wild where the response was
> proportional to the number of segments, thus my optimize experiments.{color}
> {color:#000000}Here are the results, explanation below. O stands for
> optimized to one segment. I spot checked pdate against 6.6, 7.1 and 8.3 and
> they weren’t significantly different performance wise from tdate. All have
> docValues enabled. I ran these against a multiValued=“false” field. All the
> tests pegged all my CPUs. Jmeter is being run on a different machine than
> Solr. Only one Solr was running for any test.{color}
> {color:#000000}Solr version queries/min {color}
> {color:#000000}6.6.1 3,400 {color}
> {color:#000000}6.6.1 O 4,800 {color}
> {color:#000000}7.1 2,800 {color}
> {color:#000000}7.1 O 4,200 {color}
> {color:#000000}7.7.1 2,400 {color}
> {color:#000000}7.7.1 O 3,500 {color}
> {color:#000000}8.3.1 2,000 {color}
> {color:#000000}8.3.1 O 2,600 {color}
> {color:#000000}The tests I’ve been running just index 20M docs into a single
> core, then run the exact same 10,000 queries against them from jmeter with 24
> threads. Spot checks showed no hits on the queryResultCache.{color}
> {color:#000000}A query looks like this: {color}
> {color:#000000}rows=0&\{!boost b=recip(ms(NOW,
> INSERT_FIELD_HERE),3.16e-11,1,1)}text_txt:(campaigners OR adjourned OR
> anyplace…97 more random words){color}
> {color:#000000}There is no faceting. No grouping. No sorting.{color}
> {color:#000000}I fill in INSERT_FIELD_HERE through jmeter magic. I’m running
> the exact same queries for every test.{color}
> {color:#000000}One wildcard is that I did regenerate the index for each major
> revision, and the chose random words from the same list of words, as well as
> random times (bounded in the same range though) so the docs are not
> completely identical. The index was in the native format for that major
> version even if slightly different between versions. I ran the test once,
> then ran it again after optimizing the index.{color}
> {color:#000000}I haven’t dug any farther, if anyone’s interested I can throw
> a profiler at, say, 8.3 and see what I can see, although I’m not going to
> have time to dive into this any time soon. I’d be glad to run some tests
> though. I saved the queries and the indexes so running a test would only
> take a few minutes.{color}
> {color:#000000}While I concentrated on date fields, the docs have date, int,
> and long fields, both docValues=true and docValues=false, each variant with
> multiValued=true and multiValued=false and both Trie and Point (where
> possible) variants as well as a pretty simple text field.{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]