There was also a bug in the lazy loading of multivalued fields at one point
recently in Solr 4.2
https://issues.apache.org/jira/browse/SOLR-4589
"4.x + enableLazyFieldLoading + large multivalued fields + varying fl =
pathological CPU load & response time"
Do you use multivalued fields very heavily?
I'm still not ready to suggest that 1,000 fields is an okay thing to do, but
there are still plenty of nuances in Solr performance that could explain the
difficulties, before we even get to the 1,000 field issue itself.
The real bottom line is that as you increase field count, there are lots of
other aspects of Solr memory and performance degradation that increase as
well. Some of those factors can be dealt with simply with more memory, more
and faster CPU cores, or even more sharding, or other tuning, but not
necessarily all of them.
I think that I am already on the record on other threads as suggesting that
"a couple hundred" is about the limit for field count for a "slam dunk" use
of Solr. That doesn't mean you can't go above a couple hundred fields, just
that you are in uncharted territory and may need to take extraordinary
measures to get everything working satisfactorily. There's no magic hard
limit, just a general sense that smaller numbers of of field are like "a
walk in a park", while higher numbers of fields are like "chopping through a
jungle." We each have our own threshold for... "adventure."
We need answers to the previous questions we raised before we can analyze
this a lot further.
Oh, and make sure there is enough OS system memory available for caching of
the index pages. Sometimes, it is little things like this that can crush
Solr performance.
Unfortunately, Solr is not a packaged "solution" that automatically and
magically auto-configures everything to "work just right". Instead, it is a
powerful toolkit that lets you do amazing things, but you the
developer/architect need to supply amazing intelligence, wisdom, foresight,
and insight to get it (and its hardware and software environment) to do
those amazing things.
-- Jack Krupansky
-----Original Message-----
From: Alexandre Rafalovitch
Sent: Tuesday, July 23, 2013 9:54 AM
To: solr-user@lucene.apache.org
Subject: Re: how number of indexed fields effect performance
Do you need all of the fields loaded every time and are they stored? Maybe
there is a document with gigantic content that you don't actually need but
it gets deserialized anyway. Try lazy loading
setting: enableLazyFieldLoading in solrconfig.xml
Regards,
Alex.
Personal website: http://www.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is the quality of nature that keeps events from happening all at
once. Lately, it doesn't seem to be working. (Anonymous - via GTD book)
On Tue, Jul 23, 2013 at 12:36 AM, Jack Krupansky
<j...@basetechnology.com>wrote:
After restarting Solr and doing a couple of queries to warm the caches,
are queries already slow/failing, or does it take some time and a number
of
queries before failures start occurring?
One possibility is that you just need a lot more memory for caches for
this amount of data. So, maybe the failures are caused by heavy garbage
collections. So, after restarting Solr, check how much Java heap is
available, then do some warming queries, then check the Java heap
available
again.
Add the debugQuery=true parameter to your queries and look at the timings
to see what phases of query processing are taking the most time. Also
check
whether the reported QTime seems to match actual wall clock time;
sometimes
formatting of the results and network transfer time can dwarf actual query
time.
How many fields are you returning on a typical query?
-- Jack Krupansky
-----Original Message----- From: Suryansh Purwar
Sent: Monday, July 22, 2013 11:06 PM
To: solr-user@lucene.apache.org ; j...@basetechnology.com
Subject: how number of indexed fields effect performance
It was running fine initially when we just had around 100 fields
indexed. In this case as well it runs fine but after sometime broken pipe
exception starts coming which results in shard getting down.
Regards,
Suryansh
On Tuesday, July 23, 2013, Jack Krupansky wrote:
Was all of this running fine previously and only started running slow
recently, or is this your first measurement?
Are very simple queries (single keyword, no filters or facets or sorting
or anything else, and returning only a few fields) working reasonably
well?
-- Jack Krupansky
-----Original Message----- From: Suryansh Purwar
Sent: Monday, July 22, 2013 4:07 PM
To: solr-user@lucene.apache.org
Subject: how number of indexed fields effect performance
Hi,
We have a two shard solrcloud cluster with each shard allocated 3
separate
machines. We do complex queries involving a number of filter queries
coupled with group queries and faceting. All of our machines are 64 bit
with 32 gb ram. Our index size is around 10gb with around 8,00,000
documents. We have around 1000 indexed fields per document. 6gb of
memeory
is allocated to tomcat under which solr is running on each of the six
machines. We have a zookeeper ensemble consisting of 3 zookeeper
instances
running on 3 of the six machines with 4gb memory allocated to each of the
zookeeper instance. First solr start taking too much time with "Broken
pipe
exception because of timeout from client side" coming again and again,
then
after sometime a whole shard goes down with one machine at at time
followed
by other machines. Is having 1000 fields indexed with each document
resulting in this problem? If it is so, what would be the ideal number of
indexed fields in such environment.
Regards,
Suryansh
