Working for a week now, no signs of fatigue.
Many thanks for all the hints
R
--
View this message in context:
http://lucene.472066.n3.nabble.com/SOLR-hangs-update-timeout-please-help-tp3863851p3899004.html
Sent from the Solr - User mailing list archive at Nabble.com.
/SOLR-hangs-update-timeout-please-help-tp3863851p3881674.html
Sent from the Solr - User mailing list archive at Nabble.com.
Lance, I know there are many variables that's why I'm asking where to start
and what to check.
Updates are sent every 5-7 seconds, each update contains between 1 and 50
docs. Commit is done every time (on each update).
Currently queries aren't very frequent - about 1 query every 3-5 seconds,
but
5-7 seconds- there's the problem. If you want to have documents
visible for search within that time, you want to use the trunk and
near-real-time search. A hard commit does several hard writes to the
disk (with the fsync() system call). It does not run smoothly at that
rate. It is no surprise that
That's bad news.
If 5-7 seconds is not safe then what is the safe interval for updates?
Near real-time is not for me as it works only when querying by document Id
- this doesn't solve anything in my case. I just want the index to be
updated in real-time, 30-40 seconds delay is acceptable but not
Could be garbage collection. Could be larger and larger merges. At some point
your commit will cause all segments to be merged. It's likely that what's
happening is you need to hit the magic combination of events, particularly
the problem of too many warming searchers.
So, look at your log files
On 3/29/2012 2:49 AM, Rafal Gwizdala wrote:
That's bad news.
If 5-7 seconds is not safe then what is the safe interval for updates?
Near real-time is not for me as it works only when querying by document Id
- this doesn't solve anything in my case. I just want the index to be
updated in
Have you tried using Solr 3.5 with RankingAlgorithm 1.4.1 ? Has NRT
support and is very fast, updates about 5000 documents in about 490 ms
(while updating 1m docs in batches of 5k).
You can get more info from here:
http://solr-ra.tgels.com/wiki/en/Near_Real_Time_Search_ver_3.x
Regards,
On Thu, Mar 29, 2012 at 4:24 AM, Lance Norskog goks...@gmail.com wrote:
5-7 seconds- there's the problem. If you want to have documents
visible for search within that time, you want to use the trunk and
near-real-time search. A hard commit does several hard writes to the
disk (with the fsync()
If you must have real-time search, you might look at systems that are designed
to do that. MarkLogic isn't free, but it is fast and real-time. You can use
their no-charge Express license for development and prototyping:
http://developer.marklogic.com/express
OK, back to Solr.
wunder
Search
Guys, thanks for all the suggestions
I will be trying them, one at a time. Imho it's too early to give up and
look for another tool, I'll try to work on configuration and see what
happens.
The NRT looks quite promising, there are also tons of config options to
change.
As for now, I have made the
More memory is not necessarily better, it can lead to longer, more
intense garbage collections that cause things to stop. You might
also consider lowering your memory allocation, but 2G is really not
all that much so I somewhat doubt it's a problem but thought I'd
mention it.
Best
Erick
On Thu,
On Thu, Mar 29, 2012 at 1:50 PM, Rafal Gwizdala
rafal.gwizd...@gmail.com wrote:
Below i'm pasting the thread dump taken when the update was hung (it's also
attached to the first message of this topic)
Interesting...
It looks like there's only one thread in solr code (the one generating
the
Yonik, I didn't say there was an update request active at the moment the
thread dump was made, only that previous update requests failed with a
timeout. So maybe this is the missing piece.
I didn't enable nio with Jetty, probably it's there by default. Disabling
it is the next thing to check.
If
Oops... my previous replies accidentally went off-list. I'll cut-n-paste below.
OK, so it looks like there is probably no bug here - it's simply that
commits can sometimes take a long time and updates were blocked during
that time (and would have succeeded eventually except the jetty
timeout was
How often are updates? And when are commits? How many CPUs? How much
query load? There are so many variables.
Check the mailing list archives and Solr issues, there might be a
similar problem already discussed. Also, attachments do not work with
Apache mailing lists. (Well, ok, they work for
16 matches
Mail list logo
