not only the old index. automatic refresh ? XD
--
View this message in context:
http://lucene.472066.n3.nabble.com/Tuning-Solr-caches-with-high-commit-rates-NRT-tp1461275p2005738.html
Sent from the Solr - User mailing list archive at Nabble.com.
-with-high-commit-rates-NRT-tp1461275p2005738.html
Sent from the Solr - User mailing list archive at Nabble.com.
Many thanks, Peter K. for posting up on the wiki - great!
Yes, fc = field cache. Field Collapsing is something very nice indeed,
but is entirely different.
As Erik mentions in the wiki post, using per-segment faceting can be a
huge boon to performance. It does require the latest Solr trunk build
Just in case someone is interested:
I put the emails of Peter Sturge with some minor edits in the wiki:
http://wiki.apache.org/solr/NearRealtimeSearchTuning
I found myself search the thread again and again ;-)
Feel free to add and edit content!
Regards,
Peter.
Hi Erik,
I thought this
Awesome. I'm not sure his point 1 about facet.method=enum is still valid
in Solr 1.4+. The fc facet.method was changed significantly in 1.4,
and generally no longer takes a lot of memory -- for facets with many
unique values, method fc in fact should take less than enum, I think?
Peter
Hi Jonathan,
I am too using fc because it simply was faster. Not sure if this can be
applied in general.
I will add this info to the wiki.
Regards,
Peter.
Awesome. I'm not sure his point 1 about facet.method=enum is still
valid in Solr 1.4+. The fc facet.method was changed significantly
=nl.e036'
EARTH has a Right To Life,
otherwise we all die.
- Original Message
From: Peter Karich peat...@yahoo.de
To: solr-user@lucene.apache.org
Sent: Mon, November 15, 2010 1:37:00 PM
Subject: Re: Tuning Solr caches with high commit rates (NRT)
Hi Jonathan,
I am too using fc
://blogs.techrepublic.com.com/security/?p=4501tag=nl.e036'
EARTH has a Right To Life,
otherwise we all die.
- Original Message
From: Peter Karich peat...@yahoo.de
To: solr-user@lucene.apache.org
Sent: Mon, November 15, 2010 1:37:00 PM
Subject: Re: Tuning Solr caches with high commit rates (NRT)
Hi
, November 15, 2010 1:37:00 PM
Subject: Re: Tuning Solr caches with high commit rates (NRT)
Hi Jonathan,
I am too using fc because it simply was faster. Not sure if this can be applied
in general.
I will add this info to the wiki.
Regards,
Peter.
Awesome. I'm not sure his point 1 about facet.method
(10/11/16 6:43), Dennis Gearon wrote:
fc='field collapsing'?
fc of facet.method=fc stands for Lucene's FieldCache.
enum of facet.method=enum stands for Lucene's TermEnum.
Usually, you do not need to set facet.method because Solr
automatically uses most appropriate facet method for
each field
Koji Sekiguchi wrote:
Usually, you do not need to set facet.method because Solr
automatically uses most appropriate facet method for
each field type:
boolean: TermEnum
multiValued/tokenized: UnInvertedField
other than those above: FieldCache
As I understand it, in Solr 1.4, (and I may NOT
(10/11/16 8:36), Jonathan Rochkind wrote:
In Solr 1.4, facet.method=enum DOES work on multi-valued fields, I'm pretty
certain.
Correct, and I didn't say that facet.method=enum doesn't work for multiValued/tokenized field in my
previous mail.
I think Koji's explanation is based on before
Hi,
It's great to see such a fantastic response to this thread - NRT is
alive and well!
I'm hoping to collate this information and add it to the wiki when I
get a few free cycles (thanks Erik for the heads up).
In the meantime, I thought I'd add a few tidbits of additional
information that
Subject: Re: Tuning Solr caches with high commit rates (NRT)
To: solr-user@lucene.apache.org
Date: Friday, September 17, 2010, 2:18 AM
Hi,
It's great to see such a fantastic response to this thread
- NRT is
alive and well!
I'm hoping to collate this information and add it to the
wiki
--- On Fri, 9/17/10, Peter Sturge peter.stu...@gmail.com wrote:
From: Peter Sturge peter.stu...@gmail.com
Subject: Re: Tuning Solr caches with high commit rates (NRT)
To: solr-user@lucene.apache.org
Date: Friday, September 17, 2010, 2:18 AM
Hi,
It's great to see such a fantastic
:
From: Erick Erickson erickerick...@gmail.com
Subject: Re: Tuning Solr caches with high commit rates (NRT)
To: solr-user@lucene.apache.org
Date: Friday, September 17, 2010, 10:05 AM
Near Real Time...
Erick
On Fri, Sep 17, 2010 at 12:55 PM, Dennis Gearon gear...@sbcglobal.netwrote:
BTW
Does Solr use Lucene NRT?
--- On Fri, 9/17/10, Erick Erickson erickerick...@gmail.com wrote:
From: Erick Erickson erickerick...@gmail.com
Subject: Re: Tuning Solr caches with high commit rates (NRT)
To: solr-user@lucene.apache.org
Date: Friday, September 17, 2010, 1:05 PM
Near Real Time
wrote:
From: Erick Erickson erickerick...@gmail.com
Subject: Re: Tuning Solr caches with high commit rates (NRT)
To: solr-user@lucene.apache.org
Date: Friday, September 17, 2010, 1:05 PM
Near Real Time...
Erick
On Fri, Sep 17, 2010 at 12:55 PM, Dennis Gearon gear...@sbcglobal.netwrote:
BTW
Hi Peter,
this scenario would be really great for us - I didn't know that this is
possible and works, so: thanks!
At the moment we are doing similar with replicating to the readonly
instance but
the replication is somewhat lengthy and resource-intensive at this
datavolume ;-)
Regards,
Peter.
Rutherglen jason.rutherg...@gmail.com wrote:
From: Jason Rutherglen jason.rutherg...@gmail.com
Subject: Re: Tuning Solr caches with high commit rates (NRT)
To: solr-user@lucene.apache.org
Date: Sunday, September 12, 2010, 7:52 PM
Yeah there's no patch... I think
Yonik can write it. :-) Yah
The balanced segment merging is a really cool idea. I'll definetely
have a look at this, thanks!
One thing I forgot to mention in the original post is we use a
mergeFactor of 25. Somewhat on the high side, so that incoming commits
aren't trying to merge new data into large segments.
25 is a good
1. You can run multiple Solr instances in separate JVMs, with both
having their solr.xml configured to use the same index folder.
You need to be careful that one and only one of these instances will
ever update the index at a time. The best way to ensure this is to use
one for writing only,
and
Hi Erik,
I thought this would be good for the wiki, but I've not submitted to
the wiki before, so I thought I'd put this info out there first, then
add it if it was deemed useful.
If you could let me know the procedure for submitting, it probably
would be worth getting it into the wiki (couldn't
: Jason Rutherglen jason.rutherg...@gmail.com
Subject: Re: Tuning Solr caches with high commit rates (NRT)
To: solr-user@lucene.apache.org
Date: Sunday, September 12, 2010, 7:52 PM
Yeah there's no patch... I think
Yonik can write it. :-) Yah... The
Lucene version shouldn't matter. The distributed
...@gmail.com wrote:
From: Jason Rutherglen jason.rutherg...@gmail.com
Subject: Re: Tuning Solr caches with high commit rates (NRT)
To: solr-user@lucene.apache.org
Date: Sunday, September 12, 2010, 7:52 PM
Yeah there's no patch... I think
Yonik can write it. :-) Yah... The
Lucene version shouldn't
Willnauer simon.willna...@googlemail.com
Subject: Re: Tuning Solr caches with high commit rates (NRT)
To: solr-user@lucene.apache.org
Date: Monday, September 13, 2010, 1:33 AM
On Mon, Sep 13, 2010 at 8:02 AM,
Dennis Gearon gear...@sbcglobal.net
wrote:
BTW, what is a segment?
On the Lucene
Hi,
Below are some notes regarding Solr cache tuning that should prove
useful for anyone who uses Solr with frequent commits (e.g. 5min).
Environment:
Solr 1.4.1 or branch_3x trunk.
Note the 4.x trunk has lots of neat new features, so the notes here
are likely less relevant to the 4.x
Peter:
This kind of information is extremely useful to document, thanks! Do you
have the time/energy to put it up on the Wiki? Anyone can edit it by
creating
a logon. If you don't, would it be OK if someone else did it (with
attribution,
of course)? I guess that by bringing it up I'm volunteering
:
From: Peter Sturge peter.stu...@gmail.com
Subject: Tuning Solr caches with high commit rates (NRT)
To: solr-user@lucene.apache.org
Date: Sunday, September 12, 2010, 9:26 AM
Hi,
Below are some notes regarding Solr cache tuning that
should prove
useful for anyone who uses Solr with frequent
Peter,
thanks a lot for your in-depth explanations!
Your findings will be definitely helpful for my next performance
improvement tests :-)
Two questions:
1. How would I do that:
or a local read-only instance that reads the same core as the indexing
instance (for the latter, you'll need
Peter,
Are you using per-segment faceting, eg, SOLR-1617? That could help
your situation.
On Sun, Sep 12, 2010 at 12:26 PM, Peter Sturge peter.stu...@gmail.com wrote:
Hi,
Below are some notes regarding Solr cache tuning that should prove
useful for anyone who uses Solr with frequent commits
Hi Jason,
I've tried some limited testing with the 4.x trunk using fcs, and I
must say, I really like the idea of per-segment faceting.
I was hoping to see it in 3.x, but I don't see this option in the
branch_3x trunk. Is your SOLR-1606 patch referred to in SOLR-1617 the
one to use with 3.1?
Bravo!
Other tricks: here is a policy for deciding when to merge segments that
attempts to balance merging with performance. It was contributed by
LinkedIn- they also run indexsearch in the same instance (not Solr, a
different Lucene app).
Thanks, Peter. This is really great info.
One setting I've found to be very useful for the problem of overlapping
onDeskSearchers is to reduce the value of maxWarmingSearchers in
solrconfig.xml. I've reduced this to 1, so if a slave is already busy doing
pre-warming, it won't try to also
Yeah there's no patch... I think Yonik can write it. :-) Yah... The
Lucene version shouldn't matter. The distributed faceting
theoretically can easily be applied to multiple segments, however the
way it's written for me is a challenge to untangle and apply
successfully to a working patch. Also
35 matches
Mail list logo
