OK, so I copied my index and ran solr3.1 against it.
Qtime dropped, from about 40s to 17s! This is good news, but still longer
than i hoped for.
I tried to do the same text with 4.0, but i'm getting
IndexFormatTooOldException since my index was created using 1.4.1. Is my
only chance to test this
On Thu, Apr 21, 2011 at 9:24 AM, Ofer Fort o...@tra.cx wrote:
Another strange behavior is that the Qtime seems pretty stable, no matter
how many object match my query. 200K and 20K both take about 17s.
I would have guessed that since the time is going over all the terms of all
the subset
Not sure i fully understand,
If facet.method=enum steps over all terms in the index for that field,
than what does setting the q=field:subset do? if i set the q=*:*, than how
do i get the frequency only on my subset?
Ofer
On Thu, Apr 21, 2011 at 4:40 PM, Yonik Seeley
On Thu, Apr 21, 2011 at 9:44 AM, Ofer Fort o...@tra.cx wrote:
Not sure i fully understand,
If facet.method=enum steps over all terms in the index for that field,
than what does setting the q=field:subset do? if i set the q=*:*, than how
do i get the frequency only on my subset?
It's an
I see, thanks.
So if I would want to implement something that would fit my needs, would
going through the subset of documents and counting all the terms in each
one, would be faster? and easier to implement?
On Thu, Apr 21, 2011 at 5:36 PM, Yonik Seeley yo...@lucidimagination.comwrote:
On Thu,
On Thu, Apr 21, 2011 at 10:41 AM, Ofer Fort o...@tra.cx wrote:
I see, thanks.
So if I would want to implement something that would fit my needs, would
going through the subset of documents and counting all the terms in each
one, would be faster? and easier to implement?
That's not just your
So if i want to use the facet.method=fc, is there a way to speed it up? and
remove the bucket size limitation?
On Thu, Apr 21, 2011 at 5:58 PM, Yonik Seeley yo...@lucidimagination.comwrote:
On Thu, Apr 21, 2011 at 10:41 AM, Ofer Fort o...@tra.cx wrote:
I see, thanks.
So if I would want to
On Thu, Apr 21, 2011 at 11:15 AM, Ofer Fort o...@tra.cx wrote:
So if i want to use the facet.method=fc, is there a way to speed it up? and
remove the bucket size limitation?
Not really - else we would have done it already ;-)
We don't really have great methods for faceting on full-text fields
Well, it was worth the try;-)
But will using the facet.method=fc, will reducing the subset size
reduce the time and memory? Meaning is it an O( ndocs of the set)?
Thanks
On Thursday, April 21, 2011, Yonik Seeley yo...@lucidimagination.com wrote:
On Thu, Apr 21, 2011 at 11:15 AM, Ofer Fort
On Thu, Apr 21, 2011 at 6:25 PM, Ofer Fort o...@tra.cx wrote:
Well, it was worth the try;-)
But will using the facet.method=fc, will reducing the subset size
reduce the time and memory? Meaning is it an O( ndocs of the set)?
facet.method=fc builds a multi-valued fieldcache like structure
So I'm guessing my best approach now would be to test trunk, and hope
that as 3.1 cut the performance in half, trunk will do the same
Thanks for the info
Ofer
On Friday, April 22, 2011, Yonik Seeley yo...@lucidimagination.com wrote:
On Thu, Apr 21, 2011 at 6:25 PM, Ofer Fort o...@tra.cx wrote:
On Thu, Apr 21, 2011 at 6:34 PM, Ofer Fort o...@tra.cx wrote:
So I'm guessing my best approach now would be to test trunk, and hope
that as 3.1 cut the performance in half, trunk will do the same
Trunk prob won't be much better... but the bulkpostings branch
possibly could be.
-Yonik
Ok, I'll give it a try, as this is a server I am willing to risk.
How is the competability between solrj of bulkpostings, trunk, 3.1 and 1.4.1?
On Friday, April 22, 2011, Yonik Seeley yo...@lucidimagination.com wrote:
On Thu, Apr 21, 2011 at 6:34 PM, Ofer Fort o...@tra.cx wrote:
So I'm guessing
On Thu, Apr 21, 2011 at 6:50 PM, Ofer Fort o...@tra.cx wrote:
Ok, I'll give it a try, as this is a server I am willing to risk.
How is the competability between solrj of bulkpostings, trunk, 3.1 and 1.4.1?
bulkpostings, trunk, and 3.1 should all be relatively solrj
compatible. But the SolrJ
Ok, thanks
On Friday, April 22, 2011, Yonik Seeley yo...@lucidimagination.com wrote:
On Thu, Apr 21, 2011 at 6:50 PM, Ofer Fort o...@tra.cx wrote:
Ok, I'll give it a try, as this is a server I am willing to risk.
How is the competability between solrj of bulkpostings, trunk, 3.1 and 1.4.1?
I think faceting is probably the best way to do that, indeed. It might be slow,
but it's kind of set up for exactly that case, I can't imagine any other
technique being faster -- there's stuff that has to be done to look up the info
you want.
BUT, I see your problem: don't use
thanks, but that's what i started with, but it took an even longer time and
threw this:
Approaching too many values for UnInvertedField faceting on field 'text' :
bucket size=15560140
Approaching too many values for UnInvertedField faceting on field 'text :
bucket size=15619075
Exception during
seems like the facet search is not all that suited for a full text field. (
http://search.lucidimagination.com/search/document/178f1a82ff19070c/solr_severe_error_when_doing_a_faceted_search#16562790cda76197
)
Maybe i should go another direction. I think that the HighFreqTerms
approach, just not
: thanks, but that's what i started with, but it took an even longer time and
: threw this:
: Approaching too many values for UnInvertedField faceting on field 'text' :
: bucket size=15560140
: Approaching too many values for UnInvertedField faceting on field 'text :
: bucket size=15619075
:
On Wed, Apr 20, 2011 at 7:34 PM, Chris Hostetter
hossman_luc...@fucit.org wrote:
: thanks, but that's what i started with, but it took an even longer time and
: threw this:
: Approaching too many values for UnInvertedField faceting on field 'text' :
: bucket size=15560140
: Approaching too
Thanks
but i've disabled the cache already, since my concern is speed and i'm
willing to pay the price (memory), and my subset are not fixed.
Does the facet search do any extra work that i don't need, that i might be
able to disable (either by a flag or by a code change),
Somehow i feel, or rather
BTW,
i'm using solr 1.4.1, does 3.1 or 4.0 contain any performance improvements
that will make a difference as far as facet search?
thanks again
Ofer
On Thu, Apr 21, 2011 at 2:45 AM, Ofer Fort o...@tra.cx wrote:
Thanks
but i've disabled the cache already, since my concern is speed and i'm
On Wed, Apr 20, 2011 at 7:45 PM, Ofer Fort o...@tra.cx wrote:
Thanks
but i've disabled the cache already, since my concern is speed and i'm
willing to pay the price (memory)
Then you should not disable the cache.
, and my subset are not fixed.
Does the facet search do any extra work that i
my documents are user entries, so i'm guessing they vary a lot.
Tomorrow i'll try 3.1 and also 4.0, and see if they have an improvement.
thanks guys!
On Thu, Apr 21, 2011 at 3:02 AM, Yonik Seeley yo...@lucidimagination.comwrote:
On Wed, Apr 20, 2011 at 7:45 PM, Ofer Fort o...@tra.cx wrote:
24 matches
Mail list logo
