Re: Does solr supports indexing of files other than UTF-8

2011-01-28 Thread Yonik Seeley
On Thu, Jan 27, 2011 at 3:51 AM, prasad deshpande prasad.deshpand...@gmail.com wrote: The size of docs can be huge, like suppose there are 800MB pdf file to index it I need to translate it in UTF-8 and then send this file to index. PDF is binary AFAIK... you shouldn't need to do any charset

Re: Searching for negative numbers very slow

2011-01-28 Thread Yonik Seeley
On Thu, Jan 27, 2011 at 6:32 PM, Simon Wistow si...@thegestalt.org wrote: If I do        qt=dismax    fq=uid:1 (or any other positive number) then queries are as quick as normal - in the 20ms range. However, any of        fq=uid:\-1 or    fq=uid:[* TO -1] or    fq=uid:[-1 to -1]

Re: edismax vs dismax

2011-01-28 Thread Yonik Seeley
On Fri, Jan 28, 2011 at 3:00 PM, Thumuluri, Sai sai.thumul...@verizonwireless.com wrote: I recently upgraded to Solr 1.4.1 from Solr 1.3 and with the upgrade used edismax query parser. Here is my solrconfig.xml . When I search for mw verification and payment information - I get no results with

Re: Local param tag voodoo ?

2011-01-20 Thread Yonik Seeley
On Thu, Jan 20, 2011 at 4:59 AM, Xavier SCHEPLER xavier.schep...@sciences-po.fr wrote: Ok, I tryed to use nested queries this way: wt=jsonindent=truefl=qFRq=sarkozy _query_:{!tag=test}chiracfacet=truefacet.field={!ex=test}studyDescriptionId It resulted in this error: facet_counts:{  

Re: utf-8 tomcat and solr problem

2011-01-06 Thread Yonik Seeley
On Thu, Jan 6, 2011 at 2:23 AM, Julian Hille julian.hi...@netimpact.de wrote: Hi, if i search for a german umlaut like ä or ö i get something like weird conversions from latin to utf in query response. The encoding of the result is ok, but not the you queried for this part. There is my ä

Re: Including Small Amounts of New Data in Searches (MultiSearcher ?)

2011-01-06 Thread Yonik Seeley
On Thu, Jan 6, 2011 at 12:37 PM, Stephen Boesch java...@gmail.com wrote: Solr/lucene newbie here .. We would like searches against a solr/lucene index to immediately be able to view data that was added.  I stress small amount of new data given that any significant amount would require

Re: Will Result Grouping return documents that don't contain the specified group.field?

2011-01-06 Thread Yonik Seeley
On Thu, Jan 6, 2011 at 5:55 PM, Andy angelf...@yahoo.com wrote: So by default Solr will not return documents that don't contain the specified group.field? Solr will. Documents without a value for that field should be grouped under the null value. -Yonik http://www.lucidimagination.com

Re: Replication: the web application [/solr] .. likely to create a memory leak

2011-01-04 Thread Yonik Seeley
On Tue, Jan 4, 2011 at 9:34 AM, Robert Muir rcm...@gmail.com wrote:    [junit] WARNING: test class left thread running: Thread[MultiThreadedHttpConnectionManager cleanup,5,main] I suppose we should move MultiThreadedHttpConnectionManager to CoreContainer. -Yonik http://www.lucidimagination.com

Re: SpatialTierQueryParserPlugin Loading Error

2010-12-28 Thread Yonik Seeley
On Tue, Dec 28, 2010 at 8:54 PM, Adam Estrada estrada.a...@gmail.com wrote: I would gladly update this page if I could just get it working. http://wiki.apache.org/solr/SpatialSearch Everything on that wiki page should work w/o patches on trunk. I just ran through all of the examples, and

Re: Map failed at getSearcher

2010-12-24 Thread Yonik Seeley
On Fri, Dec 24, 2010 at 10:23 AM, Robert Muir rcm...@gmail.com wrote: hmm, i think you are actually running out of virtual address space, even on 64-bit! I don't know if there are any x86 processors that allow 64 bits of address space yet. AFAIK, they are mostly 48 bit.

Re: White space in facet values

2010-12-22 Thread Yonik Seeley
On Wed, Dec 22, 2010 at 9:53 AM, Dyer, James james.d...@ingrambook.com wrote: The phrase solution works as does escaping the space with a backslash:   fq=Product:Electric\ Guitar ... actually a lot of characters need to be escaped like this (amperstands and parenthesis come to mind)... One

Re: Faceting memory requirements

2010-12-21 Thread Yonik Seeley
On Tue, Dec 21, 2010 at 4:02 PM, Rok Rejc rokrej...@gmail.com wrote: Dear all, I have created an index with aprox. 1.1 billion of documents (around 500GB) running on Solr 1.4.1. (64 bit JVM). I want to enable faceted navigation on am int field, which contains around 250 unique values.

Re: Why does Solr commit block indexing?

2010-12-17 Thread Yonik Seeley
On Fri, Dec 17, 2010 at 8:05 AM, Grant Ingersoll gsing...@apache.org wrote: I'm not sure if there is a issue open, but I know I've talked w/ Yonik about this and a few other changes to the DirectUpdateHandler2 in the past.  It does indeed need to be fixed. It stems from the APIs that were

Re: WARNING: re-index all trunk indices!

2010-12-17 Thread Yonik Seeley
On Fri, Dec 17, 2010 at 11:18 AM, Michael McCandless luc...@mikemccandless.com wrote: If you are using Lucene's trunk (nightly build) release, read on... I just committed a change (for LUCENE-2811) that changes the index format on trunk, thus breaking (w/ likely strange exceptions on reading

Re: bulk commits

2010-12-16 Thread Yonik Seeley
On Thu, Dec 16, 2010 at 3:06 PM, Dennis Gearon gear...@sbcglobal.net wrote: That easy, huh? Heck, this gets better and better. BTW, how about escaping? The CSV escaping? It's configurable to allow for loading different CSV dialects. http://wiki.apache.org/solr/UpdateCSV By default it uses

Re: Memory use during merges (OOM)

2010-12-16 Thread Yonik Seeley
On Thu, Dec 16, 2010 at 5:51 AM, Michael McCandless luc...@mikemccandless.com wrote: If you are doing false deletions (calling .updateDocument when in fact the Term you are replacing cannot exist) it'd be best if possible to change the app to not call .updateDocument if you know the Term

Re: Faceted Search Slows Down as index gets larger

2010-12-16 Thread Yonik Seeley
=solr.FastLRUCache   size=512   initialSize=512   autowarmCount=0/ queryResultCache   class=solr.LRUCache   size=512   initialSize=512   autowarmCount=0/ Any help is appreciated. On Sun, Jun 6, 2010 at 8:54 PM, Yonik Seeley yo...@lucidimagination.com wrote: On Sun

Re: OutOfMemory GC: GC overhead limit exceeded - Why isn't WeakHashMap getting collected?

2010-12-13 Thread Yonik Seeley
On Mon, Dec 13, 2010 at 8:47 PM, John Russell jjruss...@gmail.com wrote: Wow, you read my mind.  We are committing very frequently.  We are trying to get as close to realtime access to the stuff we put in as possible.  Our current commit time is... ahem every 4 seconds. Is that insane?

Re: OutOfMemory GC: GC overhead limit exceeded - Why isn't WeakHashMap getting collected?

2010-12-13 Thread Yonik Seeley
On Mon, Dec 13, 2010 at 9:27 PM, Jonathan Rochkind rochk...@jhu.edu wrote: Yonik, how will maxWarmingSearchers in this scenario effect replication?  If a slave is pulling down new indexes so quickly that the warming searchers would ordinarily pile up, but maxWarmingSearchers is set to 1

Re: Userdefined Field type - Faceting

2010-12-13 Thread Yonik Seeley
Perhaps try overriding indexedToReadable() also? -Yonik http://www.lucidimagination.com On Mon, Dec 13, 2010 at 10:00 PM, Viswa S svis...@hotmail.com wrote: Hello, We implemented an IP-Addr field type which internally stored the ips as hex-ed string (e.g. 192.2.103.29 will be stored as

Re: Shards + dismax - scoring process?

2010-12-11 Thread Yonik Seeley
On Sat, Dec 11, 2010 at 2:18 AM, bbarani bbar...@gmail.com wrote: Also, if I try to sort the query result from shards.. will sorting happens on the consolidated data or on each individual core data? Both - to find the top 10 docs by any sort, the top 10 docs from each shard are collected and

Re: Map size must not be negative with spatial results + php serialized

2010-12-08 Thread Yonik Seeley
On Wed, Dec 8, 2010 at 9:45 AM, Markus Jelsma markus.jel...@openindex.io wrote: I know, but since it's an Apache component throwing the exception, i'd figure someone just might know more about this. That's fine - it could be a Solr bug too. IMO, solr-user traffic just needs to be solr related

Webcast: Better Search Results Faster with Apache Solr and LucidWorks Enterprise

2010-12-08 Thread Yonik Seeley
We're holding a free webinar about relevancy enhancements in our commercial version of Solr. Details below. -Yonik http://www.lucidimagination.com - Join us for a free technical webcast Better Search Results Faster with

Re: How to handle multivalued hierarchical facets?

2010-12-08 Thread Yonik Seeley
Hoss had a great webinar on faceting that also covered how you could do hierarchical. http://www.lucidimagination.com/solutions/webcasts/faceting See taxonomy facets, about 28 minutes in. -Yonik http://www.lucidimagination.com On Wed, Dec 8, 2010 at 5:28 PM, Andy angelf...@yahoo.com wrote: I

Re: Changing a solr schema from non-stored to stored on the fly

2010-12-08 Thread Yonik Seeley
On Wed, Dec 8, 2010 at 6:07 PM, Kaktu Chakarabati jimmoe...@gmail.com wrote: Can I do this? i.e change that value in schema, and then incrementally re-index documents to populate it? would that work? what would be returned if at all for documents that werent re-indexed post-schema change?

Re: Field Collapsing - sort by group count, get total groups

2010-12-07 Thread Yonik Seeley
On Tue, Dec 7, 2010 at 7:03 AM, ssetem sse...@googlemail.com wrote: I wondered if it is possible to sort groups by the total within the group, and to bring back total amount groups? That is planned, but not currently implemented. You can use faceting to get both totals and sort by highest total

Re: Field Collapsing - sort by group count, get total groups

2010-12-07 Thread Yonik Seeley
On Tue, Dec 7, 2010 at 9:07 AM, ssetem sse...@googlemail.com wrote: Thanks for the reply, How would i get the total amount of possible facets(non zero), I've searched around but have no luck. Only current way would be to request them all. Just like field collapsing, this is a number we don't

Re: autocommit commented out -- what is the default?

2010-12-04 Thread Yonik Seeley
On Sat, Dec 4, 2010 at 10:36 AM, Brian Whitman br...@echonest.com wrote: Hi, if you comment out the block in solrconfig.xml !--   autoCommit      maxDocs1/maxDocs      maxTime60/maxTime    /autoCommit -- Does this mean that (a) commits never happen automatically or (b) some

Re: ramBufferSizeMB not reflected in segment sizes in index

2010-12-02 Thread Yonik Seeley
On Wed, Dec 1, 2010 at 3:01 PM, Shawn Heisey s...@elyograg.org wrote: I have seen this.  In Solr 1.4.1, the .fdt, .fdx, and the .tv* files do not segment, but all the other files do.  I can't remember whether it behaves the same under 3.1, or whether it also creates these files in each segment.

Re: ArrayIndexOutOfBoundsException for query with rows=0 and sort param

2010-11-30 Thread Yonik Seeley
On Tue, Nov 30, 2010 at 8:24 AM, Martin Grotzke martin.grot...@googlemail.com wrote: Still I'm wondering, why this issue does not occur with the plain example solr setup with 2 indexed docs. Any explanation? It's an old option you have in your solrconfig.xml that causes a different code path to

Re: entire farm fails at the same time with OOM issues

2010-11-30 Thread Yonik Seeley
On Tue, Nov 30, 2010 at 6:04 PM, Robert Petersen rober...@buy.com wrote: My question is this.  Why in the world would all of my slaves, after running fine for some days, suddenly all at the exact same minute experience OOM heap errors and go dead? If there is no change in query traffic when

Re: Preventing index segment corruption when windows crashes

2010-11-29 Thread Yonik Seeley
On Mon, Nov 29, 2010 at 10:46 AM, Peter Sturge peter.stu...@gmail.com wrote: If a Solr index is running at the time of a system halt, this can often corrupt a segments file, requiring the index to be -fix'ed by rewriting the offending file. Really? That shouldn't be possible (if you mean the

Re: solr admin

2010-11-29 Thread Yonik Seeley
On Mon, Nov 29, 2010 at 8:02 PM, Ahmet Arslan iori...@yahoo.com wrote: in Solr admin (http://localhost:8180/services/admin/) I can specify something like: +category_id:200 +xxx:300 but how can I specify a sort option? sort:category_id+asc There is an [FULL INTERFACE] /admin/form.jsp link

Re: geospatial

2010-11-24 Thread Yonik Seeley
On Wed, Nov 24, 2010 at 2:41 PM, Dennis Gearon gear...@sbcglobal.net wrote: What is the recommended Solr version and/or plugin combination to get geospatial search up and running the quickest and easiest? It depends on what capabilities you need. The current state of what is committed to

Re: Problem with synonyms

2010-11-22 Thread Yonik Seeley
On Sat, Nov 20, 2010 at 5:59 AM, sivaprasad sivaprasa...@echidnainc.com wrote: Even after expanding the synonyms also i am unable to get same results. What you are trying to do should work with index-time synonym expansion. Just make sure to remove the synonym filter at query time (or use a

Re: Problem with synonyms

2010-11-22 Thread Yonik Seeley
On Mon, Nov 22, 2010 at 10:29 AM, Yonik Seeley yo...@lucidimagination.com wrote: On Sat, Nov 20, 2010 at 5:59 AM, sivaprasad sivaprasa...@echidnainc.com wrote: Even after expanding the synonyms also i am unable to get same results. What you are trying to do should work with index-time

Re: Must require quote with single word token query?

2010-11-19 Thread Yonik Seeley
On Tue, Nov 16, 2010 at 10:28 PM, Chamnap Chhorn chamnapchh...@gmail.com wrote: I have one question related to single word token with dismax query. In order to be found I need to add the quote around the search query all the time. This is quite hard for me to do since it is part of full text

Re: Must require quote with single word token query?

2010-11-19 Thread Yonik Seeley
://www.lucidimagination.com On 11/19/10, Yonik Seeley yo...@lucidimagination.com wrote: On Tue, Nov 16, 2010 at 10:28 PM, Chamnap Chhorn chamnapchh...@gmail.com wrote: I have one question related to single word token with dismax query. In order to be found I need to add the quote around the search query all

result grouping / field collapsing changes

2010-11-16 Thread Yonik Seeley
We've recently added randomized testing for result grouping that resulted in finding + fixing a number of bugs. I've you've been using this feature, you should move to the latest trunk version. I've also added a section at the bottom of the wiki page to list current limitations.

Re: hash uniqueKey generation?

2010-11-16 Thread Yonik Seeley
On Tue, Nov 16, 2010 at 5:31 AM, Dennis Gearon gear...@sbcglobal.net wrote: hashing is not 100% guaranteed to produce unique values. But if you go to enough bits with a good hash function, you can get the odds lower than the odds of something else changing the value like cosmic rays flipping a

Re: hash uniqueKey generation?

2010-11-16 Thread Yonik Seeley
On Tue, Nov 16, 2010 at 9:05 PM, Dennis Gearon gear...@sbcglobal.net wrote: Read up on WikiPedia, but I believe that no Hash Function is much good above 50% of the address space it generates. 50% is way to high - collisions will happen before that. But given that something like MD5 has 128

Re: Solr Negative query

2010-11-15 Thread Yonik Seeley
On Mon, Nov 15, 2010 at 12:42 AM, Viswa S svis...@hotmail.com wrote: Apologies for starting a new thread again, my mailing list subscription didn't finalize till later than Yonik's response. Using Field1:Val1 AND (*:* NOT Field2:Val2) works, thanks. Does my original query Field1:Value1 AND

Re: Solr Negative query

2010-11-14 Thread Yonik Seeley
On Sun, Nov 14, 2010 at 4:17 AM, Leonardo Menezes leonardo.menez...@googlemail.com wrote: try Field1:Val1 AND (*:* NOT Field2:Val2), that shoud work ok That should be equivalent to Field1:Val1 -Field2:Val2 You only need the *:* trick if all of the clauses of a boolean query are negative.

Re: facetting when using field collapsing

2010-11-13 Thread Yonik Seeley
On Wed, Nov 10, 2010 at 9:12 AM, Lukas Kahwe Smith m...@pooteeweet.org wrote: The above wiki page seems to be out of date. Reading the comments in https://issues.apache.org/jira/browse/SOLR-236 it seems like group should be replaced with collapse. The Wiki page is not expansive, but I've

Re: facetting when using field collapsing

2010-11-13 Thread Yonik Seeley
On Sat, Nov 13, 2010 at 10:46 AM, Lukas Kahwe Smith m...@pooteeweet.org wrote: On 13.11.2010, at 10:30, Yonik Seeley wrote: On Wed, Nov 10, 2010 at 9:12 AM, Lukas Kahwe Smith m...@pooteeweet.org wrote: The above wiki page seems to be out of date. Reading the comments in https

Re: IndexableBinaryStringTools (was FieldCache)

2010-11-13 Thread Yonik Seeley
On Sat, Nov 13, 2010 at 1:50 PM, Steven A Rowe sar...@syr.edu wrote: Looks to me like the returned value is in a Solr-internal form of XML character escaping: \u is represented as #0; and \u0008 is represented as #8;.  (The escaping code is in

FAST ESP - Solr migration webinar

2010-11-11 Thread Yonik Seeley
We're holding a free webinar on migration from FAST to Solr. Details below. -Yonik http://www.lucidimagination.com = Solr To The Rescue: Successful Migration From FAST ESP to Open Source Search Based on Apache Solr

Re: solr 4.0 - pagination

2010-11-07 Thread Yonik Seeley
On Sun, Nov 7, 2010 at 10:55 AM, Papp Richard ccode...@gmail.com wrote:  this is fantastic, but can you tell any time it will be ready ? It already is ;-) Grab the latest trunk or the latest nightly build. -Yonik http://www.lucidimagination.com

Re: solr 4.0 - pagination

2010-11-07 Thread Yonik Seeley
On Sun, Nov 7, 2010 at 2:45 PM, Papp Richard ccode...@gmail.com wrote: Hi Yonik,  I've just tried the latest stable version from nightly build: apache-solr-4.0-2010-11-05_08-06-28.war  I have some concerns however: I have 3 documents; 2 in the first group, 1 in the 2nd group.  1. I got

Re: Negative or zero value for fieldNorm

2010-11-04 Thread Yonik Seeley
On Thu, Nov 4, 2010 at 8:04 AM, Markus Jelsma markus.jel...@openindex.io wrote: The question remains, why does the title field return a fieldNorm=0 for many queries? Because the index-time boost was set to 0 when the doc was indexed. I can't say how that happened... look to your indexing code.

Re: Negative or zero value for fieldNorm

2010-11-04 Thread Yonik Seeley
On Thu, Nov 4, 2010 at 9:51 AM, Markus Jelsma markus.jel...@openindex.io wrote: I've done some testing with the example docs and it behaves similar when there is a zero doc boost. Luke, however, does not show me the index-time boosts. Remember that the norm is a product of the length norm and

Re: Negative or zero value for fieldNorm

2010-11-03 Thread Yonik Seeley
Regarding Negative or zero value for fieldNorm, I don't see any negative fieldNorms here... just very small positive ones? Anyway the fieldNorm is the product of the lengthNorm and the index-time boost of the field (which is itself the product of the index time boost on the document and the index

Re: blacklist docs by uniqueKey

2010-11-03 Thread Yonik Seeley
On Wed, Nov 3, 2010 at 3:05 PM, Erick Erickson erickerick...@gmail.com wrote: How dynamic is this list? Is it feasable to add a field to your docs like blacklisteddocs, and at editorial's discretion add values to that field like app1, app2? At that point you can just filter them out via a

Re: Possible memory leaks with frequent replication

2010-11-02 Thread Yonik Seeley
On Tue, Nov 2, 2010 at 12:32 PM, Simon Wistow si...@thegestalt.org wrote: On Mon, Nov 01, 2010 at 05:42:51PM -0700, Lance Norskog said: You should query against the indexer. I'm impressed that you got 5s replication to work reliably. That's our current solution - I was just wondering if there

Re: big terms in UnInvertedField

2010-11-01 Thread Yonik Seeley
2010/11/1 Koji Sekiguchi k...@r.email.ne.jp: With solr example, using facet.field=text creates UnInvertedField for the text field in fieldValueCache. After that, I saw stats page and I was surprised at counters in *filterCache* were up: Do they cause of big words in UnInvertedField? Yes.

Re: Facet count of zero

2010-11-01 Thread Yonik Seeley
On Mon, Nov 1, 2010 at 12:55 PM, Tod listac...@gmail.com wrote: I'm trying to exclude certain facet results from a facet query.  It seems to work but rather than being excluded from the facet list its returned with a count of zero. If you don't want to see 0 counts, use facet.mincount=1

Re: solr 4.0 - pagination

2010-10-30 Thread Yonik Seeley
On Sat, Oct 30, 2010 at 12:22 PM, Papp Richard ccode...@gmail.com wrote:  I'm using Solr 4.0 with grouping (field collapsing), but unfortunately I can't solve the pagination. It's not implemented yet, but I'm working on that right now. -Yonik http://www.lucidimagination.com

Re: eDismax result differs from Dismax

2010-10-29 Thread Yonik Seeley
On Fri, Oct 29, 2010 at 9:30 AM, Ryan Walker r...@recruitmilitary.com wrote: We are launching a new version of our job board helping returning veterans find a civilian job, and we chose Solr and Sunspot[1] to power our search. We really didn't consider the power users in the HR world who are

Re: Custom Sorting in Solr

2010-10-29 Thread Yonik Seeley
On Fri, Oct 29, 2010 at 3:39 PM, Ezequiel Calderara ezech...@gmail.com wrote: Hi all guys! I'm in a weird situation here. We have index a set of documents which are ordered using a linked list (each documents has the reference of the previous and the next). Is there a way when sorting in the

Re: documentCache clarification

2010-10-29 Thread Yonik Seeley
On Fri, Oct 29, 2010 at 3:49 PM, Chris Hostetter hossman_luc...@fucit.org wrote: : This is a limitation in the SolrCache API. : The key into the cache does not contain rows, so the cache returns the : first 10 docs and increments it's hit count.  Then the cache user : (SolrIndexSearcher)

Re: documentCache clarification

2010-10-29 Thread Yonik Seeley
On Fri, Oct 29, 2010 at 4:21 PM, Chris Hostetter hossman_luc...@fucit.org wrote: : Why don't we just include the start rows (modulo the window size) in : the cache key? : : The implementation of equals() would be rather difficult... actually : impossible w/o abusing the semantics. : It

Re: SolrCore.getSearcher() and postCommit()

2010-10-29 Thread Yonik Seeley
On Fri, Oct 29, 2010 at 5:36 PM, Grant Ingersoll gsing...@apache.org wrote: Is it OK to call and increment a Searcher ref (i.e. SolrCore.getSearcher()) in a SolrEventListener.postCommit() hook as long as I decrement it when I am done?  I need to get a handle on an IndexReader so I can dump

Re: How to index long words with StandardTokenizerFactory?

2010-10-24 Thread Yonik Seeley
On Sun, Oct 24, 2010 at 10:47 AM, Sergey Bartunov sbos@gmail.com wrote: I did it just as you recommended. Solr indexes files around 15kb, but no more. The same effect was with patched constants Lucene also has max token sizes it can index. IIRC, lengths used to be stored inline with the

Re: How to index long words with StandardTokenizerFactory?

2010-10-24 Thread Yonik Seeley
On Sun, Oct 24, 2010 at 11:29 AM, Sergey Bartunov sbos@gmail.com wrote: It's a kind of research. There is no particular practical use case as far as I know. Do you know how to set all these max token lengths? It's a practical limit given how things are coded, not an arbitrary one. Given

Re: How to index long words with StandardTokenizerFactory?

2010-10-23 Thread Yonik Seeley
On Fri, Oct 22, 2010 at 12:07 PM, Sergey Bartunov sbos@gmail.com wrote: I'm trying to force solr to index words which length is more than 255 If the field is not a text field, the Solr's default analyzer is used, which currently limits the token to 256 bytes. Out of curiosity, what's your

Re: Date faceting +1MONTH problem

2010-10-22 Thread Yonik Seeley
On Fri, Sep 17, 2010 at 9:51 PM, Chris Hostetter hossman_luc...@fucit.org wrote: the default query parser doesn't support range queries with mixed upper/lower bound inclusion. This has just been added to trunk. Things like [0 TO 100} now work. -Yonik http://www.lucidimagination.com

Re: Date faceting +1MONTH problem

2010-10-22 Thread Yonik Seeley
On Fri, Oct 22, 2010 at 6:02 PM, Shawn Heisey s...@elyograg.org wrote: On 10/22/2010 3:01 PM, Yonik Seeley wrote: On Fri, Sep 17, 2010 at 9:51 PM, Chris Hostetter hossman_luc...@fucit.org  wrote:  the default query parser doesn't support range queries with mixed upper/lower bound inclusion

Re: why sorl is slower than lucene so much?

2010-10-21 Thread Yonik Seeley
2010/10/21 kafka0102 kafka0...@163.com: I found the problem's cause.It's the DocSetCollector. my fitler query result's size is about 300,so the DocSetCollector.getDocSet() is OpenBitSet. And 300 OpenBitSet.fastSet(doc) op is too slow. As I said in my other response to you, that's a

Re: why solr search is slower than lucene so much?

2010-10-20 Thread Yonik Seeley
Careful comparing apples to oranges ;-) For one, your lucene code doesn't retrieve stored fields. Did you try the solr request more than once (with a different q, but the same filters?) Also, by default, Solr independently caches the filters. This can be higher up-front cost, but a win when

Re: filter query from external list of Solr unique IDs

2010-10-15 Thread Yonik Seeley
On Fri, Oct 15, 2010 at 11:49 AM, Burton-West, Tom tburt...@umich.edu wrote: At the Lucene Revolution conference I asked about efficiently building a filter query from an external list of Solr unique ids. Yeah, I've thought about a special query parser and query to deal with this (relatively)

Re: facet.field :java.lang.NullPointerException

2010-10-15 Thread Yonik Seeley
This is https://issues.apache.org/jira/browse/SOLR-2142 I'll look into it soon. -Yonik http://www.lucidimagination.com On Fri, Oct 15, 2010 at 3:12 PM, Pradeep Singh pksing...@gmail.com wrote: Faceting blows up when the field has no data. And this seems to be random. Sometimes it will work

Re: Which version of Solr to use?

2010-10-14 Thread Yonik Seeley
On Thu, Oct 14, 2010 at 1:58 PM, Lukas Kahwe Smith m...@pooteeweet.org wrote: the current confusing list of branches is a result of the merge of the lucene and solr svn repositories. what baffpes me is that so far the countless plea's for at least a rough roadmap or even just explanation for

Re: Which version of Solr to use?

2010-10-14 Thread Yonik Seeley
On Thu, Oct 14, 2010 at 1:50 PM, Jonathan Rochkind rochk...@jhu.edu wrote: I'm kind of confused about Solr development plans in general, highlighted by this thread. I think 1.4.1 is the latest officially stable release, yes? Why is there both a 1.5 and a 3.x, anyway?  Not to mention a 4.x?  

Re: Which version of Solr to use?

2010-10-14 Thread Yonik Seeley
On Thu, Oct 14, 2010 at 2:39 PM, Jonathan Rochkind rochk...@jhu.edu wrote: Thanks Yonik!  So I gather that the 1.5 branch has essentially been abandoned, we can pretend it doesn't exist at all, it's been entirely superceded by the 3.x branch, with the changes made just for the purposes of

Re: Which version of Solr to use?

2010-10-14 Thread Yonik Seeley
On Thu, Oct 14, 2010 at 2:55 PM, Mike Squire mike.squ...@gmail.com wrote: As pointed out before it would be useful to have some kind of documented road map for development, and some kind of indication of how close certain versions are to release. Such things have proven to be very unreliable

Re: Faceting and first letter of fields

2010-10-14 Thread Yonik Seeley
On Thu, Oct 14, 2010 at 3:42 PM, Jonathan Rochkind rochk...@jhu.edu wrote: I believe that should work fine in Solr 1.4.1.  Creating a field with just first letter of author is definitely the right (possibly only) way to allow facetting on first letter of author's name. I have very voluminous

Re: LuceneRevolution - NoSQL: A comparison

2010-10-13 Thread Yonik Seeley
On Tue, Oct 12, 2010 at 12:11 PM, Jan Høydahl / Cominvent jan@cominvent.com wrote: I'm pretty sure the 2nd phase to fetch doc-summaries goes directly to same server as first phase. But what if you stick a LB in between? A related point - the load balancing implementation that's part of

Re: Spatial search in Solr 1.5

2010-10-13 Thread Yonik Seeley
On Wed, Oct 13, 2010 at 7:28 AM, PeterKerk vettepa...@hotmail.com wrote: Hi, Thanks for the quick reply :) I downloaded the latest version from the trunk. Got it up and running, and got the error below: Hopefully the QuickStart on the wiki all worked for you, but you only got the error when

Re: Spatial search in Solr 1.5

2010-10-13 Thread Yonik Seeley
On Wed, Oct 13, 2010 at 9:42 AM, PeterKerk vettepa...@hotmail.com wrote: Im now thinking I downloaded the wrong solr zip, I tried this one: https://hudson.apache.org/hudson/job/Solr-trunk/lastSuccessfulBuild/artifact/trunk/solr/dist/apache-solr-4.0-2010-10-12_08-05-48.zip In that example

Re: Spatial search in Solr 1.5

2010-10-13 Thread Yonik Seeley
On Wed, Oct 13, 2010 at 10:06 AM, PeterKerk vettepa...@hotmail.com wrote: haha ;) But so I DO have the right solr version? Anyways...I have added the lines you mentioned, what else can I do? The fact that the geolocation field does not show up in the results means that it's not getting

Re: Spatial search in Solr 1.5

2010-10-12 Thread Yonik Seeley
You may want to check the docs, which were recently updated to reflect the state of trunk: http://wiki.apache.org/solr/SpatialSearch -Yonik http://www.lucidimagination.com On Tue, Oct 12, 2010 at 7:49 PM, PeterKerk vettepa...@hotmail.com wrote: Hey Grant, Just came accross this post of

Re: Spatial search in Solr 1.5

2010-10-12 Thread Yonik Seeley
On Tue, Oct 12, 2010 at 8:07 PM, PeterKerk vettepa...@hotmail.com wrote: Ok, so does this actually say: for now you have to do calculations based on bounding box instead of great circle? I tried to make the documentation a little simpler... there's - geofilt... filters within a radius of d

Re: LuceneRevolution - NoSQL: A comparison

2010-10-11 Thread Yonik Seeley
On Mon, Oct 11, 2010 at 8:32 PM, Peter Keegan peterlkee...@gmail.com wrote: I listened with great interest to Grant's presentation of the NoSQL comparisons/alternatives to Solr/Lucene. It sounds like the jury is still out on much of this. Here's a use case that might favor using a NoSQL

Re: Upgrade to Solr 1.4, very slow at start up when loading all cores

2010-10-02 Thread Yonik Seeley
On Fri, Oct 1, 2010 at 5:42 PM, Renee Sun renee_...@mcafee.com wrote: Hi Yonik, I attached the solrconfig.xml to you in previous post, and we do have firstSearch and newSearch hook ups. I commented them out, all 130 cores loaded up in 1 minute, same as in solr 1.3.  total memory took about

Re: Local Solr, Spatial Search, and LatLonType clarification

2010-09-30 Thread Yonik Seeley
On Thu, Sep 30, 2010 at 1:09 PM, webdev1977 webdev1...@gmail.com wrote: 1.  I noticed that it said that the type of LatLongType can not be mulitvalued. Does that mean that I can not have multiple lat/lon values for one document. That means that if you want to have multiple points per document,

Re: Local Solr, Spatial Search, and LatLonType clarification

2010-09-30 Thread Yonik Seeley
On Thu, Sep 30, 2010 at 1:40 PM, webdev1977 webdev1...@gmail.com wrote: Or.. do you mean each field must have a unique name, but both be of type latLon(solr.LatLonType). work x,y/work homex,y/home Yes. If the statement directly above is true (I hope that it is not), how does one

Re: Local Solr, Spatial Search, and LatLonType clarification

2010-09-30 Thread Yonik Seeley
On Thu, Sep 30, 2010 at 1:48 PM, Yonik Seeley yo...@lucidimagination.com wrote: Dynamic field types.  You can configure it such that anything ending with _latlon is of type LatLonType. Perhaps we should do this in the example schema. Looks like we already have it: dynamicField name=*_p

Re: Upgrade to Solr 1.4, very slow at start up when loading all cores

2010-09-30 Thread Yonik Seeley
On Thu, Sep 30, 2010 at 10:41 AM, Renee Sun renee_...@mcafee.com wrote: Hi - I posted this problem but no response, I guess I need to post this in the Solr-User forum. Hopefully you will help me on this. We were running Solr 1.3 for long time, with 130 cores. Just upgrade to Solr 1.4, then

Re: Queries, Functions, and Params

2010-09-29 Thread Yonik Seeley
On Tue, Sep 28, 2010 at 6:08 PM, Robert Thayer robert.tha...@bankserv.com wrote: On the http://wiki.apache.org/solr/FunctionQuery page, the following query function is listed: q={!func}add($v1,$v2)v1=sqrt(popularity)v2=100.0 When run against the default solr instance, server returns the

Re: Conditional Function Queries

2010-09-28 Thread Yonik Seeley
On Tue, Sep 28, 2010 at 11:33 AM, Jan Høydahl / Cominvent jan@cominvent.com wrote: Have anyone written any conditional functions yet for use in Function Queries? Nope - but it makes sense and has been on my list of things to do for a long time. -Y http://lucenerevolution.org Lucene/Solr

Re: bi-grams for common terms - any analyzers do that?

2010-09-25 Thread Yonik Seeley
On Sat, Sep 25, 2010 at 8:21 PM, Jonathan Rochkind rochk...@jhu.edu wrote: Huh, okay, I didn't know that #2 happened at all. Can you explain or point me to documentation to explain when it happens?  I'm afraid I'm having trouble understanding  if the analyzer returns more than one position

Re: matches in result grouping

2010-09-23 Thread Yonik Seeley
2010/9/23 Koji Sekiguchi k...@r.email.ne.jp:  (10/09/23 18:14), Koji Sekiguchi wrote:  I'm using recent committed field collapsing / result grouping feature in trunk. I'm confusing matches parameter in the result at the second sample output of Wiki:

Re: Range query not working

2010-09-23 Thread Yonik Seeley
On Thu, Sep 23, 2010 at 4:30 PM, PeterKerk vettepa...@hotmail.com wrote: I have this in my query: q=*:*facet.query=location_rating_total:[3 TO 100] And this document: result name=response numFound=6 start=0 maxScore=1.0 - doc float name=score1.0/float str name=id1/str int

Re: Range query not working

2010-09-23 Thread Yonik Seeley
On Thu, Sep 23, 2010 at 5:44 PM, Jonathan Rochkind rochk...@jhu.edu wrote: The field type in a standard schema.xml that's defined as integer is NOT sortable. Right - before 1.4. There is no integer field type in 1.4 and beyond in the example schema. You can not sort on this and get what you

Re: multiple spatial values

2010-09-21 Thread Yonik Seeley
On Tue, Sep 21, 2010 at 12:12 PM, dan sutton danbsut...@gmail.com wrote: I was looking at the LatLonType and how it might represent multiple lon/lat values ... it looks to me like the lat would go in {latlongfield}_0_LatLon and the long in {latlongfield}_1_LatLon ... how then if we have

Re: Version stability [was: svn branch issues]

2010-09-17 Thread Yonik Seeley
I think we aim for a stable trunk (4.0-dev) too, as we always have (in the functional sense... i.e. operate correctly, don't crash, etc). The stability is more a reference to API stability - the Java APIs are much more likely to change on trunk. Solr's *external* APIs are much less likely to

Re: Version stability [was: svn branch issues]

2010-09-17 Thread Yonik Seeley
On Fri, Sep 17, 2010 at 10:46 AM, Mark Miller markrmil...@gmail.com wrote: I agree it's mainly API wise, but there are other issues - largely due to Lucene right now - consider the bugs that have been dug up this year on the 4.x line because flex has been such a large rewrite deep in Lucene.

Re: doc into doc

2010-09-17 Thread Yonik Seeley
On Fri, Sep 17, 2010 at 4:12 PM, facholi rfach...@gmail.com wrote: Hi, I would like a json result like that: {   id:2342,   name:Abracadabra,   metadatas: [      {type:tag, name:tutorial},      {type:value, name:2323.434/434},   ] } Do you mean JSON with the tags not quoted (that's

Re: Null Pointer Exception while indexing

2010-09-16 Thread Yonik Seeley
On Wed, Sep 15, 2010 at 2:01 PM, andrewdps mstpa...@gmail.com wrote: I still get the same error when I try to index the mrc file... If you get the exact same error, then you are still using GCJ. When you type java it's probably going to GCJ because of your path (i.e. change it or directly

Re: SOLR interface with PHP using javabin?

2010-09-16 Thread Yonik Seeley
On Thu, Sep 16, 2010 at 2:30 PM, onlinespend...@gmail.com onlinespend...@gmail.com wrote:  I am planning on creating a website that has some SOLR search capabilities for the users, and was also planning on using PHP for the server-side scripting. My goal is to find the most efficient way to

<    6   7   8   9   10   11   12   13   14   15   >