Re: Sub entities

2011-03-01 Thread Stefan Matheis
Brian, except for your sql-syntax error in the specie_relations-query SELECT specie_id FROMspecie_relations .. (missing whitespace after FROM) your config looks okay. following questions: * is there a field named specie in your schema? (otherwise dih will silently ignore it) * did you check your

Re: Disabling caching for fq param?

2011-03-01 Thread Markus Jelsma
If filterCache hitratio is low then just disable it in solrconfig by deleting the section or setting its values to 0. Based on what I've read here and what I could find on the web, it seems that each fq clause essentially gets its own results cache. Is that correct? We have a corporate

Re: Problem with sorting using functions.

2011-03-01 Thread Jan Høydahl
Also, if you're on 3.1, the function needs to be without spaces since sort will split on space to find the sort order. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com On 28. feb. 2011, at 22.34, John Sherwood wrote: Fair call. Thanks. On Tue, Mar 1, 2011 at 8:21

Re: multi-core solr, specifying the data directory

2011-03-01 Thread Jan Høydahl
Have you tried removing the dataDir tag from solrconfig.xml? Then it should fall back to default ./data relative to core instancedir. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com On 1. mars 2011, at 00.00, Jonathan Rochkind wrote: Unless I'm doing something

Re: Problem with Solr and Nutch integration

2011-03-01 Thread Paul Rogers
Hi Anurag The request handler has been added the solrconfig file. I'll try your attached requesthandler and see if that helps. Interestingly enough the whole setup when I was using nutch 1.2/solr 1.4.1. It is only since moving to nutch trunk/solr branch_3x that the problem has occurred. I

Re: Problem with Solr and Nutch integration

2011-03-01 Thread Anurag
i have nutch-1.0 and Apache-solr-1.3.0 (integrated these two). On 3/1/11, Paul Rogers [via Lucene] ml-node+2601915-1461428819-146...@n3.nabble.com wrote: Hi Anurag The request handler has been added the solrconfig file. I'll try your attached requesthandler and see if that helps.

Error during auto-warming of key

2011-03-01 Thread Markus Jelsma
Hi, Yesterday's error log contains something peculiar: ERROR [solr.search.SolrCache] - [pool-29-thread-1] - : Error during auto- warming of key:+*:* (1.0/(7.71E-8*float(ms(const(1298682616680),date(sort_date)))+1.0))^20.0:java.lang.NullPointerException at

Problem with sorting using functions

2011-03-01 Thread John Sherwood
This works: /select/?q=*:*sort=price desc This throws a 400 error: /select/?q=*:*sort=sum(1, 1) desc Missing sort order. I'm using 1.4.2. I've tried all sorts of different numbers/functions/fields and nothing seems to change that error. Any ideas? **

Retrieving payload from each highlighted term

2011-03-01 Thread Fabiano Nunes
How can I get the payload from each highlighted term?

RE: Query on multivalue field

2011-03-01 Thread Steven A Rowe
Hi Scott, Querying against a multi-valued field just works - no special incantation required. Steve -Original Message- From: Scott Yeadon [mailto:scott.yea...@anu.edu.au] Sent: Monday, February 28, 2011 11:50 PM To: solr-user@lucene.apache.org Subject: Query on multivalue field

Help with explain query syntax

2011-03-01 Thread Glòria Martínez
Hello, I can't understand why this query is not matching anything. Could someone help me please? *Query* http://localhost:8894/solr/select?q=linguajob.plqf=company_namewt=xmlqt=dismaxdebugQuery=onexplainOther=id%3A1 response - lst name=responseHeader int name=status0/int int name=QTime12/int -

Re: multi-core solr, specifying the data directory

2011-03-01 Thread Jonathan Rochkind
I did try that, yes. I tried that first in fact! It seems to fall back to a ./data directory relative to the _main_ solr directory (the one above all the cores), not the core instancedir. Which is not what I expected either. I wonder if this should be considered a bug? I wonder if anyone

MLT with boost

2011-03-01 Thread Mark
Is it possible to add function queries/boosts to the results that are by MLT? If not out of the box how would one go about achieving this functionality? Thanks

Re: please make JSONWriter public

2011-03-01 Thread Ryan McKinley
You may have noticed the ResponseWriter code is pretty hairy! Things are package protected so that the API can change between minor release without concern for back compatibility. In 4.0 (/trunk) I hope to rework the whole ResponseWriter framework so that it is more clean and hopefully stable

Re: Sub entities

2011-03-01 Thread Brian Lamb
Yes, it looks like I had left off the field (misspelled it actually). I reran the full import and the fields did properly show up. However, it is still not working as expected. Using the example below, a result returned would only list one specie instead of a list of species. I have the following

Re: Indexed, but cannot search

2011-03-01 Thread Brian Lamb
Thank you for your reply but the searching is still not working out. For example, when I go to: http://localhost:8983/solr/select/?q=*%3A*http://localhost:8983/solr/select/?q=*%3A*version=2.2start=0rows=10indent=on I get the following as a response: result name=response numFound=249943 start=0

Re: Sub entities

2011-03-01 Thread Stefan Matheis
Brian, On Tue, Mar 1, 2011 at 4:52 PM, Brian Lamb brian.l...@journalexperts.com wrote: field column=specie multiValued=true name=specie type=string indexed=true stored=true required=false / Not sure, but iirc field in this context has no column-Attribute .. that should normally not break your

Re: Indexed, but cannot search

2011-03-01 Thread Edoardo Tosca
Hi, i'm not sure if it is a typo, anyway the second query you mentioned should be: http://localhost:8983/solr/select/?q=type:* HTH, Edo On Tue, Mar 1, 2011 at 4:06 PM, Brian Lamb brian.l...@journalexperts.comwrote: Thank you for your reply but the searching is still not working out. For

Re: Indexed, but cannot search

2011-03-01 Thread Upayavira
Next question, do you have your type field set to index=true in your schema? Upayavira On Tue, 01 Mar 2011 11:06 -0500, Brian Lamb brian.l...@journalexperts.com wrote: Thank you for your reply but the searching is still not working out. For example, when I go to:

Re: Question on writing custom UpdateHandler

2011-03-01 Thread Chris Hostetter
In your first attempt, the crux of your problem was probably that you were never closing the searcher/reader. : Or how can I perform a query on the current state of the index from within an : UpdateProcessor? If you implement UpdateRequestProcessorFactory, the getInstance method is given the

Re: multi-core solr, specifying the data directory

2011-03-01 Thread Chris Hostetter
: Unless I'm doing something wrong, in my experience in multi-core Solr in : 1.4.1, you NEED to explicitly provide an absolute path to the 'data' dir. have you looked at the example/multicore directory that was included in the 1.4.1 release? it has a solr.xml that loads two cores w/o

Re: please make JSONWriter public

2011-03-01 Thread Paul Libbrecht
Ryan, honestly, hairyness was rather mild. I found it fairly readable. paul Le 1 mars 2011 à 16:46, Ryan McKinley a écrit : You may have noticed the ResponseWriter code is pretty hairy! Things are package protected so that the API can change between minor release without concern for back

solr different sizes on master and slave

2011-03-01 Thread Mike Franon
I was curious why would the size be dramatically different even though the index versions are the same? One is 1.2 Gb, and on the slave it is 512 MB I would think they should both be the same size no? Thanks

Re: Sub entities

2011-03-01 Thread Brian Lamb
Thanks for the help Stefan. It seems removing column=specie fixed it. On Tue, Mar 1, 2011 at 11:18 AM, Stefan Matheis matheis.ste...@googlemail.com wrote: Brian, On Tue, Mar 1, 2011 at 4:52 PM, Brian Lamb brian.l...@journalexperts.com wrote: field column=specie multiValued=true

Re: Indexed, but cannot search

2011-03-01 Thread Brian Lamb
Hi all, The problem was that my fields were defined as type=string instead of type=text. Once I corrected that, it seems to be fixed. The only part that still is not working though is the search across all fields. For example: http://localhost:8983/solr/select/?q=type%3AMammal Now correctly

Re: Indexed, but cannot search

2011-03-01 Thread Markus Jelsma
Traditionally, people forget to reindex ;) Hi all, The problem was that my fields were defined as type=string instead of type=text. Once I corrected that, it seems to be fixed. The only part that still is not working though is the search across all fields. For example:

Re: solr different sizes on master and slave

2011-03-01 Thread Markus Jelsma
Are there pending commits on the master? I was curious why would the size be dramatically different even though the index versions are the same? One is 1.2 Gb, and on the slave it is 512 MB I would think they should both be the same size no? Thanks

Re: Indexed, but cannot search

2011-03-01 Thread Brian Lamb
Oh if only it were that easy :-). I have reindexed since making that change which is how I was able to get the regular search working. I have not however been able to get the search across all fields to work. On Tue, Mar 1, 2011 at 3:01 PM, Markus Jelsma markus.jel...@openindex.iowrote:

Re: solr different sizes on master and slave

2011-03-01 Thread Mike Franon
No pending commits, what it looks like is there are almost two copies of the index on the master, not sure how that happened. On Tue, Mar 1, 2011 at 3:08 PM, Markus Jelsma markus.jel...@openindex.io wrote: Are there pending commits on the master? I was curious why would the size be

numberic or string type for non-sortable field?

2011-03-01 Thread cyang2010
I wonder if i shall use solr int or string for such field with following requirement multi-value facet needed sort not needed The field value is a an id. Therefore, i can store as either numeric field or just a string. Shall i choose string for efficiency? Thanks. -- View this message in

Re: numberic or string type for non-sortable field?

2011-03-01 Thread Ahmet Arslan
I wonder if i shall use solr int or string for such field with following requirement multi-value facet needed sort not needed The field value is a an id.  Therefore, i can store as either numeric field or just a string.   Shall i choose string for efficiency? Trie based integer

Re: Query on multivalue field

2011-03-01 Thread Scott Yeadon
Thanks, but just to confirm the way multiValued fields work: In a multiValued field, call it field1, if I have two values indexed to this field, say value 1 = some text...termA...more text and value 2 = some text...termB...more text and do a search such as field1:(termA termB) (where

Re: solr different sizes on master and slave

2011-03-01 Thread Mike Franon
ok doing some more research I noticed, on the slave it has multiple folders where it keeps them for example index index.20110204010900 index.20110204013355 index.20110218125400 and then there is an index.properties that shows which index it is using. I am just curious why does it keep multiple

Re: numberic or string type for non-sortable field?

2011-03-01 Thread Chris Hostetter
: The field value is a an id.  Therefore, i can store as : either numeric field : or just a string.   Shall i choose string : for efficiency? : : Trie based integer (tint) is preferred for faster faceting. range faceting/filtering yes -- not for field faceting which is what i think he's

Re: multi-core solr, specifying the data directory

2011-03-01 Thread Jonathan Rochkind
Hmm, okay, have to try to find time to install the example/multicore and see. It's definitely never worked for me, weird. Thanks. On 3/1/2011 2:38 PM, Chris Hostetter wrote: : Unless I'm doing something wrong, in my experience in multi-core Solr in : 1.4.1, you NEED to explicitly provide an

Re: solr different sizes on master and slave

2011-03-01 Thread Jonathan Rochkind
The slave should not keep multiple copies _permanently_, but might temporarily after it's fetched the new files from master, but before it's committed them and fully wamred the new index searchers in the slave. Could that be what's going on, is your slave just still working on committing and

Re: numberic or string type for non-sortable field?

2011-03-01 Thread cyang2010
Sorry i didn't make my question clear. I will only facet based on field value, not ranged query (it is just some ids for a multi-value field). And i won't do sort on the field either. In that case, is string more efficient for the requirement? -- View this message in context:

Re: Query on multivalue field

2011-03-01 Thread Ahmet Arslan
In a multiValued field, call it field1, if I have two values indexed to this field, say value 1 = some text...termA...more text and value 2 = some text...termB...more text and do a search such as field1:(termA termB) (where solrQueryParser defaultOperator=AND/) I'm getting a hit

Re: numberic or string type for non-sortable field?

2011-03-01 Thread Ahmet Arslan
I will only facet based on field value, not ranged query  (it is just some ids for a  multi-value field).   And i won't do sort on the field either. In that case, is string more efficient for the requirement? Hoss was saying to use, fieldType name=int class=solr.TrieIntField

Searching all terms - SolrJ

2011-03-01 Thread openvictor Open
Dear all, First I am sorry if this question has already been asked ( I am sure it was...) but I can't find the right option with solrj. I want to query only documents that contains ALL query terms. Let me take an example, I have 4 documents that are simple sequences ( they have only one field :

Re: Searching all terms - SolrJ

2011-03-01 Thread Ahmet Arslan
--- On Wed, 3/2/11, openvictor Open openvic...@gmail.com wrote: From: openvictor Open openvic...@gmail.com Subject: Searching all terms - SolrJ To: solr-user@lucene.apache.org Date: Wednesday, March 2, 2011, 12:20 AM Dear all, First I am sorry if this question has already been asked ( I

Re: Query on multivalue field

2011-03-01 Thread Scott Yeadon
The only trick with this is ensuring the searches return the right results and don't go across value boundaries. If I set the gap to the largest text size we expect (approx 5000 chars) what impact does such a large value have (i.e. does Solr physically separate these fragments in the index or

Re: Distances in spatial search (Solr 4.0)

2011-03-01 Thread Alexandre Rocco
Hi Bill, I was using a different approach to sort by the distance with the dist() function, since geodist() is not documented on the wiki ( http://wiki.apache.org/solr/FunctionQuery) Tried something like: sort=dist(2, 45.15,-93.85, lat, lng) asc I made some tests with geodist() function as you

Re: multi-core solr, specifying the data directory

2011-03-01 Thread Michael Sokolov
I tried this in my 1.4.0 installation (commenting out what had been working, hoping the default would be as you said works in the example): solr persistent=true sharedLib=lib cores adminPath=/admin/cores core name=bpro instanceDir=bpro !-- property name=solr.data.dir value=solr/bpro/data/ --

Re: Query on multivalue field

2011-03-01 Thread Jonathan Rochkind
Each token has a position set on it. So if you index the value alpha beta gamma, it winds up stored in Solr as (sort of, for the way we want to look at it) document1: alpha:position 1 beta:position 2 gamma: postition 3 If you set the position increment gap large, then

Re: multi-core solr, specifying the data directory

2011-03-01 Thread Jonathan Rochkind
This definitely matches my own experience, and I've heard it from others. I haven't heard of anyone who HAS gotten it to work like that. But apparently there's a distributed multi-core example which claims to work like it doesn't for us. One of us has to try the Solr distro multi-core

[ANNOUNCE] Web Crawler

2011-03-01 Thread Dominique Bejean
Hi, I would like to announce Crawl Anywhere. Crawl-Anywhere is a Java Web Crawler. It includes : * a crawler * a document processing pipeline * a solr indexer The crawler has a web administration in order to manage web sites to be crawled. Each web site crawl is configured with a

Re: Searching all terms - SolrJ

2011-03-01 Thread openvictor Open
Yes but I want to leave the choice to the user. He can either search all the terms or just some. Is there any more flexible solution ? Even if I have to code it by hand ? 2011/3/1 Ahmet Arslan iori...@yahoo.com --- On Wed, 3/2/11, openvictor Open openvic...@gmail.com wrote: From:

Re: Query on multivalue field

2011-03-01 Thread Scott Yeadon
Tested it out and seems to work well as long as I set the gap to a value much longer than the text - 1 appear to work fine for our current data. Thanks heaps for all the help guys! Scott. On 2/03/11 11:13 AM, Jonathan Rochkind wrote: Each token has a position set on it. So if you index

Re: numberic or string type for non-sortable field?

2011-03-01 Thread cyang2010
Can I know why? I thought solr is tuned for string if no sorting of facet by range query is needed. -- View this message in context: http://lucene.472066.n3.nabble.com/numberic-or-string-type-for-non-sortable-field-tp2606353p2607932.html Sent from the Solr - User mailing list archive at

Re: solr different sizes on master and slave

2011-03-01 Thread Markus Jelsma
Indeed, the slave should not have useless copies but it does, at least in 1.4.0, i haven't seen it in 3.x, but that was just a small test that did not exactly meet my other production installs. In 1.4.0 Solr does not remove old copies at startup and it does not cleanly abort running

Re: Indexed, but cannot search

2011-03-01 Thread Markus Jelsma
Hmm, please provide analyzer of text and output of debugQuery=true. Anyway, if field type is fieldType text and the catchall field text is fieldType text as well and you reindexed, it should work as expected. Oh if only it were that easy :-). I have reindexed since making that change which

Re: multi-core solr, specifying the data directory

2011-03-01 Thread Chris Hostetter
: !-- Used to specify an alternate directory to hold all index data :other than the default ./data under the Solr home. :If replication is in use, this should match the replication : configuration : . -- : dataDir${solr.data.dir:./solr/data}/dataDir that directive says use the

Re: numberic or string type for non-sortable field?

2011-03-01 Thread Chris Hostetter
: Can I know why? I thought solr is tuned for string if no sorting of facet by : range query is needed. tuned for string doesn't really mean anything to me, i'm not sure what that's in refrence to. nothing thta i know of is particularly optimized for strings. Almost anything can be

Re: Distances in spatial search (Solr 4.0)

2011-03-01 Thread William Bell
See http://wiki.apache.org/solr/SpatialSearch and yest use sort=geodist()+asc This Wiki page has everything you should need\. On Tue, Mar 1, 2011 at 3:49 PM, Alexandre Rocco alel...@gmail.com wrote: Hi Bill, I was using a different approach to sort by the distance with the dist() function,

Re: Question about Nested Span Near Query

2011-03-01 Thread William Bell
I am not 100% sure. But I why did you not use the standard confix for text ? fieldType name=text class=solr.TextField positionIncrementGap=100 autoGeneratePhraseQueries=true analyzer type=index tokenizer class=solr.WhitespaceTokenizerFactory/ !-- in this example, we will

indexing mysql dateTime/timestamp into solr date field

2011-03-01 Thread cyang2010
Hi, I can't seem to be able to index to a solr date field from a query result using DataImportHandler. Anyone else know how to resoleve the problem? entity name=title query=select ID, title_full as TITLE_NAME, YEAR, COUNTRY_OF_ORIGIN, modified as RELEASE_DATE

Re: More Date Math: NOW/WEEK

2011-03-01 Thread Chris Hostetter
: Digging into the source code of DateMathParser.java, i found the following : comment: :99 // NOTE: consciously choosing not to support WEEK at this time, : 100 // because of complexity in rounding down to the nearest week 101 : // arround a month/year boundry. 102

Re: indexing mysql dateTime/timestamp into solr date field

2011-03-01 Thread Chris Hostetter
: query=select ID, title_full as TITLE_NAME, YEAR, : COUNTRY_OF_ORIGIN, modified as RELEASE_DATE from title limit 10 Are you certian that the first 10 results returned (you have limit 10) all have a value in the modified field? if modified is nullable you could very

Re: Searching all terms - SolrJ

2011-03-01 Thread Chris Hostetter
: Yes but I want to leave the choice to the user. : : He can either search all the terms or just some. : : Is there any more flexible solution ? Even if I have to code it by hand ? the declaration in the schema dictates the default. you can override the default at query time using the q.op

Re: indexing mysql dateTime/timestamp into solr date field

2011-03-01 Thread cyang2010
Yes, I am pretty sure every row has a modified field. I did my testing before posting question. I tried with adding DateFormatTransformer, still not help. entity name=title query=select ID, title_full as TITLE_NAME, YEAR, COUNTRY_OF_ORIGIN, modified as

Re: indexing mysql dateTime/timestamp into solr date field

2011-03-01 Thread William Bell
field column=date dateTimeFormat=-MM-dd'T'hh:mm:ss / Did you convert the date to standard GMT format as above in DIH? Also add transformer=DateFormatTransformer,... http://lucene.apache.org/solr/api/org/apache/solr/schema/DateField.html On Tue, Mar 1, 2011 at 7:54 PM, cyang2010

Re: [ANNOUNCE] Web Crawler

2011-03-01 Thread David Smiley (@MITRE.org)
Dominique, The obvious number one question is of course why you re-invented this wheel when there are several existing crawlers to choose from. Your website says the reason is that the UIs on existing crawlers (e.g. Nutch, Heritrix, ...) weren't sufficiently user-friendly or had the site-specific

Re: indexing mysql dateTime/timestamp into solr date field

2011-03-01 Thread cyang2010
Bill, I did try to use the way you suggested above. Unfortunately it does not work either. It is pretty much the same as my last reply, except the dateTimeFormat=-MM-dd'T'hh:mm:ss Thanks, cyang -- View this message in context:

Re: Searching all terms - SolrJ

2011-03-01 Thread openvictor Open
Great ! Thank you very much Chris, it will come handy ! Best regards, Victor 2011/3/1 Chris Hostetter hossman_luc...@fucit.org : Yes but I want to leave the choice to the user. : : He can either search all the terms or just some. : : Is there any more flexible solution ? Even if I have

how to debug dataimporthandler

2011-03-01 Thread cyang2010
I wonder how to run dataimporthandler in debug mode. Currently i can't get data correctly into index through dataimporthandler, especially a timestamp column to solr date field. I want to debug the process. According to this wiki page: Commands The handler exposes all its API as http requests

MorelikeThis not working with Shards(distributed Search)

2011-03-01 Thread Isha Garg
Hi, I am experimenting with the *morelikethis* to see if it also works with *distributed* search.But i did not get the solution yet.Can anyone help me regarding this. please provide me detailed description. as I didnt find it by updating