wildcard searches not consistent
Hi, Still working on extending my proof of concept by working off the example configuration and modifying the schema.xml. Having trouble with wildcard searches: factory OR faction -- 40 results (ok) factory -- 1 result (ok) faction -- 39 results (ok) facti?n -- 39 results (ok) fact* -- 40 results (ok) ?act* -- 40 results (ok) facto* -- 1 result (ok) factor? -- 1 result (ok) ?actor? -- 1 result (ok) but facto?y -- 0 (expecting 1) I thought these are all valid searches but am I missing something? Any config setting? Also, Iraq and ?raq returns the same number of results bu Ira? returns zero. Thanks again.
Re: wildcard searches not consistent
That was it! Thank you very much. - Original Message From: Robert Muir rcm...@gmail.com To: solr-user@lucene.apache.org Sent: Tue, September 14, 2010 5:58:03 PM Subject: Re: wildcard searches not consistent but facto?y -- 0 (expecting 1) you have stemming enabled for the field? stemming will make your wildcards behave strangely. I would recommend you turn it off. because stemming likely turned factory into factori or similar I thought these are all valid searches but am I missing something? Any config setting? Also, Iraq and ?raq returns the same number of results bu Ira? returns zero. wildcard queries are not analyzed, so they are not lowercased and in fact case sensitive..., but if you lowercased all your terms in your analyzer then Ira? will not match Iraq because Iraq was downcased to iraq at index time. -- Robert Muir rcm...@gmail.com
How to load data to embedded server
Hi, I'm using EmbeddedSolrServer for my unit tests. I just can't figure out how to add my data (stored in xml files similar to those in the example application example/exampleDocs) after instantiating the server. The source code for the simple post tool seems to require a stream to write the file contents to. How can I do that with the embedded server? Thanks, Rico
How to import data with a different date format
Hi, I am attempting to import some of our data into SOLR. I did it the quickest way I know because I literally only have 2 days to import the data and do some queries for a proof-of-concept. So I have this data in XML format and I wrote a short XSLT script to convert it to the format in solr/example/exampledocs (except I retained the element names so I had to modify schema.xml in the conf directory. So far so good -- the import works and I can search the data. One of my immediate problems is that there is a date field with the format MM/DD/. Looking at schema.xml, it seems SOLR accepts only full date fields -- everything seems to be mandatory including the Z for Zulu/UTC time according to the doc. Is there a way to specify the date format? Thanks very much. Rico
Re: How to import data with a different date format
That was my first thought :-) But it would be nice to be able to do date queries. I guess when I export the data I can just add 00:00:00Z. Thanks. - Original Message From: Markus Jelsma markus.jel...@buyways.nl To: solr-user@lucene.apache.org Sent: Wed, September 8, 2010 11:34:32 AM Subject: RE: How to import data with a different date format No. The Datefield [1] will not accept it any other way. You could, however, fool your boss and dump your dates in an ordinary string field. But then you cannot use some of the nice date features. [1]: http://lucene.apache.org/solr/api/org/apache/solr/schema/DateField.html -Original message- From: Rico Lelina rlel...@yahoo.com Sent: Wed 08-09-2010 17:36 To: solr-user@lucene.apache.org; Subject: How to import data with a different date format Hi, I am attempting to import some of our data into SOLR. I did it the quickest way I know because I literally only have 2 days to import the data and do some queries for a proof-of-concept. So I have this data in XML format and I wrote a short XSLT script to convert it to the format in solr/example/exampledocs (except I retained the element names so I had to modify schema.xml in the conf directory. So far so good -- the import works and I can search the data. One of my immediate problems is that there is a date field with the format MM/DD/. Looking at schema.xml, it seems SOLR accepts only full date fields -- everything seems to be mandatory including the Z for Zulu/UTC time according to the doc. Is there a way to specify the date format? Thanks very much. Rico
Re: Re: How to import data with a different date format
It will work. The original data is in XML format. I have an XSLT that transforms the data into the same format as that in exampledocs: adddocfield name=../field/doc.../add. - Original Message From: Markus Jelsma markus.jel...@buyways.nl To: solr-user@lucene.apache.org Sent: Wed, September 8, 2010 12:06:39 PM Subject: RE: Re: How to import data with a different date format Your format (MM/DD/) is not compatible. -Original message- From: Rico Lelina rlel...@yahoo.com Sent: Wed 08-09-2010 19:03 To: solr-user@lucene.apache.org; Subject: Re: How to import data with a different date format That was my first thought :-) But it would be nice to be able to do date queries. I guess when I export the data I can just add 00:00:00Z. Thanks. - Original Message From: Markus Jelsma markus.jel...@buyways.nl To: solr-user@lucene.apache.org Sent: Wed, September 8, 2010 11:34:32 AM Subject: RE: How to import data with a different date format No. The Datefield [1] will not accept it any other way. You could, however, fool your boss and dump your dates in an ordinary string field. But then you cannot use some of the nice date features. [1]: http://lucene.apache.org/solr/api/org/apache/solr/schema/DateField.html -Original message- From: Rico Lelina rlel...@yahoo.com Sent: Wed 08-09-2010 17:36 To: solr-user@lucene.apache.org; Subject: How to import data with a different date format Hi, I am attempting to import some of our data into SOLR. I did it the quickest way I know because I literally only have 2 days to import the data and do some queries for a proof-of-concept. So I have this data in XML format and I wrote a short XSLT script to convert it to the format in solr/example/exampledocs (except I retained the element names so I had to modify schema.xml in the conf directory. So far so good -- the import works and I can search the data. One of my immediate problems is that there is a date field with the format MM/DD/. Looking at schema.xml, it seems SOLR accepts only full date fields -- everything seems to be mandatory including the Z for Zulu/UTC time according to the doc. Is there a way to specify the date format? Thanks very much. Rico
Re: How to import data with a different date format
I'm going with option 1, converting MM/DD/ to -MM-DD (which is fairly easy in XSLT) and then adding T00:00:00Z to it. Thanks. - Original Message From: Erick Erickson erickerick...@gmail.com To: solr-user@lucene.apache.org Sent: Wed, September 8, 2010 12:09:55 PM Subject: Re: How to import data with a different date format I think Markus is spot-on given the fact that you have 2 days. Using a string field is quickest. However, if you absolutely MUST have functioning dates, there are three options I can think of: 1 can you make your XSLT transform the dates? Confession; I'm XSLT-ignorant 2 use DIH and DateTransformer, see: http://wiki.apache.org/solr/DataImportHandler#DateFormatTransformer you can walk a directory importing all the XML files with FileDataSource. http://wiki.apache.org/solr/DataImportHandler#DateFormatTransformer3 you could write a program to do this manually. But given the time constraints, I suspect your time would be better spent doing the other stuff and just using string as per Markus. I have no clue how SOLR-savvy you are, so pardon if this is something you already know. But lots of people trip up over the string field type, which is NOT tokenized. You usually want text unless it's some sort of ID So it might be worth it to do some searching earlier rather than later G Best Erick On Wed, Sep 8, 2010 at 12:34 PM, Markus Jelsma markus.jel...@buyways.nlwrote: No. The Datefield [1] will not accept it any other way. You could, however, fool your boss and dump your dates in an ordinary string field. But then you cannot use some of the nice date features. [1]: http://lucene.apache.org/solr/api/org/apache/solr/schema/DateField.html -Original message- From: Rico Lelina rlel...@yahoo.com Sent: Wed 08-09-2010 17:36 To: solr-user@lucene.apache.org; Subject: How to import data with a different date format Hi, I am attempting to import some of our data into SOLR. I did it the quickest way I know because I literally only have 2 days to import the data and do some queries for a proof-of-concept. So I have this data in XML format and I wrote a short XSLT script to convert it to the format in solr/example/exampledocs (except I retained the element names so I had to modify schema.xml in the conf directory. So far so good -- the import works and I can search the data. One of my immediate problems is that there is a date field with the format MM/DD/. Looking at schema.xml, it seems SOLR accepts only full date fields -- everything seems to be mandatory including the Z for Zulu/UTC time according to the doc. Is there a way to specify the date format? Thanks very much. Rico
