Re: Missing required field: id Using ExtractingRequestHandler

2009-03-19 Thread Larry Reid
Doh! I get it. Ignore my questions in the previous e-mail. The XML files have the id in them. For Word/Excel/PDF etc., it's up to the client (crawler) or whatever to create a unique id if I want a unique id. Thanks again for pointing me in the right direction. I'm really impressed with how easy it

Re: Missing required field: id Using ExtractingRequestHandler

2009-03-19 Thread Larry Reid
Chris: Thanks for the prompt and clear response. If I specify a key myself with "ext.literal.id=" it works fine. I wouldn't have figured out how to specify the key, so thanks for making it clear. Still, if I understand what's going on here: The default XML handler seems to generate a unique key f

Re: large number of cores

2009-03-19 Thread karthik c
Thanks Otis. Will try out using a single index. karthik c http://cantspellathing.blogspot.com On Thu, Mar 19, 2009 at 11:24 PM, Otis Gospodnetic < otis_gospodne...@yahoo.com> wrote: > > You can really go either way. Empty fields are OK. Having lots of cores > seems harder to maintain. Search

Re: Special character indexing

2009-03-19 Thread Shalin Shekhar Mangar
On Fri, Mar 20, 2009 at 10:17 AM, Gargate, Siddharth wrote: > I tried with Jetty but the same issue. Just a guess, but looks like the fix > for SOLR-973 might have introduced this issue. > I'm not sure how SOLR-973 can cause this issue. Can you try using the BinaryRequestWriter and see if it succ

Re: delta-import commit=false doesn't seems to work

2009-03-19 Thread Noble Paul നോബിള്‍ नोब्ळ्
sorry, the whole thing was commented . I did not notice that. I'll look into that 2009/3/20 Noble Paul നോബിള്‍ नोब्ळ् : > you have set autoCommit every x minutes . it must have invoked commit > automatically > > > On Thu, Mar 19, 2009 at 4:17 PM, sunnyfr wrote: >> >> Hi, >> >> Even if I hit comm

RE: Special character indexing

2009-03-19 Thread Gargate, Siddharth
I tried with Jetty but the same issue. Just a guess, but looks like the fix for SOLR-973 might have introduced this issue. Thanks, Siddharth -Original Message- From: Koji Sekiguchi [mailto:k...@r.email.ne.jp] Sent: Friday, March 20, 2009 6:22 AM To: solr-user@lucene.apache.org Subject

Re: delta-import commit=false doesn't seems to work

2009-03-19 Thread Noble Paul നോബിള്‍ नोब्ळ्
you have set autoCommit every x minutes . it must have invoked commit automatically On Thu, Mar 19, 2009 at 4:17 PM, sunnyfr wrote: > > Hi, > > Even if I hit command=delta-import&commit=false&optimize=false > I still have commit set in my logs and sometimes even optimize=true, > > About optimize

Re: Null pointer exception on use of ImportDataHandler (useSolrAddSchema="true")

2009-03-19 Thread Noble Paul നോബിള്‍ नोब्ळ्
it is a bug , I have raised an issue https://issues.apache.org/jira/browse/SOLR-1077 On Fri, Mar 20, 2009 at 4:41 AM, Sam Keen wrote: > guess I spoke to soon.  The above setup (with Shalin's fix) works for > a mock run of 2 records.  But when I try it with the production data > of about 450 reco

RE: Exact Match

2009-03-19 Thread Dean Missikowski (Consultant), CLSA
Hi Laurent, I use the copy field approach and copy the text fields to a custom type "text_exact" that I define in my schema.xml. This allows searching for "exact matches" anywhere within the text field, which doesn't use tokens injected by stemming, synonyms or other index-time filters. In my ap

Re: Special character indexing

2009-03-19 Thread Koji Sekiguchi
Gargate, Siddharth wrote: Hi all, I am trying to index words containing special characters like 'Räikkönen'. Using EmbeddedSolrServer indexing is working fine, but if I use CommonHttpSolrServer then it is indexing garbage values. I am using Solr 1.4 and set URLEcoding as UTF-8 in tomcat. Is thi

Re: Problem with UTF-8 and Solr ISOLatin1AccentFilterFactory

2009-03-19 Thread lupiss
hola! yo tengo el mismo problema, ya también había tratado con ISOLAtin1Accent pero tampoco me funcionó y eso que yo uso otra codificación, ayuda por favor :) -- View this message in context: http://www.nabble.com/Problem-with-UTF-8-and-Solr-ISOLatin1AccentFilterFactory-tp22607642p22612319.html

Re: Problem with UTF-8 and Solr ISOLatin1AccentFilterFactory

2009-03-19 Thread Koji Sekiguchi
aerox7 wrote: Hi, I have a mysql data base in UTF-8. I have a row with "Solène" (solène). I want to transforme this to solene, so i use Solr ISOLatin1AccentFilterFactory to perform this task but it dosn't work ?!! i gess that "Solène" is "solène" in UTF-8 ?! i also set tomcat to utf-8 so norma

Re: Missing required field: id Using ExtractingRequestHandler

2009-03-19 Thread Chris Harris
Unless there's a regression in the ExtractingRequestHandler, then this should be caused because both A) you have an id field defined in your solr schema file that's marked as a required field and B) you did not specify an ID parameter when you submitted your document to the handler. If you don'

Re: Null pointer exception on use of ImportDataHandler (useSolrAddSchema="true")

2009-03-19 Thread Sam Keen
guess I spoke to soon. The above setup (with Shalin's fix) works for a mock run of 2 records. But when I try it with the production data of about 450 records, I get this error. again, any help is greatly appreciated sam keen Mar 19, 2009 3:59:20 PM org.apache.solr.handler.dataimport.DataImport

Missing required field: id Using ExtractingRequestHandler

2009-03-19 Thread Larry Reid
I trying to index Word, PDF and other documents with Solr. I installed the latest nightly build of Solr on March 17. I followed the instructions in the Wiki for ExtractingRequestHandler at http://wiki.apache.org/solr/ExtractingRequestHandler#head-c95841f9eda007b6b4e4594ead12a04223cf7b6e. I have pr

Exact Match

2009-03-19 Thread Vauthrin, Laurent
Hello again, I believe that this question has been posed before but I just wanted to make sure I understood my options. Here's the situation: We have a few fields that are specified as 'text' and a few field that are specified as 'string'. As far as I understand, 'string' will do exact ma

Page-Rank algorithm

2009-03-19 Thread Huang, Zijian(Victor)
Hi, Do you guys know if there is some versions of the page-rank algorithm already implemented in Solr(Lucene)? If not, how hard is it to implement. I am trying to improve the ranking relevance for Solr. Thanks Vic

multiple queries in a single request?

2009-03-19 Thread Jonathan Haddad
Is it possible to do multiple queries in a single request, essentially the equivalent of an SQL UNION? -- Jonathan Haddad http://www.rustyrazorblade.com

Re: Special Characters in facets

2009-03-19 Thread Grant Ingersoll
I'd probably do some light filtering on them to replace them by using the Regex Token Filter so that they don't appear. On Mar 19, 2009, at 11:46 AM, ANKITBHATNAGAR wrote: Hi My app, has a facet with “hello” in it.These quotes are from microsoft word 2003. They show up as this �hello� in

Re: Problem with UTF-8 and Solr ISOLatin1AccentFilterFactory

2009-03-19 Thread Grant Ingersoll
You can try out analysis in http://localhost:8983/solr/admin/analysis.jsp I would check there to make sure it is doing what you think it is doing. Also, where are you seeing it as ""Solène" as opposed to the correct way of solène? On Mar 19, 2009, at 3:25 PM, aerox7 wrote: Hi, I have

Re: Commit is taking very long time

2009-03-19 Thread mahendra mahendra
Hi,   Sorry in delaying to mail!   My application is in prod and quite frequently getting NullPointerException. Initially I thought this is happening because of memory issue, so I reduced mergeFactor to 5 and reduced number of document per commit to 2000. After these changes for some time it stop

Problem with UTF-8 and Solr ISOLatin1AccentFilterFactory

2009-03-19 Thread aerox7
Hi, I have a mysql data base in UTF-8. I have a row with "Solène" (solène). I want to transforme this to solene, so i use Solr ISOLatin1AccentFilterFactory to perform this task but it dosn't work ?!! i gess that "Solène" is "solène" in UTF-8 ?! i also set tomcat to utf-8 so normaly ISOLatin1Acc

Re: DIH - read datasource param values from property file or configure JNDI datasource

2009-03-19 Thread Shalin Shekhar Mangar
On Thu, Mar 19, 2009 at 11:39 PM, Erik Hatcher wrote: > > I don't believe it is the case that all parameters to DIH can be controlled > dynamically on a request. Isn't it the case that the init of a DIH config > is required to pick up some settings? Like the JDBC URL? I just recall > that in m

Re: DIH - read datasource param values from property file or configure JNDI datasource

2009-03-19 Thread Erik Hatcher
On Mar 19, 2009, at 1:59 PM, Fergus McMenemie wrote: I am looking for a implementation of DIH feature: It also takes in a properties file for the data source configuration (http://issues.apache.org/jira/browse/SOLR-469 ) I want to externalize the data source parameters like driver, url, us

Re: DIH - read datasource param values from property file or configure JNDI datasource

2009-03-19 Thread Shalin Shekhar Mangar
On Thu, Mar 19, 2009 at 9:39 PM, Shyamsunder Reddy wrote: > > I am looking for a implementation of DIH feature: It also takes in a > properties file for the data source configuration ( > http://issues.apache.org/jira/browse/SOLR-469) > > I want to externalize the data source parameters like drive

Re: DIH - read datasource param values from property file or configure JNDI datasource

2009-03-19 Thread Fergus McMenemie
>I am looking for a implementation of DIH feature: It also takes in a >properties file for the data source configuration >(http://issues.apache.org/jira/browse/SOLR-469) > >I want to externalize the data source parameters like driver, url, user and >password to property file outside the solr. My

Re: alternative lucene directories support

2009-03-19 Thread Otis Gospodnetic
My quick grep of the sources and scan of the results doesn't see any problematic areas, but if you see some places that still need a fix, yes, please reopen the issue and submit the patch. Do you also plan on submitting the actual alternative Directory impl? $ ffjg FSDire | egrep 'SolrIndexW|

Re: large number of cores

2009-03-19 Thread Otis Gospodnetic
You can really go either way. Empty fields are OK. Having lots of cores seems harder to maintain. Searching against a small core will be faster than searching against a single core/index with all data, but you can use 'fq' to make things really fast. The numbers you quote are not really big

Re: Question about incremental index update

2009-03-19 Thread Otis Gospodnetic
Victor, Yes, if you use the same ID (and a URL could serve as a Document ID), Solr will update the Document. Note that Solr doesn't do crawling/web page fetching, but Nutch and Droids do. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: "H

DIH - read datasource param values from property file or configure JNDI datasource

2009-03-19 Thread Shyamsunder Reddy
I am looking for a implementation of DIH feature: It also takes in a properties file for the data source configuration (http://issues.apache.org/jira/browse/SOLR-469) I want to externalize the data source parameters like driver, url, user and password to property file outside the solr. My aim

Special Characters in facets

2009-03-19 Thread ANKITBHATNAGAR
Hi My app, has a facet with “hello” in it.These quotes are from microsoft word 2003. They show up as this �hello� in side the facet on webpage. Also if I save the selection the text becomes�hello� I have UTF-8 encoding enabled on my solr 1.3 Let me know how can i fix this Ankit -- Vi

Problem for replication : segment optimized automaticly

2009-03-19 Thread sunnyfr
Hi everybody ... still me :) hoo happy day :) Just, I dont get where I miss something, I will try to be clear. this is my index folder (and we can notice the evolution according to the delta import every 30mn) : r...@search-01:/data/solr# ls video/data/index/ _2bel.fdt _2bel.fnm _2bel.nrm _2

Re: Combination of solr.xml and solrconfig.xml

2009-03-19 Thread Kraus, Ralf | pixelhouse GmbH
Erik Hatcher schrieb: Unfortunately Solr doesn't offer that kind of reusability (yet). But you could just go old school and use XML entity includes to pull in a fragment of a file, and externalize the common pieces that way. Not as pretty, but oughta work. thx a lot ! This sounds perfect...

alternative lucene directories support

2009-03-19 Thread Andrey Klochkov
Hi all We want to use Solr with lucene Directory implementation which places index into Coherence data grid. I fact I managed to run Solr in such configuration although I had to patch it. I think that the issue about alternate directories support (SOLR-465) should be re-opened because there are so

Re: Combination of solr.xml and solrconfig.xml

2009-03-19 Thread Erik Hatcher
Unfortunately Solr doesn't offer that kind of reusability (yet). But you could just go old school and use XML entity includes to pull in a fragment of a file, and externalize the common pieces that way. Not as pretty, but oughta work. Erik On Mar 19, 2009, at 10:05 AM, Giovanni De

Re: Combination of solr.xml and solrconfig.xml

2009-03-19 Thread Giovanni De Stefano
Hello Ralf, what you can do is the following: 1) you have the following in all your solrconfig.xml files: Type your query and press Search! 2) you want to move this into the single solr.xml. To achieve this you have to do the following: 2.1) in solr.xml: ** 2.2) in ALL solrc

Re: Combination of solr.xml and solrconfig.xml

2009-03-19 Thread Kraus, Ralf | pixelhouse GmbH
Giovanni De Stefano schrieb: On the other hand, if your files have 199 lines in *common*, then yes, you could/should move those 199 common lines in solr.xml. YES ! Thats my case But how could I transfer this lines (solrconfig.xml) /www/solr//xx ${solr.abortOnConfigurationError

Re: Combination of solr.xml and solrconfig.xml

2009-03-19 Thread Giovanni De Stefano
Hello Ralf, I am confused now. Perhaps you can post a short extract of your files and show me what you are trying to achieve? If your files *differs* for 199 lines...well...they should stay the way they are! On the other hand, if your files have 199 lines in *common*, then yes, you could/should

Re: Combination of solr.xml and solrconfig.xml

2009-03-19 Thread Kraus, Ralf | pixelhouse GmbH
Giovanni De Stefano schrieb: Hello Ralf, I believe you want to do something like: Okay I will try to explain it better :-) - my two solrconfig.xml files are around 200 lines long - ONLY one line is the difference between this files : /www/solr//x - I want to t

Re: Combination of solr.xml and solrconfig.xml

2009-03-19 Thread Giovanni De Stefano
Hello Ralf, I believe you want to do something like: Cheers, Giovanni On 3/19/09, Kraus, Ralf | pixelhouse GmbH wrote: > > Giovanni De Stefano schrieb: > >> Guten Tag Ralf, >> >> try to look here: http://wiki.apache.org/solr/CoreAdmin >> >> It says: "You can also specify p

Re: Combination of solr.xml and solrconfig.xml

2009-03-19 Thread Kraus, Ralf | pixelhouse GmbH
Giovanni De Stefano schrieb: Guten Tag Ralf, try to look here: http://wiki.apache.org/solr/CoreAdmin It says: "You can also specify properties in solr.xml which can be used in the solrconfig.xml and schema.xml files." I believe this might help. thx !! I tried this : (My solr.xml :)

large number of cores

2009-03-19 Thread karthik c
Hi guys, We need to index data of a large number of types. I was wondering if it is better to create separate cores for each type or add everything to one core with a "type" field ? Here are some more details: The database: Currently we have around 200 types of data. The data for each type is sto

Re: delta-import commit=false doesn't seems to work

2009-03-19 Thread sunnyfr
So I've several searcher opened for just on delta import hit and the param commit=false no cronjob set --> 32 2147483647 1 1000 false 32 25 2147483647 1 Huge thanks for your help Mar 19 11:38:32 search-01 jsvc.exec

delta-import commit=false doesn't seems to work

2009-03-19 Thread sunnyfr
Hi, Even if I hit command=delta-import&commit=false&optimize=false I still have commit set in my logs and sometimes even optimize=true, About optimize I wonder if it comes from commitment too close and one is not done, but still I don't know really. Any idea? Thanks a lot, -- View this messa

Re: Combination of solr.xml and solrconfig.xml

2009-03-19 Thread Giovanni De Stefano
Guten Tag Ralf, try to look here: http://wiki.apache.org/solr/CoreAdmin It says: "You can also specify properties in solr.xml which can be used in the solrconfig.xml and schema.xml files." I believe this might help. Cheers, Giovanni On 3/19/09, Kraus, Ralf | pixelhouse GmbH wrote: > > Hi, > >

Combination of solr.xml and solrconfig.xml

2009-03-19 Thread Kraus, Ralf | pixelhouse GmbH
Hi, I have a little question concerning the combination of solr.xml and solrconfig.xml ! I am running a multicore configuration for doing multiple language index files ... my structure is like \solr (solr.xml) \solr\core_de (solrconfig.xml for german profile) \solr\core_uk (solrconfig.xml fo

Re: Field tokenizer question

2009-03-19 Thread Koji Sekiguchi
Ashish P wrote: I have created a field, Set class="solr.TextField" instead of class="solr.StrField" in your fieldType definition. Then reindex and commit. Koji

Re: Problem with Facet Date Query

2009-03-19 Thread dabboo
Thanks Eric. But where in schema.xml, I will have to give this value and how. Erik Hatcher wrote: > > > On Mar 19, 2009, at 1:57 AM, dabboo wrote: >> This is my query: >> q=productPublicationDate_product_dt:[*%20TO >> %20NOW]&facet=true&facet.field=productPublicationDate_product_dt:[* >> %20T

Re: Problem with Facet Date Query

2009-03-19 Thread Erik Hatcher
On Mar 19, 2009, at 1:57 AM, dabboo wrote: This is my query: q=productPublicationDate_product_dt:[*%20TO %20NOW]&facet=true&facet.field=productPublicationDate_product_dt:[* %20TO%20NOW]&qt=dismaxrequest This is exception, I am facing after running this query. - org.apache.solr.common.So

Re: Solr multiple indexes

2009-03-19 Thread Giovanni De Stefano
Hello Otis, thank you for your reply. What I am trying to achieve is to index different tables with different primary keys and different fields (basically different documents/entity). Is it possible to create a data-config with different root entities/documents and index/search everything transp