RE: Regarding Copyfield

2013-01-15 Thread Harshvardhan Ojha
What is your text_general type definition in schema.xml? -Original Message- From: anurag.jain [mailto:anurag.k...@gmail.com] Sent: Tuesday, January 15, 2013 12:16 PM To: solr-user@lucene.apache.org Subject: Regarding Copyfield hi in copy field i am not storing first_name last_name

Multicore configuration

2013-01-15 Thread Bruno Dusausoy
Hi, I'd like to use two separate indexes (Solr 3.6.1). I've read several wiki pages and looked at the multicore example bundled with the distribution but it seems I missing something. I have this hierarchy : solr-home/ | -- conf | -- solr.xml -- solrconfig.xml (if I don't put it,

Re: Multicore configuration

2013-01-15 Thread Dariusz Borowski
Hi Bruno, Maybe this helps. I wrote something about it: http://www.coderthing.com/solr-with-multicore-and-database-hook-part-1/ Dariusz On Tue, Jan 15, 2013 at 9:52 AM, Bruno Dusausoy bdusau...@yp5.be wrote: Hi, I'd like to use two separate indexes (Solr 3.6.1). I've read several wiki

Solr Query | Loading documents with large content (Performance)

2013-01-15 Thread Uwe Clement
Hi there, sometimes we have to load very big documents, 1-2 multi-value-fields of it can contain 10.000 items. And unfortunately we need this informations. We have to load 50 documents in order to show to the result table in the UI. The query takes around 50 seconds. I guess 48 seconds of

Re: Multicore configuration

2013-01-15 Thread Upayavira
You should put your solr.xml into your 'cores' directory, and set -Dsolr.solr.home=cores That should get you going. 'cores' *is* your Solr Home. Otherwise, your instanceDir entries in your current solr.xml will need correct paths to ../cores/procedure/ etc. Upayavira On Tue, Jan 15, 2013, at

Re: Multicore configuration

2013-01-15 Thread Bruno Dusausoy
Dariusz Borowski a écrit : Hi Bruno, Maybe this helps. I wrote something about it: http://www.coderthing.com/solr-with-multicore-and-database-hook-part-1/ Hi Darius, Thanks for the link. I've found my - terrible - mistake : solr.xml was not in solr.home dir but in solr.home/conf dir, so it

Re: Performance issue with group.ngroups=true

2013-01-15 Thread Mickael Magniez
Hi, Retry on a better machine (2CPU, 8GB RAM, 1.5GB for java half used according admin interface) still have the same issue. It seems to grow with matches count : with a search matching 100k documents, it takes 700ms, vs 70ms without ngroup (CPU is 100% during request) For information, my index

RE: Results in same or different fields

2013-01-15 Thread Harshvardhan Ojha
Hi Gastone, I am not very sure, but I think phrase query will resolve this problem. q=title:white house will always have higher relevance that term white and house separately. Regards Harshvardhan Ojha -Original Message- From: Gastone Penzo [mailto:gastone.pe...@gmail.com] Sent:

DataImportHandlerException: Unable to execute query with OPTIM

2013-01-15 Thread ashimbose
I have tried to search for my specific problem but have not found solution. I have also read the wiki on the DIH and seem to have everything set up right but my Query still fails. Thank you for your help I am running Solr 3.6.1 with Tomcat 6.0 Windows7 64bit and IBM Optim Archive File I have all

Re: Index data from multiple tables into Solr

2013-01-15 Thread Naresh
Get user's input, form the solr query and send a request to the server (you can also pass a parameter called wt (xml,json etc) to direct solr to return output in that format). Parse the results from solr and display them to user in your website. Depending on what kind of server-side programming

Re: Performance issue with group.ngroups=true

2013-01-15 Thread Mikhail Khludnev
Mickael, I just wonder you have considered BlockJoin? it performs much better than query time approaches http://blog.griddynamics.com/2012/08/block-join-query-performs.html ,but faceting hasn't been implemented for it yet. On Tue, Jan 15, 2013 at 2:01 PM, Mickael Magniez

Re: access matched token ids in the FacetComponent?

2013-01-15 Thread Mikhail Khludnev
Dmitry, I have some relevant experience and ready to help, but I can not get the core problem. Could you please expand the description and/or provide a sample? On Tue, Jan 15, 2013 at 11:01 AM, Dmitry Kan solrexp...@gmail.com wrote: Hello! Is there a simple way of accessing the matched

Re: SOlr 3.5 and sharding

2013-01-15 Thread Erick Erickson
You're confusing shards and slaves here. Shards are splitting a logical index amongst N machines, where each machine contains a portion of the index. In that setup, you have to configure the slaves to know about the other shards, and the incoming query has to be distributed amongst all the shards

Re: SolrCloud :: Adding replica :: Sync-up issue

2013-01-15 Thread Erick Erickson
Trying again, original reply rejected as spam. This won't be all that helpful, but 4.1 has a lot of improvements as far as SolrCloud is concerned, and it's in the process of being put together now. So I suspect the best use of time would be to work with 4.1 (or a nightly build between now and

Error loading plugin

2013-01-15 Thread Mickael Magniez
Hi, I'm trying to write my own search handler, but i have problem loading it into solr. Error message is : Caused by: org.apache.solr.common.SolrException: Error loading class 'com.company.solr.GroupRequestHandler' at

Re: retrieving latest document **only**

2013-01-15 Thread J Mohamed Zahoor
The sum of all the count in the groups… does not match the total no of docs found. ./zahoor On 12-Jan-2013, at 1:27 PM, Upayavira u...@odoko.co.uk wrote: Not sure exactly what you mean, can you give an example? Upayavira On Sat, Jan 12, 2013, at 06:32 AM, J Mohamed Zahoor wrote:

Re: retrieving latest document **only**

2013-01-15 Thread Upayavira
Is your group field multivalued? Could docs appear in more than one group? Upayavira On Tue, Jan 15, 2013, at 01:22 PM, J Mohamed Zahoor wrote: The sum of all the count in the groups… does not match the total no of docs found. ./zahoor On 12-Jan-2013, at 1:27 PM, Upayavira

RE: SOlr 3.5 and sharding

2013-01-15 Thread Jean-Sebastien Vachon
Hi Erick, Thanks for your comments but I am migrating an existing index (single instance) to a sharded setup and currently I have no access to the code involved in the indexation process. That`s why I made a simple copy of the index on each shards. In the end, the data will be distributed

Tutorial for Solr query language, dismax and edismax?

2013-01-15 Thread eShard
Does anyone have a great tutorial for learning the solr query language, dismax and edismax? I've searched endlessly for one but I haven't been able to locate one that is comprehensive enough and has a lot of examples (that actually work!). I also tried to use wildcards, logical operators, and a

Re: SOlr 3.5 and sharding

2013-01-15 Thread Upayavira
He was referring to master/slave setup, where a slave will poll the master periodically asking for index updates. That frequency is configured in solrconfig.xml on the slave. So, you are saying that you have, say 1m documents in your master index. You then copy your index to four other boxes. At

Re: how to optimize same query with different start values

2013-01-15 Thread Upayavira
You are setting yourself up for disaster. If you ask Solr for documents 1000 to 1010, it needs to sort documents 1 to 1010, and discard the first 1000, which causes horrible performance. I'm curious to hear if others have strategies to extract content sequentially from an index. I suspect a new

Re: Results in same or different fields

2013-01-15 Thread Uwe Reh
Hi, maybe it helps to have a closer look on the other params of edismax. http://wiki.apache.org/solr/ExtendedDisMax#pf_.28Phrase_Fields.29 'mm=2' will be to strong, but th usage of pf, pf2, and pf is likely your solution. uwe Am 15.01.2013 10:15, schrieb Gastone Penzo: Hi, i'm using

RE: SOlr 3.5 and sharding

2013-01-15 Thread Jean-Sebastien Vachon
Ok I see what Erick`s meant now.. Thanks. The original index I`m working on contains about 120k documents. Since I have no access to the code that pushes documents into the index, I made four copies of the same index. The master node contains no data at all, it simply use the data available in

Re: how to optimize same query with different start values

2013-01-15 Thread Mikhail Khludnev
It's a well know search engines limitation. This post will help you get into the core problem http://www.searchworkings.org/blog/-/blogs/lucene-solr-and-deep-paging . it seems that the solution is contributed into Lucene, but not yet for Solr. On Tue, Jan 15, 2013 at 6:36 PM, Upayavira

RE: DataImportHandlerException: Unable to execute query with OPTIM

2013-01-15 Thread Dyer, James
I think your JDBC driver is complaining because it doesn't like what is being set for the fetch size on the Statement. Fetch size is controlled by the batchSize parameter on dataSource / . Using batchSize=-1, I believe, is a workaround for MySql but I suspect your driver requires it to be 0

RE: Disabling document cache usage

2013-01-15 Thread Markus Jelsma
No, SolrIndexSearcher has no mechanism to do that. The only way is to disable the cache altogether or patch it up :) -Original message- From:Otis Gospodnetic otis.gospodne...@gmail.com Sent: Tue 15-Jan-2013 16:57 To: solr-user@lucene.apache.org Subject: Disabling document cache

Re: Disabling document cache usage

2013-01-15 Thread Otis Gospodnetic
Hi, Thanks Markus. How are caches disabled these days... in Solr 4.0 that is? I remember trying to comment them out in the past, but seeing them still enabled and used with some custom size and other settings. Thanks, Otis -- Solr ElasticSearch Support http://sematext.com/ On Tue, Jan 15,

V 4.0.0.0 insert

2013-01-15 Thread Николай Измаилов
I don't understand how to add data into the document. I created a core in version 4.0.0 test_core I can read the data on solr/test_core/select and insert does not work. How to add data?

Re: Search across a specified number of boundaries

2013-01-15 Thread Mike Ree
Mikhail, Yeah, I considered that originally, but then after analyzing the data noticed that was not possible. Some of the content we analyze contains large tables that after ocr get turned into long running sentences which contain 500k+ words per a sentence. Overall there are probably around 10k

Re: how to optimize same query with different start values

2013-01-15 Thread Andre Bois-Crettez
It looks like a use case for using Solrj with queryAndStreamResponse ? http://lucene.apache.org/solr/api-4_0_0-BETA/org/apache/solr/client/solrj/SolrServer.html#queryAndStreamResponse%28org.apache.solr.common.params.SolrParams,%20org.apache.solr.client.solrj.StreamingResponseCallback%29 André

Re: Index data from multiple tables into Solr

2013-01-15 Thread hassancrowdc
Hi, once i have indexed data from multiple tables from mysql database into solr, is there any way that it update data(automatically) if any change is made to the data in mysql? On Tue, Jan 15, 2013 at 6:13 AM, Naresh [via Lucene] ml-node+s472066n403343...@n3.nabble.com wrote: Get user's

Re: Tutorial for Solr query language, dismax and edismax?

2013-01-15 Thread Walter Underwood
You should not need to use wildcards. Most configurations of Solr will index space-separated words as separate tokens. They can be matched separately. DId you use a string field type (probably the wrong choice)? How are your fields tokenized? Solr/Lucene query syntax:

Re: Suggestion that preserve original phrase case

2013-01-15 Thread Selvam
Thanks Erick, can you tell me how to do the appending (lowercaseversion:LowerCaseVersion) before indexing. I tried pattern factory filters, but I could not get it right. On Sun, Jan 13, 2013 at 8:49 PM, Erick Erickson erickerick...@gmail.comwrote: One way I've seen this done is to index pairs

RE: Disabling document cache usage

2013-01-15 Thread Markus Jelsma
Hi, Commenting them out works fine. We don't use documentCaches either as they eat too much and return only so little. Cheers -Original message- From:Otis Gospodnetic otis.gospodne...@gmail.com Sent: Tue 15-Jan-2013 17:29 To: solr-user@lucene.apache.org Subject: Re: Disabling

Re: V 4.0.0.0 insert

2013-01-15 Thread Alexandre Rafalovitch
Have you gone through the tutorial on the wiki first? It should cover basic use cases. If you have, how do you send the data in? Regards, Alex On 15 Jan 2013 11:22, Николай Измаилов bob...@mail.ru wrote: I don't understand how to add data into the document. I created a core in version

Re: Solr Query | Loading documents with large content (Performance)

2013-01-15 Thread Otis Gospodnetic
Hi, Have a look under http://wiki.apache.org/solr/UpdateCSV#Methods_of_uploading_CSV_recordsabout uploading a *local* file. Otis -- Solr ElasticSearch Support http://sematext.com/ On Tue, Jan 15, 2013 at 3:59 AM, Uwe Clement uwe.clem...@exxcellent.dewrote: Hi there, sometimes we have

SolrCloud Performance for High Query Volume

2013-01-15 Thread Niran Fajemisin
Hi all, I'm currently in the process of doing some performance testing in preparations for upgrading from Solr 3.6.1 to Solr 4.0. (We're badly in need of NRT functionality) Our existing deployment is not a typical deployment for Solr, as we use it to search and facet on financial data such as

Re: Index data from multiple tables into Solr

2013-01-15 Thread Shawn Heisey
On 1/15/2013 9:20 AM, hassancrowdc wrote: Hi, once i have indexed data from multiple tables from mysql database into solr, is there any way that it update data(automatically) if any change is made to the data in mysql? You need to write a program to do this. Although this list can provide

Re: Stored hierachical data in Solr

2013-01-15 Thread Upayavira
You can store structured data in Solr. You can't *query* it, in such a way as respects its structure. E.g. If I had xmlthisband/bthat/xml, I could parse that into terms: [this] [and] [that], and do searches upon them. But you couldn't search for documents that match an xpath such as

RE: Index data from multiple tables into Solr

2013-01-15 Thread Swati Swoboda
He is talking about this list, the list we are using to communicate. You are sending your messages to a mailing list -- thousands are on it. Example of programs that will run the delta-import/full-import commands: Cron You are basically calling a URL with specific parameters to pull data from

RE: Index data from multiple tables into Solr

2013-01-15 Thread Swati Swoboda
https://wiki.apache.org/solr/Solrj client. You'd have to configure it / use it based on your application needs. -Original Message- From: hassancrowdc [mailto:hassancrowdc...@gmail.com] Sent: Tuesday, January 15, 2013 2:38 PM To: solr-user@lucene.apache.org Subject: Re: Index data from

Re: Top Terms Using Luke

2013-01-15 Thread Shawn Heisey
On 1/15/2013 11:54 AM, Lighton Phiri wrote: I would like to get a sense of the top terms for fields in my index and just enable the LukeRequestHandler [1] in my solrconfig.xml file. However, Luke seems to include stopwords as well. I've tried searching previous threads but nothing I've come

Re: Index data from multiple tables into Solr

2013-01-15 Thread Shawn Heisey
On 1/15/2013 12:00 PM, hassancrowdc wrote: Which list are you reffering to? The solr-user mailing list that we are both using here. and can you please give an example of such program(doesn't matter if it is for your setup)? I can't do that. It is confidential and proprietary code.

Solr exception when parsing XML

2013-01-15 Thread Zhang, Lisheng
Hi, I got SolrException when submitting XML for indexing (using solr 3.6.1) Jan 15, 2013 10:22:42 AM org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: Illegal character ((CTRL-CHAR, cod e 31)) at [row,col {unknown-source}]: [2,1169] at

Re: Index data from multiple tables into Solr

2013-01-15 Thread hassancrowdc
okay, thank you. After indexing data from database to solr. I want to search such that if i write any word (that is included in the documents been indexed) it should return all the documents that include that word. But it does not. When i write http://localhost:8983/solr/select?q=anyword i

RE: Index data from multiple tables into Solr

2013-01-15 Thread Swati Swoboda
What error are you getting? Which field are you searching (default field)? Did you try specifying a default field? What is your schema like? Which analyzers did you use? Which version of solr are you using? I highly recommend going through the tutorial to get a basic understanding of

Re: Index data from multiple tables into Solr

2013-01-15 Thread hassancrowdc
I dont want to search by one field, i want to search as a whole. I am following that tutorial i got indexing, updating but now for search i would like to search through everything i have indexed not a specific field. I can do by using defaultfield but i would like to search through everything i

Re: Solr exception when parsing XML

2013-01-15 Thread Alexandre Rafalovitch
Interesting point. Looks like CDATA is more limiting than I thought: http://en.wikipedia.org/wiki/CDATA#Issues_with_encoding . Basically, the recommendation is to avoid CDATA and automatically encode characters such as yours, as well as less/more and ampersand. Regards, Alex.

RE: Index data from multiple tables into Solr

2013-01-15 Thread Swati Swoboda
http://wiki.apache.org/solr/ExtendedDisMax Specify your query fields in the qf parameter. Take a look at the example at the bottom of the page. -Original Message- From: hassancrowdc [mailto:hassancrowdc...@gmail.com] Sent: Tuesday, January 15, 2013 3:56 PM To:

Missing documents with ConcurrentUpdateSolrServer (vs. HttpSolrServer) ?

2013-01-15 Thread Mark Bennett
First off, just reporting this: I wound up with approx 58% few documents having submitted via ConcurrentUpdateSolrServer. I went back and changed the code to use HttpSolrServer and had 100% This was a long running test, approx 12 hours, with gigabytes of data, so conveniently shared /

Re: Index data from multiple tables into Solr

2013-01-15 Thread Shawn Heisey
On 1/15/2013 1:37 PM, hassancrowdc wrote: After indexing data from database to solr. I want to search such that if i write any word (that is included in the documents been indexed) it should return all the documents that include that word. But it does not. When i write

Re: Synonyms and trailing wildcard

2013-01-15 Thread Jack Krupansky
It's certainly true that wildcard suppresses the synonym filter since it is not multi-term aware. Other than implementing your own version of the synonym filter that was multi-term aware and interpreted wildcards, you may have to do your own preprocessor. Or, you could do index-time

Re: Missing documents with ConcurrentUpdateSolrServer (vs. HttpSolrServer) ?

2013-01-15 Thread Shawn Heisey
On 1/15/2013 2:10 PM, Mark Bennett wrote: First off, just reporting this: I wound up with approx 58% few documents having submitted via ConcurrentUpdateSolrServer. I went back and changed the code to use HttpSolrServer and had 100% This was a long running test, approx 12 hours, with gigabytes

from 1.4 to 3.6

2013-01-15 Thread kaveh minooie
HI I hope this doesn't turn out to be a very stupid question. I have upgraded from solr 1.4 to 3.6 and now in the response that I am getting from solr maxScore field in the [response] is missing. I am doing something wrong? how can I get it back? thanks, -- Kaveh Minooie www.plutoz.com

Re: from 1.4 to 3.6

2013-01-15 Thread Shawn Heisey
On 1/15/2013 4:14 PM, kaveh minooie wrote: HI I hope this doesn't turn out to be a very stupid question. I have upgraded from solr 1.4 to 3.6 and now in the response that I am getting from solr maxScore field in the [response] is missing. I am doing something wrong? how can I get it back?

Is *:* the only possible search with * on the left-hand-side?

2013-01-15 Thread Alexandre Rafalovitch
Hello, Is *:* hardcoded somewhere as a unique special pattern or is there actually a class of queries with *:'something'? I tried searching for it, but I suspect this is not the patterns most tokenizers will actually index as searchable. :-) Regards, Alex. Personal blog:

Re: Is *:* the only possible search with * on the left-hand-side?

2013-01-15 Thread Jack Krupansky
Semi-hard-coded. In QueryParserBase.java: protected Query getWildcardQuery(String field, String termStr) throws ParseException { if (*.equals(field)) { if (*.equals(termStr)) return newMatchAllDocsQuery(); Otherwise, if you try *:x, * is an undefined field. -- Jack Krupansky

Re: Top Terms Using Luke

2013-01-15 Thread Lighton Phiri
I suppose this will do; I just figured they'd be a built-in way of excluding stopwords. Thank you. On 15 January 2013 22:08, Shawn Heisey s...@elyograg.org wrote: To get an idea for which non-stopwords are dominant in your index, just ask for more top terms, instead of just the top ten or top

RE: DataImportHandlerException: Unable to execute query with OPTIM

2013-01-15 Thread ashimbose
Dear James Dyer , Thank You Very Much. Its really working now. I was struggling past 3 weeks to solve it. You are really awesome. I am really happy now. Thank you to make me happy. Regards, Ashim -- View this message in context: