Re: Solr 4.x/3.x update javabin incompatibility?

On 5/10/2013 8:56 PM, Michael Sokolov wrote: > On 5/10/2013 10:18 PM, Shawn Heisey wrote: >> I don't know why I'm not having any trouble. I'm certainly glad that >> I'm not, though! Thanks, Shawn > Shawn, one question - in your server setup do you have: > > _querySolr.setRequestWriter(new BinaryRe

Re: Solr 4.x/3.x update javabin incompatibility?

On 5/10/2013 10:18 PM, Shawn Heisey wrote: I don't know why I'm not having any trouble. I'm certainly glad that I'm not, though! Thanks, Shawn Shawn, one question - in your server setup do you have: _querySolr.setRequestWriter(new BinaryRequestWriter()); ? I didn't see that - it (used to be)

Re: Solr 4.x/3.x update javabin incompatibility?

On 5/10/2013 10:18 PM, Shawn Heisey wrote: On 5/10/2013 7:42 PM, Michael Sokolov wrote: My question is: is this intentional? It's unfortunate that we don't seem to be able to update the client and have it continue to work with (ie send updates to) the old servers. We have a centralized client

Re: Solr 4.x/3.x update javabin incompatibility?

On 5/10/2013 7:42 PM, Michael Sokolov wrote: > My question is: is this intentional? It's unfortunate that we don't > seem to be able to update the client and have it continue to work with > (ie send updates to) the old servers. We have a centralized client > library that we share across a large n

Re: Solr 4.x/3.x update javabin incompatibility?

Just a quick followup to this - I see now that this change is logged in jira as SOLR-2857. It doesn't seem that anybody considered the backwards incompatibility of new client/old server there. And it's not really clear to me how I can get SolrJ to emit requests to /update/javabin - is there a

Solr 4.x/3.x update javabin incompatibility?

I upgraded one of my solrj clients to 4.2.0, and am testing using it with a 3.4 server. We generally use a BinaryRequestWriter (ie javabin). With the 3.4 solrj client, this caused update requests to be directed to /update/javabin. However, in 4.2, the dispatch seems to be getting handled usi

Re: Tika not extracting content from ODT / ODS (open document / libreoffice) in Solr 4.2.1

I would try DIH with the flags as in jira issue I linked to. If possible. Just in case. Regards, Alex On 10 May 2013 19:53, "Sebastián Ramírez" wrote: > OK Jack, I'll switch to MS Office ...hahaha > > Many thanks for your interest and help... and the bug report in JIRA. > > Best, > > Sebast

Multi dimensional spatial search

Hi, Does Solr support multi dimensional spatial search ? http://en.wikipedia.org/wiki/K-d_tree Thanks

Is there a migration document for migration SOLR 3.x to SOLR 4.x?

Hi, We have created lots of custom components using SOLR 3.5 API. we are now in the process of migrating the components to work with SOLR 4.2 API but its becoming very difficult to make the changes to our custom components by going through the API one by one and understanding each and every method

Re: Tika not extracting content from ODT / ODS (open document / libreoffice) in Solr 4.2.1

OK Jack, I'll switch to MS Office ...hahaha Many thanks for your interest and help... and the bug report in JIRA. Best, Sebastián Ramírez On Fri, May 10, 2013 at 5:48 PM, Jack Krupansky wrote: > I filed SOLR-4809 - "OpenOffice document body is not indexed by > SolrCell", including some test

Re: Limiting number of facet results for path based facets

Actually it occurred to me that doing something like this might work: if I have a category of root/books/scifi for a document I'd create the following values for the facet: 0/root 1/root 1/root/books 2/root 2/root/books 2/root/books/scifi Then to get level 1 I'd do: http://localhost

Limiting number of facet results for path based facets

Hi, I'm using the PathHierarchyTokenizer like this: to create path based facets. A query like: http://localhost:8982/solr/articles/select?facet=true&start=0&q=*:*&facet.field=my_path_based_facet&wt=ruby&indent=on gives facets that look like

Re: Tika not extracting content from ODT / ODS (open document / libreoffice) in Solr 4.2.1

I filed SOLR-4809 - "OpenOffice document body is not indexed by SolrCell", including some test files. https://issues.apache.org/jira/browse/SOLR-4809 Yeah, at this stage, switching to Microsoft Office seems like the best bet! -- Jack Krupansky -Original Message- From: Sebastián Ramí

Re: SOLR 4.3.0 - How to use DeleteUpdateCommand?

Shawn, This is indeed a custom SOLR update component used by the message queue to perform various operations (add, update , delete etc..). We have implemented custom logic in this component... Thanks, BB -- View this message in context: http://lucene.472066.n3.nabble.com/SOLR-4-3-0-Migration

Re: Tika not extracting content from ODT / ODS (open document / libreoffice) in Solr 4.2.1

Many thanks Jack for your attention and effort on solving the problem. Best, Sebastián Ramírez On Fri, May 10, 2013 at 5:23 PM, Jack Krupansky wrote: > I downloaded the latest Apache OpenOffice 3.4.1 and it does in fact fail > to index the proper content, both for .ODP and .ODT files. > > If I

Re: Tika not extracting content from ODT / ODS (open document / libreoffice) in Solr 4.2.1

I downloaded the latest Apache OpenOffice 3.4.1 and it does in fact fail to index the proper content, both for .ODP and .ODT files. If I do extractOnly=true&extractFormat=text, I see the extracted text clearly in addition to the metadata. I tested on 4.3, and then tested on Solr 3.6.1 and it

Re: Does Distributed Search are Cached Only the By Node That Runs Query?

I have 5 shard and they are at Amazon EC2 as Large instances. I just make some tests and when I start pre-production step at my data center I will have many Solr machines and millions of documents so this issue maybe a problem for me. 2013/5/10 Jason Hellman > And for 10,000 documents across n

Re: SOLR 4.3.0 - How to use DeleteUpdateCommand?

On 5/10/2013 2:57 PM, bbarani wrote: I am in the process of migrating our existing SOLR (version 3.5) to new version of SOLR (4.3.0). Currently we delete the document from SOLR using the below code (deletion happens via message queue). UpdateRequestProcessor processor = _getProcessor();

Re: Install Solr on EC2

Done -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com 10. mai 2013 kl. 22:57 skrev Michael Della Bitta : > I can update this page if someone adds me as a contributor: > MichaelDellaBitta. > > > Michael Della Bitta > >

Re: Install Solr on EC2

I can update this page if someone adds me as a contributor: MichaelDellaBitta. Michael Della Bitta Appinions 18 East 41st Street, 2nd Floor New York, NY 10017-6271 www.appinions.com The science of influence marketing. On Fri, May 10, 2013 at 8

SOLR 4.3.0 - How to use DeleteUpdateCommand?

I am in the process of migrating our existing SOLR (version 3.5) to new version of SOLR (4.3.0). Currently we delete the document from SOLR using the below code (deletion happens via message queue). UpdateRequestProcessor processor = _getProcessor(); // return SOLR core information Dele

Re: SolrCloud: IOException occured when talking to server at

On 5/10/2013 1:06 PM, heaven wrote: Again, just finished reindexing, server utilization was about 5-10%, I started index optimization. As result I now lost (again) entire index, got a lot of errors, they are appear so fast and contain 0 useful information.

Re: dataimport handler

Hmm, I will fix. https://issues.apache.org/jira/browse/SOLR-4788 On Thu, May 9, 2013 at 8:35 PM, William Bell wrote: > It does not work anymore in 4.x. > > ${dih.last_index_time} does work, but the entity version does not. > > Bill > > > > On Tue, May 7, 2013 at 4:19 PM, Shalin Shekhar Mangar

Re: Tika not extracting content from ODT / ODS (open document / libreoffice) in Solr 4.2.1

Thanks Walter and Alex, You are right Walter. In fact, if I'm not wrong, Tika doesn't use an externar parser for those formats as it does with MS Office files or PDFs, it uses java ZIP and XML libraries to parse those files directly. I guess that would be my last resort. But I would certainly like

Re: Per Shard Replication Factor

There's an issue already: https://issues.apache.org/jira/browse/SOLR-4808 On Fri, May 10, 2013 at 11:50 AM, Joel Bernstein wrote: > I agree this would be a nice feature. Steven can update this thread with > ticket? Thanks Joel > > > On Fri, May 10, 2013 at 9:58 AM, Steven Bower wrote: > > > Th

Re: SolrCloud: IOException occured when talking to server at

Again, just finished reindexing, server utilization was about 5-10%, I started index optimization. As result I now lost (again) entire index, got a lot of errors, they are appear so fast and contain 0 useful information.

Re: Tika not extracting content from ODT / ODS (open document / libreoffice) in Solr 4.2.1

On Fri, May 10, 2013 at 11:24 AM, Sebastián Ramírez wrote: > Hello everyone, > > I'm having a problem indexing content from "opendocument format" files. The > files created with OpenOffice and LibreOffice (odt, ods...). I wonder if it is connected to https://issues.apache.org/jira/browse/SOLR-45

Re: Per Shard Replication Factor

I agree this would be a nice feature. Steven can update this thread with ticket? Thanks Joel On Fri, May 10, 2013 at 9:58 AM, Steven Bower wrote: > This approach would work to satisfy the requirement but I think would > generally be nice to have the ability to control this within a single > col

Re: Does Distributed Search are Cached Only the By Node That Runs Query?

And for 10,000 documents across n shards, that can be significant! On May 10, 2013, at 11:43 AM, Joel Bernstein wrote: > How many shards are in your collection? The query aggregator node will pull > pack that results from each shard and hold the results in memory. Then it > will add the results

Re: SOLR guidance required

One more tip on the use of filter queries. DO: &fq=name1:value1&fq=name2:value2&fq=namen:valuen DON'T: fq=name1:value1 AND name2:value2 AND name3:value3 Where OR operators apply, this does not matter. But your Solr cache will be much more savvy with the first construct. Jason On May 10, 20

Re: Does Distributed Search are Cached Only the By Node That Runs Query?

How many shards are in your collection? The query aggregator node will pull pack that results from each shard and hold the results in memory. Then it will add the results to a priority queue to sort them. This queue will need to be as large as the page that is being generated. After the query is f

Re: SOLR guidance required

Aditya, As suggested by others, definitely you should use the filter queries directly to query SOLR. Just keep your indexes updated. Keep all your fields indexed/stored as per your requirements. Refer through the filter query wiki http://wiki.apache.org/solr/CommonQueryParameters

Re: Tika not extracting content from ODT / ODS (open document / libreoffice) in Solr 4.2.1

The last time I looked at those formats, they were a zip archive with the content in an XML file. I think it was a an obvious name, like "content.xml". So you should be able to extract that and look at it. Opening XML in a browser can be helpful, because it will flag any parse errors. wunder O

Re: Tika not extracting content from ODT / ODS (open document / libreoffice) in Solr 4.2.1

Thanks for your reply Jack! First: LOL Second: I'm using the latest version of libreoffice, but with the "extractOnly" param in the Solr request it shows the content of the file so Tika is being able to read and extract the data but Solr isn't indexing that data. Third: I already did that with n

Error on recovery

Hi, I have a node that can't finish the recovery. The log shows this error: 3836028 [RecoveryThread] ERROR org.apache.solr.cloud.RecoveryStrategy – Recovery failed - trying again... (0) core=ST-XXX_0712 3836028 [RecoveryThread] ERROR org.apache.solr.cloud.RecoveryStrategy – Recovery failed - in

Re: Sharing index data between two Solr instances

Milen, It is possible to have the configuration shared amongst multiple cores, I have seen this…though I haven't seen multiple separate instances share the same solr.xml core configuration (and, for that matter, separate possible locking policies). It might work. Honestly, I don't like it. Y

How to get ResponseBuilder's _responseDocs value?

Hi, I am trying to override query component to implement a custom logic but I am not sure how to get the _responseDocs value since this is not visible in my class (The field ResponseBuilder._responseDocs is not visible). Is there a way to get this value? public class CustomqueryComponent extends

Re: Tika not extracting content from ODT / ODS (open document / libreoffice) in Solr 4.2.1

Switching to Microsoft Office will probably solve your problem! Sorry, I couldn't resist. Are you using a really new or really old version of the ODT/ODS software? I mean, maybe Tika doesn't have support for that version. Check the mime type that Tika generates - maybe you just need to overri

AW: Sharing index data between two Solr instances

Hello Peter, thanks for Your answer! Indeed, we are considering replication as an alternative design. However, we are appealed by the idea of having the updated index almost instantly after successful import. Decision is still pending, though. Thanks and kind regards! Milen -Ursprünglic

Re: Sharing index data between two Solr instances

Hello Milen, We do something very similar to this, except we use separate processes on the same machine for the writer and reader. We do this so we can tune caches etc. to optimize for each, and still use the same index files. On MP machines, this works very well. If you've got 2 separate machines

AW: Sharing index data between two Solr instances

Hello Jason, Thanks for Your quick response! The alternative of using the Solr replication is also still pending at this point, so we will consider its pros and cons, too. Fortunately, we are not using AutoCommit in our project, as we need to control the creation of new segments, so I will prop

RE: Looking for Best Practice of Spellchecker

Good point, Jason. In fact, even if you use WorkBreakSpellChecker "wall mart" will not correct to "walmart". The reason is the spellchecker cannot both correct a token's spelling *and* fix the wordbreak issue involving that same token. So in this case a synonym is the way to go. James Dyer I

Re: Sharing index data between two Solr instances

Milen, At some point you'll need to call a commit to search your data, either via AutoCommit policy or deterministically. There are various schools of though on which way to go but something needs to do this. If you go the AutoCommit route be sure to pay attention to the openSearcher value.

Tika not extracting content from ODT / ODS (open document / libreoffice) in Solr 4.2.1

Hello everyone, I'm having a problem indexing content from "opendocument format" files. The files created with OpenOffice and LibreOffice (odt, ods...). Tika is being able to read the files but Solr is not indexing the content. It's not a problem of commiting or something like that, after I post

Sharing index data between two Solr instances

Hello together! I've been googleing on this topic but still couldn't find a definitive answer to my question. We have a setup of two machines both running Solr 4.2 within Tomcat. We are considering sharing the index data between both webapps. One of the machines will be configured to update th

Unable to load environment info from /solr/collection1/admin/system?wt=json

I was making some query tests at one of my nodes at SolrCloud (Solr 4.2.1) When I look at admin page of my node I start to see that: "Unable to load environment info from /solr/collection1/admin/system?wt=json. This interface requires that you activate the admin request handlers in all SolrCores

Re: Looking for Best Practice of Spellchecker

Nicholas, Also consider that some misspellings are better handled through Synonyms (or injected metadata). You can garner a great deal of value out of the spell checker by following the great advice James is giving here…but you'll find a well-placed "helper" synonym or metavalue can often sa

Re: Negative Boosting at Recent Versions of Solr?

You learned the gosh-darndest things: http://localhost:8983/solr/browse?q=ipod&boost=product(price,-2)&debugQuery=on …nets: -0.3797992 = (MATCH) sum of: 0.13510442 = (MATCH) max of: 0.045963455 = (MATCH) weight(text:ipod^0.5 in 4) [DefaultSimilarity], result of: 0.045963455 = score(

RE: Looking for Best Practice of Spellchecker

Nicholas, It sounds like you might want to use WordBreakSolrSpellChecker, which gets obscure mention in the wiki. Read through this section: http://wiki.apache.org/solr/SpellCheckComponent#Configuration and you will see some information. Also, the Solr Example shows how to configure this.

RE: Negative Boosting at Recent Versions of Solr?

Despite the discussion in SOLR-3823/SOLR-3278, my experience with Solr 4.2 is that it does indeed allow negative boosts on both "bf" and "qf". I think the functionality was added under the radar possibly with SOLR-4093, not sure though. In disbelief, I did some testing and it seems to really w

Re: Index compatibility from 4.0 to 4.3

You're probably right, that it's only new and merged segments that will be of the new format. Solr 4.3 will be able to search the old segments. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com 10. mai 2013 kl. 15:27 skrev Shawn Heisey : > On 5/10/2013 5:11 AM, Jan Høyd

Re: Per Shard Replication Factor

This approach would work to satisfy the requirement but I think would generally be nice to have the ability to control this within a single collection (so you don't give up any functionality when querying between the collections and to make the management of the system easier). Anyway I'll create

Looking for Best Practice of Spellchecker

Hi guys, I'm working on a local search project, I wanna integrate spellchecker for the search. So basically, my search engines is used to search local businesses. For example, user could search for "wall mart", here is a typo, I wanna spellchecker to give me Collation for "walmart". My problems

Re: Index compatibility from 4.0 to 4.3

On 5/10/2013 5:11 AM, Jan Høydahl wrote: > Hi, > > The fastest way to upgrade is to leave luceneMatchVersion as LUCENE_40. Then > you will not get any surprises wrt changed behavior for existing > functionality. but you will get all the bug fixes. You can test it out on > your dev box. I've wo

Re: Index corrupted detection from http get command.

Would a non-successful /select?q={!cache=false}*:* query be sufficient? (if so, /admin/ping would do the trick) What's causing index corruption in your environment? That's a very unusual occurrence that would be good to have identified in gory detail if you can. Erik On May 7, 2013,

Re: Index corrupted detection from http get command.

Thanks for your help. On Thu, May 9, 2013 at 7:35 PM, Erick Erickson wrote: > There's no way to do this that I know of. There's the checkindex > tool, but it's fairly expensive resource-wise and there's no HTTP > command to do it. > > Best > Erick > > On Tue, May 7, 2013 at 8:04 PM, Michel D

Re: SOLR guidance required

Hi Kamal, It is feasible and that is the correct approach. Add additional fields like salary, experience etc to the index and filter the results. This way you could directly show the results to the user. It is always better to avoid two searches one in solr and other in db. You should maintain se

Re: Install Solr on EC2

If you find a good recipe, perhaps update the outdated WIKI page http://wiki.apache.org/solr/SolrOnAmazonEC2 ? -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com 10. mai 2013 kl. 13:55 skrev Rajesh Nikam : > Hi All, > > I am looking for steps to run Solr 3.6.2 or latest

License_ Solr

Hi, I am having an issue with some of Apache Solr's licensing. In the source code of Apache Solr 4.2.0 there is an unclear license reference in * \solr-4.2.0\solr\webapp\web\js\lib\jquery-1.7.2.min.js and * \solr-4.2.0\solr\webapp\web\js\require.jstxt Can you please t

Solr Admin: what is "Current" meaning?

Hi, In the solr admin web interface, when looking at the statistics of a collection (this page: http://{ip}:8080/{index}/#/collection1), there is "Current" under Optimized. What does it mean? Thanks. This transmission is strictly confidential, possibly legally privileged, and intended solely

Install Solr on EC2

Hi All, I am looking for steps to run Solr 3.6.2 or latest stable version on Amazon EC2. I want this image to saved once created. Could you please help with the steps that needs to be followed. I have tried steps from https://github.com/sunspot/sunspot/wiki/Configure-Solr-on-Ubuntu,-the-quickes

Re: Portability of Solr index

Hi Mukesh, This seems like something lucene developers should be aware of - you have probably spent quiet some time to find problem/solution. Could you create a JIRA ticket? Roman On 10 May 2013 03:29, "mukesh katariya" wrote: > There is a problem with Base64 encoding. There is a project specifi

Issue with getting highlight with hl.maxAnalyzedChars = -1

I am facing one weird issue while setting hl.maxAnalyzedChars to -1 to fetch highlight for some random records, for other records its working fine. Below is my solr query http://localhost:8080/solr/core0/select?q=(text:"new year") AND (id:2343287)&hl=on&hl.fl=text&hl.fragsize=500&hl.maxAnalyzedC

Re: Index compatibility from 4.0 to 4.3

Hi, The fastest way to upgrade is to leave luceneMatchVersion as LUCENE_40. Then you will not get any surprises wrt changed behavior for existing functionality. but you will get all the bug fixes. You can test it out on your dev box. The first time Solr reads the existing index, it will be conv

Index compatibility from 4.0 to 4.3

Hi, we are planning to update Solr installations from 4.0 to 4.3 and want to avoid reindexing. But solrconfig.xml contains the following comment for : "... Generally, you want to use the latest version to get all bug fixes and improvements. It is highly recommended that you fully re-index af

Re: More Like This and Caching

Hi David, Jason and Otis, Thank you for the feedback on the question. It is very much appreciated. To confirm what caches are being used, I will remove on of the Solr servers from the cluster, restart it, note the status of the various Solr caches, issue some MLT queries to it, and compare the st

Re: Re: SolrCloud: IOException occured when talking to server at

CPU utilization is 90-100%, IO wait is 0.5-3%, The server is: CPU 2 x Xeon E5606 2.13GHz 8MB Cache 4.8GT/sec (8 total cores) Memory 32 GB RAM Disk Space 10 x 300GB 15k RPM Seagate SAS Raid Card(s) OS scientific 6.3 This high CPU usage is only during indexing, since we have to recreate the whol

Re: SolrCloud: IOException occured when talking to server at

On 5/10/2013 2:00 AM, heaven wrote: > UPD: > Forget to confirm, we're using Solr 4.2.1 and will wait for Solr 4.3.1 or > for 4.4 as you advised. Solr 4.2.1 should be pretty stable. You mentioned on the previous message that your server load is below 16. This is very high. What is your CPU utili

Re: SolrCloud: IOException occured when talking to server at

UPD: Forget to confirm, we're using Solr 4.2.1 and will wait for Solr 4.3.1 or for 4.4 as you advised. -- View this message in context: http://lucene.472066.n3.nabble.com/ColrCloud-IOException-occured-when-talking-to-server-at-tp4061831p4062235.html Sent from the Solr - User mailing list archiv

Re: SolrCloud: IOException occured when talking to server at

Hi Shawn, thank you for the reply and for your advises, will try all of them today. Some of them are already applied, i.e. "Stop other software" and "zkClientTimeout". Timeout set to 60 seconds, also reduced autowarm count and increased autoCommit interval to 5 minutes. Situation improved now and

Re: Portability of Solr index

There is a problem with Base64 encoding. There is a project specific requirement where i need to do some processing on solr string field type and then base64 encode it. I was using Sun's base64 encoder which is dependent on the JRE of the system. So when i used to index the base64 it was adding sys