6.4.0 collection leader election and recovery issues

Hello, Yesterday I upgraded from 6.0.1 to 6.4.0, its been straight 12 hours of debugging spree!! Can somebody kindly help me out of this misery. I have a set has 8 single shard collections with 3 replicas. As soon as I updated the configs and started the servers one of my collection got

How long for autoAddReplica?

I added a new node an shut down a node with a shard replica on it. It has been an hour and I don’t see any activity toward making a new replica. The new node and the one I shut down are both 6.4. The rest of the 16-node cluster is 6.2.1. wunder Walter Underwood wun...@wunderwood.org

Re: How to combine third party search data as top results ?

Also this presentation discusses the RankQuery (Starting on slide 16) http://www.slideshare.net/lucidworks/managed-search-presented-by-jacob-graves-getty-images Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Feb 1, 2017 at 9:58 PM, Joel Bernstein wrote: > This type of

Re: How to combine third party search data as top results ?

This type of ranking behavior is what the RankQuery is designed to do. A RankQuery allows you to inject your own TopDocs collector into the query and take full control of the ranking. It's more complex to implement though. Here is an example RankQuery implementation:

Re: Collection will not replicate

It's worth a try to take down your entire cluster. Bring one machine back up at a time. There _may_ be something like a 3 minute wait before each of the replicas on that machine come up, the leader election process has a 180 second delay before the replicas on that node take over leadership to

Re: project related configsets need to be deployed in both data and solr install folders ?

thanks for your time! -- View this message in context: http://lucene.472066.n3.nabble.com/project-related-configsets-need-to-be-deployed-in-both-data-and-solr-install-folders-tp4317897p4318382.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: project related configsets need to be deployed in both data and solr install folders ?

Renee: Huh ... so sounds like something must have been wonky in your original install? Glad it worked out for you, and thanks for following up. : Date: Wed, 1 Feb 2017 15:09:54 -0700 (MST) : From: Renee Sun : Reply-To: solr-user@lucene.apache.org : To:

Re: project related configsets need to be deployed in both data and solr install folders ?

Hi Chris, since I have been playing with this install, and I am not certain if I have unknowingly messed some other settings. I want to avoid put in a false Jira wasting your time. I wiped out everything on my solr box and did a fresh install of solr 6.4.0, made sure my config file set are

Re: Collection will not replicate

Sounds similar to a thread last year: http://lucene.472066.n3.nabble.com/Node-not-recovering-leader-elections-not-occuring-tp4287819p4287866.html On 2/1/17, 7:49 AM, "tedsolr" wrote: I have version 5.2.1. Short of an upgrade, are there any remedies?

Re: How to combine third party search data as top results ?

I was going to say what Charlie said! I would trust Flax's work in this area :) -Doug On Wed, Feb 1, 2017 at 3:10 PM shamik wrote: > Charlie, thanks for sharing the information. I'm going to take a look and > get > back to you. > > > > -- > View this message in context: >

Re: Solr 6.0.0 Returns Blank Highlights for alpha-numeric combos

The termvectors and offsets aren't necessary, they can be beneficial for speed reasons so I'd defer them. I ran a quick test on 6.0 with your definitions and it works just fine. I did have to comment out your custom stopwords filter on the indexing but unless you're substituting for pairs like

Solr querying nested documents with ChildDocTransformerFactory, get “Parent query yields document which is not matched by parents filter”

Some background: · The data involved is catalog data, with three nested objects: Products, Items, and Skus, in that order. We have a docType field on each record as a differentiator. · The "id" field in our data is unique within datatype, but not across datatypes. We added a

RE: Need help in Tika on SolrCloud

Is there anyone to help me with my issue? Your help is much appreciated I figured out the problem but need solution In my below data-config file tikaConfig.xml is not recognized by zookeeper ( processor="TikaEntityProcessor" tikaConfig="tikaConfig.xml" )

RE: Solr 6.0.0 Returns Blank Highlights for alpha-numeric combos

Hi Erick! Thanks for the reply. The goal is to get two character terms like 1a, 1b, 2a, 2b, 3a, etc. to get highlighted in the documents. Additional testing shows that any alpha-numeric combo returns a blank highlight, regardless of length. Thus, "pr0blem" will not highlight because of the zero

Re: How to combine third party search data as top results ?

Charlie, thanks for sharing the information. I'm going to take a look and get back to you. -- View this message in context: http://lucene.472066.n3.nabble.com/How-to-combine-third-party-search-data-as-top-results-tp4318116p4318349.html Sent from the Solr - User mailing list archive at

Re: Solr 6.0.0 Returns Blank Highlights for alpha-numeric combos

How far into the text field are these tokens? The highlighter defaults to the first 10K characters under control of hl.maxAnalyzedChars. It's vaguely possible that the values happen to be farther along in the text than that. Not likely, mind you but possible. Best, Erick On Wed, Feb 1, 2017 at

Re: Do long auto commit times interfere with delete?

This should work fine. There is no requirement that a commit happen between updates and deletes for the same document. That said, this can be tricky so if you have a demonstrable case where this isn't so let us know. Best, Erick On Wed, Feb 1, 2017 at 9:53 AM, Hans Zhou

Re: Query.extractTerms dissapeared from 5.1.0 to 5.2.0

Perfect. Thanks a lot. On Wed, Feb 1, 2017 at 2:01 PM, Alan Woodward wrote: > Hi, extractTerms() is now on Weight rather than on Query. > > Alan > > > On 1 Feb 2017, at 17:43, Max Bridgewater > wrote: > > > > Hi, > > > > It seems Query.extractTerms()

Re: Solr 6.3.0 - recovery failed

I brought down the whole cluster again, and brought up one server at a time, waiting for it to go green before launching another. Now all replicas are OK, including the one that was in the perma-recovery mode before. I do notice a large amount of network activity (basically pegging the

Re: Query.extractTerms dissapeared from 5.1.0 to 5.2.0

Hi, extractTerms() is now on Weight rather than on Query. Alan > On 1 Feb 2017, at 17:43, Max Bridgewater wrote: > > Hi, > > It seems Query.extractTerms() disapeared from 5.1.0 ( > http://lucene.apache.org/core/5_1_0/core/org/apache/lucene/search/Query.html) > to

Re: Solr 6.3.0 - recovery failed

Thank you. I do not see any caused block in the solr.log. --- 2017-02-01 18:37:57.566 INFO (recoveryExecutor-3-thread-8-processing-n:bilbo:9100_solr x:Worldline2New_shard22_replica2 s:shard22 c:Worldline2New r:core_node34) [c:Worldline2New s:shard22 r:core_node34

Re: Solr 6.3.0 - recovery failed

I can't debug the code now, but if you access the logs, directly ( not from the ui), is there any " caused by" associated to the recovery failure exception? Cheers On 1 Feb 2017 6:28 p.m., "Joe Obernberger" wrote: > In HDFS when a node fails it will leave

Re: Solr 6.3.0 - recovery failed

In HDFS when a node fails it will leave behind write.lock files in HDFS. These files have to be manually removed; otherwise the shards/replicas that have write.lock files left behind will not start. Since I can't tell which physical node is hosting which shard/replica, I stop all the nodes,

Re: Solr 6.3.0 - recovery failed

Ok, it is clearer now. You have 9 solr nodes running, one per physical machine. So each node has a number cores ( both replicas and leaders). When the node died, you got a lot of indexes corrupted. I still miss why you restarted the others 8 working nodes ( I was expecting you to restart only

Re: Solr 6.3.0 - recovery failed

Thank you for the response. There are no virtual machines in the configuration. The collection has 45 shards with 3 replicas each spread across the 9 physical boxes; each box is running one copy of solr. I've tried to restart just the one node after the other 8 (and all their

Re: Solr 6.3.0 - recovery failed

Let me try to summarize . How many virtual machines on top of the 9 physical ? How many Solr processes ( replicas ? ) If you had 1 node compromised. I assume you have replicas as well right ? Can you explain a little bit better your replicas configuration ? Why you had to stop all the nodes ? I

Re: Solr Kafka DIH

Hello Joel, This definitely seems like a good feature to add. In fact I was also looking to push data into Solr from Kafka and this can be good feature to have. I have created JIRA https://issues.apache.org/jira/browse/SOLR-10086 and can contribute as well. Thanks, Susheel On Tue, Jan 31,

Do long auto commit times interfere with delete?

We have a solr cloud with a pretty long auto commit time (30 seconds for a hard commit, 180 seconds for a soft commit). We’re also doing lots of delete-then-insert operations. i.e. Each document has a product.id, and to update a product, we do an update request with { "delete": { "query":

Solr 6.3.0 - recovery failed

Hi All - I had one node in a 45 shard cluster (9 physical machines) run out of memory. I stopped all the nodes in the cluster and removed any lingering write.lock files from the OOM in HDFS. All the nodes recovered except one replica of one shard that happens to be on the node that ran out

Query.extractTerms dissapeared from 5.1.0 to 5.2.0

Hi, It seems Query.extractTerms() disapeared from 5.1.0 ( http://lucene.apache.org/core/5_1_0/core/org/apache/lucene/search/Query.html) to 5.2.0 ( http://lucene.apache.org/core/5_2_0/core/org/apache/lucene/search/Query.html ). However, I cannot find any comment on it in 5.2.0 release notes. Any

Re: Arabic words search in solr

Dear Steve,Thanks for investigating our problem. Our project is basically business directory search platform, and we have more than 100+ K business details information. I’m providing you some examples of Arabic words to reproduce the problem. please find attached word file where i explained

Re: project related configsets need to be deployed in both data and solr install folders ?

Based on your description of the problem, and the fact that explicitly setting configSetBaseDir in your solr.xml works, I suspect there is some sort of weird bug in how the "default" configSetBaseDir is determined in some diff code paths in Solr. We should definitely file a jira issue

RE: Need help in Tika on SolrCloud

Hi All, I see below code which is causing my code not to work in SolrCloud @Override public String getConfigDir() { throw new ZooKeeperException( ErrorCode.SERVER_ERROR, "ZkSolrResourceLoader does not support getConfigDir() - likely, what you are trying to do is

Re: DIH - Parent-Child-Problems - GrapQuery-Or-BlockJoin - Order with Orderlines

Have you checked https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-BlockJoinQueryParsers ? 01 февр. 2017 г. 10:42 пользователь "Kent Iversen" написал: > I'm a newbie to Solr and can't seem to get this to work, properly. Gonna > use Order with

RE: Return specific field from child documents.

ok. let's add emp.logParamsList=q,fl,rows,*row.id * *And check logs for this request?* 31 янв. 2017 г. 14:21 пользователь "Preeti Bhat" написал: Same result. Thanks, Preeti -Original Message- From: Mikhail Khludnev [mailto:m...@apache.org]

Re: Upserting doc fields from a SearchComponent

On 01/02/2017 15:55, Ugo Matrangolo wrote: Hi Erick, Personalizing a 'price' involves using an external service (luckily we could cache most of the interactions) and it is accessed using a lib that gets dropped in the Solr classpath. Hi Ugo, We built a Solr plugin 'XJoin', allowing you to

Solr 6.0.0 Returns Blank Highlights for alpha-numeric combos

Hello everyone! I'm still stuck on this issue and could really use some help. I have a Solr 6.0.0 instance that is storing documents peppered with text like "1a", "2e", "4c", etc. If I search the documents for a word, "ms", "in", "the", etc., I get the correct number of hits and the results are

Re: Streaming Expressions result-set fields not in order

Hi Joel, Thanks for your reply. I've created the jira about this, with the issue number SOLR-10085 Regards, Edwin On 28 January 2017 at 10:26, Joel Bernstein wrote: > The issue is that fields are held in HashMaps

Phrase Queries and Punctuation

Hi all, I was just thinking about Phrase Queries and punctuation ( and in general how to manage increment positions when such a sentence delimiter happens). At the moment for multi valued fields we have the "increment position gap" which allow to avoid phrase queries to span different values for

Re: Upserting doc fields from a SearchComponent

Hi Erick, Personalizing a 'price' involves using an external service (luckily we could cache most of the interactions) and it is accessed using a lib that gets dropped in the Solr classpath. What I need to do is this kind of flow: 1. Query (.. personalization params ...) 2. Find the initial

RE: Collection will not replicate

Erick, Thank you for your quick response, I appreciate your help I am new to Solr and not from Java background I have developed the code in Dev on single core and it works perfectly good On QA box I have installed Solr 6.3 on 2 nodes(SolrCloud) Made few changes to the code to fit SolrCloud I

Re: Upserting doc fields from a SearchComponent

You need to be clear about what to do when. The [subquery], is completely separate from _finding_ the top N docs. Your last question is about finding the top N. There are really two choices that spring to mind, depending on where you keep your data about the user. Solr can't magically know that

Re: Upserting doc fields from a SearchComponent

Hi, tx for the speedy response. What if I need to filter on the result matches ?? Example: I have a price I need to personalize per user/request and need then to apply filter ranges on this personalized price (show only stuff in the 10$ - 99$ range). WDYT ? Best Ugo On Wed, Feb 1, 2017 at

Re: Collection will not replicate

What version of Solr? since 5.4 there's been a FORCELEADER colelctions API call that might help. I'd run it with the newly added replicas offline. you only want it to have good replicas to choose from. Best, Erick On Wed, Feb 1, 2017 at 6:48 AM, tedsolr wrote: > Update! I

Re: Need help in Tika on SolrCloud

Not quite sure. Are all the directories you expect the Tika docs to be in available to the Solr node? Sidestepping your question, I would be very reluctant to use Tika in SolrCloud mode because you're then putting all of the heavy-duty processing on nodes that also serve queries. I have the same

Re: Upserting doc fields from a SearchComponent

If the data is from another Solr instance, consider the [subquery] Document Transformer here: https://cwiki.apache.org/confluence/display/solr/Transforming+Result+Documents#TransformingResultDocuments-[subquery] More broadly, you can write a custom DocTransformer plugin do insert anything you

Upserting doc fields from a SearchComponent

Hi, I'm trying to write a SearchComponent that personalizes on the fly a field on all the docs resulting from the initial search query. What I would like to do is to write a SearchComponent that intercepts the documents belonging to the result sets of a search query and upsert one or more of

Re: 1:n relation and function queries

Why you can't get score from child level and combining it on a parent level? On Wed, Feb 1, 2017 at 5:33 PM, Ewald Moitzi wrote: > Hello Mikhail, > > I was using the functions as fl and sort parameters, and this > gives no result. > > When sorting by score, as

Need help in Tika on SolrCloud

Hi, I am new to Solr, I have implemented Solr on single node and my code is working good When I move the same code to SolrCloud it fails (I made few changes for SolrCloud) I am trying to load data using Dataimporthandler but it throws error as below 2017-02-01 03:23:07.727 ERROR (Thread-18)

Re: Collection will not replicate

Update! I did find an error: 2017-02-01 09:23:22.673 ERROR org.apache.solr.common.SolrException :org.apache.solr.common.SolrException: Error getting leader from zk for shard shard1 Caused by: org.apache.solr.common.SolrException: Could not get leader props at

Re: 1:n relation and function queries

Hello Mikhail, I was using the functions as fl and sort parameters, and this gives no result. When sorting by score, as you did in your blog, I get the values from the child documents. (I missed the score=max parameter) However, i need to combine values from the parent and child, like this:

Collection will not replicate

I have a collection (1 shard, 2 replicas) that was doing a batch update when one solr host ran out of disk space. The batch job failed at that point, and one replica got corrupted. I deleted the bad replica. I've tried several times since then to add a new replica. The status of the request is

Re: Query structure

This was the solution. Thank you! - Original Message - From: "Maciej Ł. PCSS" To: solr-user@lucene.apache.org Sent: Wednesday, February 1, 2017 7:57:05 AM Subject: Re: Query structure You should be able to put 'facetMetatagDatePrefix4:2015 OR

Re: Fw: solr-user-unsubscribe

Gents, have you read the instructions ? Have you sent an email to : solr-user-unsubscr...@lucene.apache.org ? You don't need to send messages to the mailing list with that address as content. Just follow what's in the official Solr documentation page :

Fw: solr-user-unsubscribe

Can someone help me with unsubscription of solr emails? I tried sending "unsubscribe" emails to "solr-user@lucene.apache.org" but no luck. Thanks, Mudasseer From: Syed Mudasseer Sent: Monday, January 30, 2017 12:55 PM To:

Re: Upgrade SOLR version - facets perfomance regression

What I meant is that : "Components If you define components, the default components (see above) will not be executed, and first-components and last-components are disallowed:" Anyway it is documented in the Confluence page. If you don't override the default components, json facets will be there.

Re: Query structure

You should be able to put 'facetMetatagDatePrefix4:2015 OR facetMetatagDatePrefix4:2016' into the filtering query. Maciej W dniu 01.02.2017 o 13:43, KRIS MUSSHORN pisze: I really need some guidance on this query structure issue. I've got to get this solved today for my employer. "Help me

Re: Query structure

Why not use filtering query? I mean the 'fq' param. Regards Maciej W dniu 01.02.2017 o 13:43, KRIS MUSSHORN pisze: I really need some guidance on this query structure issue. I've got to get this solved today for my employer. "Help me Obiwan. Your my only hope" K - Original Message

Re: Query structure

I really need some guidance on this query structure issue. I've got to get this solved today for my employer. "Help me Obiwan. Your my only hope" K - Original Message - From: "KRIS MUSSHORN" To: solr-user@lucene.apache.org Sent: Tuesday, January 31, 2017

Re: Upgrade SOLR version - facets perfomance regression

And still I have a question: Is there some convertor from the legacy api to the new API? Or a search component that converts from legacy api to json facet api? I explained why I need it in my first post. Thank you -- View this message in context:

Re: Upgrade SOLR version - facets perfomance regression

Alessandro, it helped! Thank you. But I asked which changes do we do in configuration and I think these things must be documented in the reference guide. About your question, first of all I don't override default componets. Second of all, I add my own components and for many reasons (For example,

Re: 1:n relation and function queries

Ewald, Functional queries combines well with block join as well as query time join, here are examples for latter one http://blog-archive.griddynamics.com/2015/08/scoring-join-party-in-solr-53.html It must be the same for block join. What doesn't work exactly? On Wed, Feb 1, 2017 at 1:39 PM,

Re: [Benchmark SOLR] JETTY VS TOMCAT - Jetty 15% slower - need advice to improve Jetty performance

We have done some profiling with Visualvm, but nothing obvious appeared. Thank Rick for the advice. Gérald Reinhart On 02/01/2017 11:17 AM, Rick Leir wrote: There is a profiling tool in Eclipse that can show you a tree of method calls, with timing information. I have found this useful in

1:n relation and function queries

Hello, I am unsure if solr is the right solution for a problem that we have, of if it is better to stick with a relational database (and if it should be done in solr how to implement it). The explanation is a bit lengthy, but please bear with me. The problem: Sort results of a vendor search for

Re: [Benchmark SOLR] JETTY VS TOMCAT - Jetty 15% slower - need advice to improve Jetty performance

There is a profiling tool in Eclipse that can show you a tree of method calls, with timing information. I have found this useful in the past to investigate a performance problem. But it might not help if the problem only occurs at 165 queries per second (is that true?). cheers -- Rick On

Re: Upgrade SOLR version - facets perfomance regression

The reason must to be found in the default list of components : protected List getDefaultComponents() { ArrayList names = new ArrayList<>(8); names.add( QueryComponent.COMPONENT_NAME ); names.add( FacetComponent.COMPONENT_NAME ); *names.add( FacetModule.COMPONENT_NAME );*

Re: Upgrade SOLR version - facets perfomance regression

I noticed if I don't write list of components in request handler it works fine, but if I add something like query facet Facets don't work... How can you explian it? -- View this message in context:

Re: How to combine third party search data as top results ?

On 31/01/2017 19:04, Shamik Bandopadhyay wrote: Hi, I'm trying to integrate results from a third party source with our existing search. The idea is to include the top 5 results from this source as the top result of our search.Though the external data is indexed in our system, the use case