Solr 5.2: Same Document in Multiple Shard

2015-08-28 Thread Maulin Rathod
We have recently upgraded solr from 4.8 to 5.2. We have 2 shard and 2 replica in solr cloud. It shows correctly in SolrCloud via Solr Admin Panel. We found that sometimes same document is available in both the shards. We confirmed via querying individual shard (from solr admin by passing shards

Indexing Fixed length file

2015-08-28 Thread timmsn
Hello, i use Solr 5.2.1 and the bin/post tool. I try to set the index of some files they have a fixed length and no withespace to seperate the words. How can i Programm a Template or so for my fields? Or can i edit the schema.xml for my Problem? This ist one record from one file, in this file

RE: no default request handler is registered

2015-08-28 Thread Scott Hollenbeck
-Original Message- From: Shawn Heisey [mailto:apa...@elyograg.org] Sent: Thursday, August 27, 2015 3:51 PM To: solr-user@lucene.apache.org Subject: Re: no default request handler is registered On 8/27/2015 1:10 PM, Scott Hollenbeck wrote: I'm doing some experimenting with Solr

Re: What is the correct path for mysql jdbc connector on Solr?

2015-08-28 Thread Shawn Heisey
On 8/28/2015 6:18 AM, Merlin Morgenstern wrote: Full Import failed:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: Could not load driver: com.mysql.jdbc.Driver Processing Document # 1 How many directories do I have to go

What is the correct path for mysql jdbc connector on Solr?

2015-08-28 Thread Merlin Morgenstern
I have solrcloud installation running on 3 machines where I would like to import data from mysql. Unfortunatelly the import failes due to the missing jdbc connector. My guess is, that I am having trouble with the right directory. solrconfig.xml: lib dir=${solr.install.dir:../../..}/dist/

solrcloud and core swapping

2015-08-28 Thread Bill Au
Is core swapping supported in SolrCloud? If I have a 5 nodes SolrCloud cluster and I do a core swap on the leader, will the core be swapped on the other 4 nodes as well? Or do I need to do a core swap on each node? Bill

Re: Indexing Fixed length file

2015-08-28 Thread Steve Rowe
Hi Tim, I haven’t heard of people indexing this kind of input with Solr, but the format is quite similar to CSV/TSV files, with the exception that the field separators have fixed positions and are omitted. You could write a short script to insert separators (e.g. commas) at these points (but

Re: solrcloud and core swapping

2015-08-28 Thread Shawn Heisey
On 8/28/2015 8:10 AM, Bill Au wrote: Is core swapping supported in SolrCloud? If I have a 5 nodes SolrCloud cluster and I do a core swap on the leader, will the core be swapped on the other 4 nodes as well? Or do I need to do a core swap on each node? When you're running SolrCloud, swapping

Re: Indexing Fixed length file

2015-08-28 Thread Erick Erickson
Solr doesn't know anything about such a file. The post program expects well-defined structures, see the xml and json formats in example/exampledocs. So you either have to transform the data into the form expected by the bin/post tool or perhaps you can use the CSV import, see:

Re: Solr 5.2: Same Document in Multiple Shard

2015-08-28 Thread Erick Erickson
Have you done anything special in terms of routing or are you using the default compositeId? How are you indexing? Docs are considered identical in Solr based solely on the uniqueKey field. If that's the absolute same (possibly including extra whitespace) then this shouldn't be happening, nobody

Re: solrcloud and core swapping

2015-08-28 Thread Shawn Heisey
On 8/28/2015 8:25 AM, Shawn Heisey wrote: Instead, use collection aliasing. Create collections named something like foo_0 and foo_1, and update the alias foo to point to whichever of them is currently live. Your queries and update requests will never need to know about foo_0 and foo_1 ... only

Re: Indexing Fixed length file

2015-08-28 Thread Alexandre Rafalovitch
If you use DataImportHandler, you can combine LineEntityProcessor with RegexTransformer to split each line into a bunch of fields:

Re: Query timeAllowed and its behavior.

2015-08-28 Thread William Bell
As we reported, we are having issues with timeAllowed on 5.2.1. If we set a timeAllowed=1 and then run the same query with timeAllowed=3 we get the # of rows that was returned on the first query. It appears the results are cached when exceeding the timeAllowed, like the results are correct -

Re: Query timeAllowed and its behavior.

2015-08-28 Thread Shawn Heisey
On 8/28/2015 10:47 PM, William Bell wrote: As we reported, we are having issues with timeAllowed on 5.2.1. If we set a timeAllowed=1 and then run the same query with timeAllowed=3 we get the # of rows that was returned on the first query. It appears the results are cached when exceeding

Re: Indexing Fixed length file

2015-08-28 Thread Erik Hatcher
How about this incantation: $ bin/solr create -c fw $ echo Q36 | awk -v OFS=, '{ print substr($0, 1, 1), substr($0, 2, 2) }' | bin/post -c fw -params fieldnames=id,valheader=false -type text/csv -d $ curl 'http://localhost:8983/solr/fw/select?q=*:*wt=csv' val,_version_,id

Sorting by function

2015-08-28 Thread Philippe Soares
Hi, I'm trying to apply the Sort by function https://wiki.apache.org/solr/FunctionQuery#Sort_By_Function solr capabilities to solve the following use case : I have a country field in my index, with values like 'US', 'FR', 'UK', etc... Then I want our users to be able to define the order of their

Re: Sorting by function

2015-08-28 Thread Chris Hostetter
: I have a country field in my index, with values like 'US', 'FR', 'UK', : etc... : : Then I want our users to be able to define the order of their preferred : countries so that grouped results are sorted according to their preference. ... : Is there any other function that would allow

Re: Sorting by function

2015-08-28 Thread Philippe Soares
Thanks Chris ! I have the country as a single valued field so your solution works perfectly ! On Fri, Aug 28, 2015 at 1:22 PM, Chris Hostetter hossman_luc...@fucit.org wrote: : I have a country field in my index, with values like 'US', 'FR', 'UK', : etc... : : Then I want our users to be

Re: Indexing Fixed length file

2015-08-28 Thread Alexandre Rafalovitch
Erik's version might be better with tabs though to avoid CSV's requirements on escaping comas, quotes, etc. And maybe trim those fields a bit either in awk or in URP inside Solr. But it would definitely work. Regards, Alex. Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:

ping handler very doubtful

2015-08-28 Thread Davis, Daniel (NIH/NLM) [C]
So, I tested that the PingRequestHandler works in the following fashion: cd server/corename/data/index # some work with ls and awk to produce a script, and then it runs dd if=/dev/urandom of=`pwd`/segments_10 bs=160 count=1 dd if=/dev/urandom of=`pwd`/_u.fdt bs=41512

Re: Indexing Fixed length file

2015-08-28 Thread Erik Hatcher
Ah yes, I should have made my example use tabs, though that currently would have required also adding “separator=%09” to the params. I definitely support the use of tabs for what they were intended, delimiting columns of data. +1, thanks for that mention Alex On Aug 28, 2015, at 1:38

Dynamic field rule plugin?

2015-08-28 Thread Hari Iyer
Hi, I am new to Solr and am trying to create dynamic field rules in my Schema. I would like to use file name suffix to indicate other properties besides the data type and multivalued as provided in the default schema. It appears that specifying this via a pattern leads to duplication as there

RE: Data Import Handler Stays Idle

2015-08-28 Thread Allison, Timothy B.
Only a month late to respond, and the response likely won't help. I agree with Shawn that Tika can be a memory hog. I try to leave 1GB per thread, but your mileage will vary dramatically depending on your docs. I'd expect that you'd get an OOM, though, somewhere... There have been rare bugs

RE: Data Import Handler Stays Idle

2015-08-28 Thread Allison, Timothy B.
There are some zip files inside the directory and have been addressed to in the database. I'm thinking those are the one's it's jumping right over. With SOLR-7189, which should have kicked in for 5.1, Tika shouldn't skip over Zip files, it should process all the contents of those zips and

PingRequestHandler and file corruption

2015-08-28 Thread Davis, Daniel (NIH/NLM) [C]
This is a resend to correct my awful subject. From: Davis, Daniel (NIH/NLM) [C] Sent: Friday, August 28, 2015 2:15 PM To: solr-user@lucene.apache.org Subject: ping handler very doubtful So, I tested that the PingRequestHandler works in the following fashion: cd