Re: Benchmarking Solr

Hi, why don't you use JMeter? It would give you greater control over the tests you wish to make. It has many different samplers that will let you run different scenarios using your existing set of queries. ab is great when you want to evaluate the performance of your server under heavy

Collapse problem

Hi All, I'd like to know if anyone else is experiencing the same problem we are facing basically, we are running query with field collapsing (Solr 1.4 with patch 236). The responses tells us that there are about 2700 documents matching our query. However, I can not get passed the 431th

Re: Reg: Indexing Date Fields

I guess you can simply use a range query such as: fq=createdDate:[ date1 TO date2 ] On 2010-04-15, at 7:30 AM, Venkata Sai Krishna Vepakomma wrote: Hi, 1) How do I query for Data between 2 date ranges. I have specified the following field definition in Schema.xml. field

JTeam Spatial Plugin

Hi All, I am using JTeam's Spatial Plugin RC3 to perform spatial searches on my index and it works great. However, I can't seem to get it to return the computed distances. My query component is run before the geoDistanceComponent and the distanceField is set to distance Fields for lat/long

Re: JTeam Spatial Plugin

fields. On Thu, Apr 29, 2010 at 9:26 AM, Jean-Sebastien Vachon js.vac...@videotron.ca wrote: Hi All, I am using JTeam's Spatial Plugin RC3 to perform spatial searches on my index and it works great. However, I can't seem to get it to return the computed distances. My query component

Re: jmx issue with solr

Hi, Try adding these options... -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false On 2010-05-19, at 3:44 AM, Na_D wrote: Hi, I am trying to start solr with the following command : java -Dsolr.solr.home=./example-DIH/solr/

Re: Faceted search not working?

Is the FacetComponent loaded at all? requestHandler name=standard class=solr.SearchHandler default=true arr name=components strquery/str strfacet/str /arr /requestHandler On 2010-05-25, at 3:32 AM, Sascha Szott wrote: Hi Birger, Birger Lie wrote: I don't think the bolean

question about the fieldCollapseCache

Hi All, I've been running some tests using 6 shards each one containing about 1 millions documents. Each shard is running in its own virtual machine with 7 GB of ram (5GB allocated to the JVM). After about 1100 unique queries the shards start to struggle and run out of memory. I've reduced

Re: question about the fieldCollapseCache

-Sebastien Vachon js.vac...@videotron.ca wrote: Hi All, I've been running some tests using 6 shards each one containing about 1 millions documents. Each shard is running in its own virtual machine with 7 GB of ram (5GB allocated to the JVM). After about 1100 unique queries the shards start

Re: Diagnosing solr timeout

Have you looked at the garbage collector statistics? I've experienced this kind of issues in the past and I was getting huge spikes when the GC was doing its job. On 2010-06-09, at 10:52 AM, Paul wrote: Hi all, In my app, it seems like solr has become slower over time. The index has grown

Re: Diagnosing solr timeout

I use the following article as a reference when dealing with GC related issues http://www.petefreitag.com/articles/gctuning/ I suggest you activate the verbose option and send GC stats to a file. I don't remember exactly what was the option but you should find the information easily Good luck

Re: question about the fieldCollapseCache

solr 1.4.1 from the site. but it shows an empty directory. where did u get solr 1.4.1 from? Regards, Raakhi On Tue, Jun 8, 2010 at 10:35 PM, Jean-Sebastien Vachon js.vac...@videotron.ca wrote: Hi All, I've been running some tests using 6 shards each one containing about 1 millions

bug or feature???

Hi, Can someone tell me why the two following queries do not return the same results? Is that a bug or a feature? http://localhost:8983/jobs/select?fq=title:(NOT janitor)fq=description:(NOT janitor)q=*:* http://localhost:8983/jobs/select?q=title:(NOT janitor) AND description:(NOT janitor)

Re: Need help with field collapsing and out of memory error

can you tell us what are your current settings regarding the fieldCollapseCache? I had similar issues with field collapsing and I found out that this cache was responsible for most of the OOM exceptions. Reduce or even remove this cache from your configuration and it should help. On

Index size

Hi All, I'm currently looking on integrating Solr and I'd like to have some hints on the size of the index (number of documents) I could possibly host on a server running a Double-Quad server (16 cores) with 48Gb of RAM running Linux. Basically, I need to determine how many of these servers

Re: Index size

you compress it, how and how much you analyze the data, etc. Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Hadoop ecosystem search :: http://search-hadoop.com/ - Original Message From: Jean-Sebastien Vachon js.vac...@videotron.ca To: solr-user

Profiling Solr

Hi, I'm trying to identify the bottleneck to get acceptable performance of a single shard containing 4.7 millions of documents using my own machine (Mac Pro - Quad Core with 8Gb of RAM with 4Gb allocated to the JVM). I tried using YourKit but I don't get anything about Solr classes. I'm new

Multi valued fields

Hi All, I'd like to know if it is possible to do the following on a multi-value field: Given the following data: document A: field1 = [ A B C D] document B: field 1 = [A B] document C: field 1 = [A] Can I build a query such as : -field: A which will return all documents that

Spatial search in Solr 1.5

Hi All, I'm trying to figure out how to perform spatial searches using Solr 1.5 (from the trunk). Is the support for spatial search built-in? because none of the patches I tried could be applied to the source tree. If this is the case, can someone one tell me how to configure it? I find the

Re: Recommended OS

On 2010-03-18, at 1:03 PM, K Wong wrote: http://wiki.apache.org/solr/FAQ#What_are_the_Requirements_for_running_a_Solr_server.3F I have Solr running on CentOS 5.4. It runs fine on the OpenJDK 1.6.0 and Tomcat 5. If I were to do it again, I'd probably just stick with Jetty. Would you mind

Spatial queries

Hi All, I am using the package from JTeam to perform spatial searches on my index. I'd like to know if it is possible to build a query that uses multiple clauses. Here is an example: q={!spatial lat=123 long=456 radius=10} OR {!spatial lat=111 long=222 radius=20}title:java Basically that

Excluding field from the results

Hi, Is there an easy way to prevent a field from being returned in the response? we can use fl=field1, field2, field3, ... but then our software has an option that must trigger the presence or not of a field in the response. So what I'd like to do is tell Solr to return all fields except one.

RE: SOlr 3.5 and sharding

-- across multiple shards, the count will not be what you expect. You'll need to ensure that each document exists only on a single shard. -Michael -Original Message- From: Jean-Sebastien Vachon [mailto:jean-sebastien.vac...@wantedanalytics.com] Sent: Monday, January 14, 2013 9:59 AM To: solr

RE: SOlr 3.5 and sharding

the polling interval to consider. Best Erick On Mon, Jan 14, 2013 at 9:58 AM, Jean-Sebastien Vachon jean-sebastien.vac...@wantedanalytics.com wrote: Hi, I`m setting up a small Sorl setup consisting of 1 master node and 4 shards. For now, all four shards contains the exact same data. When

RE: SOlr 3.5 and sharding

be searching, BTW) is different than the slaves, there's the polling interval to consider. Best Erick On Mon, Jan 14, 2013 at 9:58 AM, Jean-Sebastien Vachon jean-sebastien.vac...@wantedanalytics.com wrote: Hi, I`m setting up a small Sorl setup consisting of 1 master node and 4

RE: SOlr 3.5 and sharding

for of these boxes? Is that correct? Upayavira On Tue, Jan 15, 2013, at 02:11 PM, Jean-Sebastien Vachon wrote: Hi Erick, Thanks for your comments but I am migrating an existing index (single instance) to a sharded setup and currently I have no access to the code involved in the indexation

Re: RAM increase

You will also need to switch to a 64 bits JVM You might have to add the `-d64` flag as well as the `-Xms` and `-Xmx` - Original Message - From: Gora Mohanty g...@mimirtech.com To: solr-user@lucene.apache.org Sent: Thursday, October 21, 2010 2:34 AM Subject: Re: RAM increase On Thu,

Re: Problem escaping question marks

Have you tried encoding it with %3F? firstname:*%3F* On 2010-11-04, at 1:44 AM, Stephen Powis wrote: I'm having difficulty properly escaping ? in my search queries. It seems as tho it matches any character. Some info, a simplified schema and query to explain the issue I'm having. I'm

problem with wildcard

Hi All, I'm having some trouble with a query using some wildcard and I was wondering if anyone could tell me why these two similar queries do not return the same number of results. Basically, the query I'm making should return all docs whose title starts (or contain) the string lowe'. I

Re: problem with wildcard

On 2010-11-11, at 3:45 PM, Ahmet Arslan wrote: I'm having some trouble with a query using some wildcard and I was wondering if anyone could tell me why these two similar queries do not return the same number of results. Basically, the query I'm making should return all docs whose title

Re: Looking for help with Solr implementation

Hi, If you're still looking for someone, I might be interested in getting more information about your project. From you initial message that does not seem to be a lot of work so I might be willing to give you some time. I've been working with Solr for the last 7 months on my full-time job

Re: Looking for help with Solr implementation

Sorry all, I obviously meant to send this to the original poster - Original Message - From: Jean-Sebastien Vachon js.vac...@videotron.ca To: solr-user@lucene.apache.org Sent: Friday, November 12, 2010 10:09 PM Subject: Re: Looking for help with Solr implementation Hi, If you're

Re: Looking for help with Solr implementation

their is mutual interest I can share with you a document that he created that explains how things have been set-up which should help you get started. Please let me know what you think. Regards, Abe From: Jean-Sebastien Vachon js.vac...@videotron.ca

Facet.query and collapsing

Hi All, I'm in a situation where I need to perform a facet on a query with field collapsing. Let's say the main query is something like this title:applefq={!tag=sources}source_id:(33 OR 44)facet=onfacet.field={!ex=sources}source_idfacet.query=source_id:(33 OR

Re: spatial query parinsg error: org.apache.lucene.queryParser.ParseException

Try this... http://localhost:8080/solr/select?wt=jsonindent=trueq={!spatial%20lat=37.326375%20lng=-121.892639%20radius=3%20unit=km%20threadCount=3}title:Art%20Loft - Original Message - From: Dennis Gearon gear...@sbcglobal.net To: solr-user@lucene.apache.org Sent: Wednesday, December

Re: spatial query parinsg error: org.apache.lucene.queryParser.ParseException

from your own mistakes. It is usually a better idea to learn from others’ mistakes, so you do not have to make them yourself. from 'http://blogs.techrepublic.com.com/security/?p=4501tag=nl.e036' EARTH has a Right To Life, otherwise we all die. - Original Message From: Jean-Sebastien

RE: Exact match on a field with stemming

I'm curious to know why Solr is not respecting the phrase. If it consider manager as a phrase... shouldn't it return only document containing that phrase? -Original Message- From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] Sent: April-11-11 3:42 PM To:

RE: Exact match on a field with stemming

Thanks for the clarification. This make sense. -Original Message- From: Jonathan Rochkind [mailto:rochk...@jhu.edu] Sent: April-11-11 7:54 PM To: solr-user@lucene.apache.org Subject: FW: Exact match on a field with stemming I'm curious to know why Solr is not respecting the phrase.

SOLR-2209

Hi All, I am having some problems with the presence of unnecessary parenthesis in my query. A query such as: title:software AND (title:engineer) will return no results. Remove the parenthesis fix the issue but then since my user can enter the parenthesis by himself I need to

RE: SOLR-2209

as I can tell. The bug you're referencing is a peculiarity of the not (-) operator I think. Best Erick On Thu, May 19, 2011 at 7:25 AM, Jean-Sebastien Vachon jean-sebastien.vac...@wantedtech.com wrote: Hi All, I am having some problems with the presence of unnecessary  parenthesis in my query

FieldCache

Hi All, Since there is no way of controlling the size of Lucene's internal FieldCache, how can we make sure that we are making good use of it? One of my shard has close to 1.5M documents and the fieldCache only contains about 10 elements. Is there anything we can do to control this?

RE: FieldCache

10 unique terms on 1.5M documents each with 50+ fields? I don't think so ;) What I mean is controlling its size like the other caches. There are currently no options in solrconfig.xml to control this cache. Is Solr/Lucene managing this all by itself? It could be that my understanding of the

Problem with document routing with Solr 4.2.1

Hi All, I just started indexing data in my brand new Solr Cloud running on 4.2.1. Since I am a big user of the grouping feature, I need to route my documents on the proper shard. Following the instruction found here: http://docs.lucidworks.com/display/solr/Shards+and+Indexing+Data+in+SolrCloud

RE: Problem with document routing with Solr 4.2.1

with Solr 4.2.1 That's strange. The default value of rows param is 10 so you should be getting 10 results back unless your StandardRequestHandler config in solrconfig has set rows to 0 or if none of your fields are stored. On Thu, May 23, 2013 at 7:40 PM, Jean-Sebastien Vachon jean

RE: Problem with document routing with Solr 4.2.1

If that can help.. adding distrib=false or shard.keys= is giving back results. -Original Message- From: Jean-Sebastien Vachon [mailto:jean-sebastien.vac...@wantedanalytics.com] Sent: May-23-13 10:39 AM To: solr-user@lucene.apache.org Subject: RE: Problem with document routing

RE: Problem with document routing with Solr 4.2.1

I must add the shard.keys= does not return anything on two on my nodes. But that is to be expected since I'm using a replication factor of 3 on a cloud of 5 servers -Original Message- From: Jean-Sebastien Vachon [mailto:jean-sebastien.vac...@wantedanalytics.com] Sent: May-23-13 11

RE: Problem with document routing with Solr 4.2.1

hashing and that the id field was tokenized, Solr could not find back documents when collecting responses from each shards. We changed back the id field to the 'string' type and it is now working -Original Message- From: Jean-Sebastien Vachon [mailto:jean-sebastien.vac...@wantedanalytics.com

queryResultCache showing all zeros

Hi, We just configured a new Solr cloud (5 nodes) running Solr 4.3, ran about 200 000 queries taken from our production environment and measured the performance of the cloud over a collection of 14M documents with the default Solr settings. We are now trying to tune the different caches and

RE: queryResultCache showing all zeros

Looks like the problem might not be related to Solr but to a proprietary system we have on top of it. I made some queries with facets and the cache was updated. We are looking into this... I should not have assumed that the problem was coming from Solr ;) I'll let you know if there is anything

RE: queryResultCache showing all zeros

and grouping. If I remove the grouping then the cache is used. Is this a normal behavior or a bug? Thanks From: Jean-Sebastien Vachon Sent: Wednesday, July 31, 2013 2:38 PM To: solr-user@lucene.apache.org Subject: RE: queryResultCache showing all zeros Looks

RE: queryResultCache showing all zeros

Also we do not have any useFilterForSortedQuery in our config. So we are relying on the default which I guess is false. From: Jean-Sebastien Vachon Sent: Wednesday, July 31, 2013 3:44 PM To: solr-user@lucene.apache.org Subject: RE: queryResultCache

Huge discrepancy between QTime and ElapsedTime

Hi All, I am running some benchmarks to tune our Solr 4.3 cloud and noticed that while the reported QTime is quite satisfactory (100 ms or so), the elapsed time is quite large (around 5 seconds). The collection contains 12.8M documents and the index size on disk is about 35 GB.. I have only

RE: Huge discrepancy between QTime and ElapsedTime

-Sebastien Vachon wrote: I am running some benchmarks to tune our Solr 4.3 cloud and noticed that while the reported QTime is quite satisfactory (100 ms or so), the elapsed time is quite large (around 5 seconds). The collection contains 12.8M documents and the index size on disk is about 35

Flushing cache without restarting everything?

Hi All, Is there a way to flush the cache of all nodes in a Solr Cloud (by reloading all the cores, through the collection API, ...) without having to restart all nodes? Thanks

RE: Flushing cache without restarting everything?

are you trying to acheive with this? --wunder On Aug 20, 2013, at 11:04 AM, Jean-Sebastien Vachon wrote: Hi All, Is there a way to flush the cache of all nodes in a Solr Cloud (by reloading all the cores, through the collection API, ...) without having to restart all nodes? Thanks

RE: Flushing cache without restarting everything?

(=production) load? Dmitry On Tue, Aug 20, 2013 at 9:39 PM, Jean-Sebastien Vachon jean- sebastien.vac...@wantedanalytics.com wrote: I just want to run benchmarks and want to have the same starting conditions. -Original Message- From: Walter Underwood [mailto:wun

RE: Flushing cache without restarting everything?

:04 +0200, Jean-Sebastien Vachon wrote: Is there a way to flush the cache of all nodes in a Solr Cloud (by reloading all the cores, through the collection API, ...) without having to restart all nodes? As MMapDirectory shares data with the OS disk cache, flushing of Solr-related caches

RE: Regarding improving performance of the solr

Have you checked the hit ratio of the different caches? Try to tune them to get rid of all evictions if possible. Tuning the size of the caches and warming you searcher can give you a pretty good improvement. You might want to check your analysis chain as well to see if you`re not doing

RE: Recherche avec et sans espaces

Bonjour Antoine, Je ne vois que 2 solutions à ton problème. 1) utilisation de synonymes mais tu seras limités au cas connus d'avance seulement alors c'est une solution qui ne scale pas à long terme. 2) sinon tu dois envisager d'avoir un deuxième champ (probablement en CopyField) qui

Update single field through SolrJ

Hi All, I`d like to know how (it is possible) to update a field`s value using SolrJ. I looked at the API and could not figure it out so for now I'm using the UpdateHandler by sending it a JSON formatted document illustrating the required changes. Is there a way to do the same through SolrJ?

RE: Update single field through SolrJ

-Original Message- From: Shawn Heisey [mailto:s...@elyograg.org] Sent: March-28-14 3:21 PM To: solr-user@lucene.apache.org Subject: Re: Update single field through SolrJ On 3/28/2014 1:02 PM, Jean-Sebastien Vachon wrote: I`d like to know how (it is possible) to update a field`s

Were changes made to facetting on multivalued fields recently?

Hi All, We just discovered that the response from Solr (4.7.1) when faceting on one of our multi-valued fields has changed considerably. In the past (4.6.1 and prior versions as well) we used to have something like this: (there are 7 possible values for this attribute) lst name=facet_counts

RE: Were changes made to facetting on multivalued fields recently?

On Wed, Apr 9, 2014 at 11:04 AM, Jean-Sebastien Vachon jean- sebastien.vac...@wantedanalytics.com wrote: Hi All, We just discovered that the response from Solr (4.7.1) when faceting on one of our multi-valued fields has changed considerably. In the past (4.6.1 and prior versions as well

RE: Were changes made to facetting on multivalued fields recently?

Here are the field definitions for both our old and new index... as you can see that are identical. We've been using this chain and field type starting with Solr 1.4 and never had any problem. As for the documents, both indexes are using the same data source. They could be slightly out of sync

RE: Were changes made to facetting on multivalued fields recently?

, Jean-Sebastien Vachon wrote: Here are the field definitions for both our old and new index... as you can see that are identical. We've been using this chain and field type starting with Solr 1.4 and never had any problem. As for the documents, both indexes are using the same data source

RE: Were changes made to facetting on multivalued fields recently?

the behavior has changed though, but my supposition is that it has nothing to do with Solr, and something about your SQL statement is different. Best, Erick On Thu, Apr 10, 2014 at 9:33 AM, Jean-Sebastien Vachon jean- sebastien.vac...@wantedanalytics.com wrote: The SQL query contains

Transformation on a numeric field

Hi All, I am looking for a way to index a numeric field and its value divided by 1 000 into another numeric field. I thought about using a CopyField with a PatternReplaceFilterFactory to keep only the first few digits (cutting the last three). Solr complains that I can not have an analysis

RE: Transformation on a numeric field

that takes a source and destination field name and then does a specified operation on it, like add an offset or multiple by a scale factor. -- Jack Krupansky -Original Message- From: Jean-Sebastien Vachon Sent: Tuesday, April 15, 2014 3:57 PM To: 'solr-user@lucene.apache.org' Subject

Question regarding the lastest version of HeliosSearch

Hi All, I spent some time today playing around with subfacets and facets functions now available in helios search 0.05 and I have some concerns... They look very promising . I indexed 10 000 documents and built some queries to look at each feature and found some weird behaviour that I could

RE: Question regarding the lastest version of HeliosSearch

:44 PM, Jean-Sebastien Vachon jean- sebastien.vac...@wantedanalytics.com wrote: I spent some time today playing around with subfacets and facets functions now available in helios search 0.05 and I have some concerns... They look very promising . Thanks, glad for the feedback

Strange behaviour when tuning the caches

Hi All, We have a 5 nodes setup running Solr 4.8.1 and we are trying to get the most out of it by tuning Solr caches. Following is the output of the script version.sh provided with Tomcat Server version: Apache Tomcat/7.0.39 Server built: Mar 22 2013 12:37:24 Server number: 7.0.39.0 OS Name:

RE: Strange behaviour when tuning the caches

tuning the caches On 6/2/2014 8:24 AM, Jean-Sebastien Vachon wrote: We have yet to determine where the exact breaking point is. The two patterns we are seeing are: - less cache (around 20-30% hit/ratio), poor performance but overall good stability When caches are too small

RE: Strange behaviour when tuning the caches

-- Performance Monitoring * Log Analytics * Search Analytics Solr Elasticsearch Support * http://sematext.com/ On Mon, Jun 2, 2014 at 11:09 AM, Jean-Sebastien Vachon jean- sebastien.vac...@wantedanalytics.com wrote: Thanks for your quick response. Our JVM is configured with a heap

RE: Strange behaviour when tuning the caches

/CollapsingQParserPlugin ? May help with the field collapsing queries. Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr Elasticsearch Support * http://sematext.com/ On Tue, Jun 3, 2014 at 8:41 AM, Jean-Sebastien Vachon jean- sebastien.vac...@wantedanalytics.com wrote: Hi

RE: Strange Behavior with Solr in Tomcat.

I would try a thread dump and check the output to see what`s going on. You could also strace the process if you`re running on Unix or changed the log level in Solr to get more information logged -Original Message- From: S.L [mailto:simpleliving...@gmail.com] Sent: June-06-14 2:33 PM

RE: ANN: Solr Next

Hi Yonik, Very impressive results. Looking forward to use this on our systems. Any idea what`s the plan for this feature? Will it make its way into Solr 4.9? or do we have to switch to HeliosSearch to be able to use it? Thanks -Original Message- From: Yonik Seeley

Re: Delete By query on a multi-value field

. The *remove *clause (atomic update) has been added to 4.9.0 release. Haven't checked it though. Thanks, Lokesh On Tue, Feb 3, 2015 at 7:26 AM, Jean-Sebastien Vachon jean-sebastien.vac...@wantedanalytics.com wrote: Hi All, Is there a way to delete a value from a Multi-value field without

Delete By query on a multi-value field

Hi All, Is there a way to delete a value from a Multi-value field without reindexing anything? Lets say I have three documents A,B and C with field XYZ set to 1,2,3, 2,3,4 and 1. I'd like to remove anything that has the value '1' in the field XYZ. That is I want to remove the value '1' from

Re: indexing db records via SolrJ

Do you have any references to such integrations (Solr + Storm)? Thanks From: mike st. john mstj...@gmail.com Sent: Monday, March 16, 2015 2:39 PM To: solr-user@lucene.apache.org Subject: Re: indexing db records via SolrJ Take a look at some of the

Re: JSON Facet Analytics API in Solr 5.1

I prefer the second way. I find it more readable and shorter. Thanks for making Solr even better ;) From: Yonik Seeley ysee...@gmail.com Sent: Friday, April 17, 2015 12:20 PM To: solr-user@lucene.apache.org Subject: Re: JSON Facet Analytics API in Solr

Re: are there any SolrCloud supervisors?

I would be interested in seeing it in action. Do you have any documentation available on what it does and how? Thanks From: r b Sent: Friday, October 2, 2015 3:09 PM To: solr-user@lucene.apache.org Subject: are there any SolrCloud

Re: Fully automated replica creation in AWS

Not sure if this will meet all your needs but you can probably do most of the work using AWS lambda. I haven't used it personally but it is supposed to launch custom code following some events. I guess you could create a small Java class to do the required work following the birth of a new