Re: Solr query help

You can add a ~3 to the query to allow the order to be reversed, but you will get extra hits. Maybe it is a ~4, i can never remember on phrases and reversals. I usually just try it. Alternatively, you can create a custom query field for what you need from dates. For example, if you want to

Re: Get results in multiple orders (multiple boosts)

Luca Walter has got the best word on this, you should use SQL for sorting (maybe mySQL or Postgres). If you also need searching, you can create a Solr index by ingesting from the SQL database. The Solr index would be just used for searching. Cheers -- Rick -- Sorry for being brief. Alternate

Re: Solr 6.6.0 - High CPU during indexing

Thank you for the detailed response Shawn! I've read it several times. Yes, that particular machine has 12 cores that are hyper-threaded. Does Solr do something special when not running in HDFS to allocate memory that would result in VIRT showing memory required for index data size? In my

Re: Solr 6.6.0 - High CPU during indexing

On 8/18/2017 1:05 PM, Joe Obernberger wrote: > Thank you Shawn.  Please see: > http://www.lovehorsepower.com/Vesta > for screen shots of top > (http://www.lovehorsepower.com/Vesta/VestaSolr6.6.0_top.jpg) and > several screen shots over various times of jvisualvm. > > There is also the GC log and

Re: Solr 6.6.0 - High CPU during indexing

Ah! Yes - that makes much more sense: CPU: http://www.lovehorsepower.com/Vesta/VestaSolr6.6.0_CPU.jpg Mem: http://www.lovehorsepower.com/Vesta/VestaSolr6.6.0_Mem.jpg -Joe On 8/18/2017 3:35 PM, Michael Braun wrote: When I recommended JVisualVM, specifically the "Sampling" portion of the app

Re: Solr 6.6.0 - High CPU during indexing

Hi Walter - I see what you are saying, but the machine is not actively swapping (that would be the concern - right?) It's the CPU usage that I'm trying to figure out. Htop reports that there is about 20G of disk cache in use, and about 76G of RAM in use by programs. VIRT memory is what was

Re: Solr 6.6.0 - High CPU during indexing

I see a server with 100Gb of memory and processes (java and jsvc) using 203Gb of virtual memory. Hmm. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Aug 18, 2017, at 12:05 PM, Joe Obernberger > wrote: > > Thank you

Re: Solr 6.6.0 - High CPU during indexing

Thank you Shawn. Please see: http://www.lovehorsepower.com/Vesta for screen shots of top (http://www.lovehorsepower.com/Vesta/VestaSolr6.6.0_top.jpg) and several screen shots over various times of jvisualvm. There is also the GC log and the regular solr.log for one server (named Vesta).

Re: Solr 6.6.0 - High CPU during indexing

On 8/18/2017 10:37 AM, Joe Obernberger wrote: > Indexing about 15 million documents per day across 100 shards on 45 > servers.  Up until about 350 million documents, each of the solr > instances was taking up about 1 core (100% CPU).  Recently, they all > jumped to 700%.  Is this normal?  Anything

Re: Solr query help

What field types are you using for your dates? Have a look at: https://cwiki.apache.org/confluence/display/solr/Working+with+Dates On Thu, Aug 17, 2017 at 10:08 AM, Nawab Zada Asad Iqbal wrote: > Hi Krishna > > I haven't used date range queries myself. But if Solr only

Re: Request Highlighting only for the final set of rows

Actually, part of me is thinking that there are valid use cases for having fl and hl.fl with different values. e.g, receive name etc. in “clean” form in fl field and receive both name and address in html formatted form (by specifying in hl.fl) On Fri, Aug 18, 2017 at 10:57 AM, Nawab Zada Asad

Re: Tlogs not being deleted/truncated

I have an update on this. While I was on vacation, there were a number of alerts. Our autoCommit settings were (and are) the following: ${solr.autoCommit.maxTime:60} false The startup script was NOT setting solr.autoCommit.maxTime. It seemed that autoCommits were

Re: Request Highlighting only for the final set of rows

Actually, i realize that it is an incorrect use on my part to pass only id+score in fl and specify more fields in the hl.fl fields. This was somehow supported in older versions but the new behavior is actually a performance improvement for the scenario when user is asking for only ids. Nawab On

Re: Solr 6.6.0 - High CPU during indexing

I was able to attach to one server by changing the startup and adding: |-Dcom.sun.management.jmxremote \ -Dcom.sun.management.jmxremote.local.only=false \ -Dcom.sun.management.jmxremote.ssl=false \ -Dcom.sun.management.jmxremote.authenticate=false \ -Dcom.sun.management.jmxremote.port=18983

Re: Solr 6.6.0 - High CPU during indexing

Thank you Michael. Oddly when I start jstatd on one of the servers, I see all the JVM processes in jvisualvm except the solr one! Any idea why? On 8/18/2017 12:43 PM, Michael Braun wrote: Have you attached JVisualVM or a similar application to the process to sample where the time is being

Re: Solr 6.6.0 - High CPU during indexing

Have you attached JVisualVM or a similar application to the process to sample where the time is being spent? It can be very helpful for debugging this sort of problem. On Fri, Aug 18, 2017 at 12:37 PM, Joe Obernberger < joseph.obernber...@gmail.com> wrote: > Indexing about 15 million documents

Solr 6.6.0 - High CPU during indexing

Indexing about 15 million documents per day across 100 shards on 45 servers. Up until about 350 million documents, each of the solr instances was taking up about 1 core (100% CPU). Recently, they all jumped to 700%. Is this normal? Anything that I can check for? I don't see anything

Re: Request Highlighting only for the final set of rows

Thanks Erick for the pointing to better option. I will explore that. After your email, I found that if i have specified 'fl=*' in the query then it is doing the right thing (a 2 pass process). However, my queries had 'fl=id+score' (or sometimes fl=id=score), in both of these cases I found that the

Re: Request Highlighting only for the final set of rows

I don't think you're reading it correctly. First of all, if you're going to do be doing deep paging you should be using cusorMark, see: https://cwiki.apache.org/confluence/display/solr/Pagination+of+Results. Second, it's a two-pass process if you don't use cursormark. The first pass gets the

Re: Match with AND across multiple fields

Solr does not implement pure boolean logic, see: https://lucidworks.com/2011/12/28/why-not-and-or-and-not/ As for your particular query, parenthesize, something like: name AND (dimension1 or dimension1x2 or dimenstion 1x2x3) Best, Erick On Fri, Aug 18, 2017 at 2:12 AM, jesseqper

Re: Get results in multiple orders (multiple boosts)

Why do you want to do this in Solr? This would be pretty easy in SQL. If you want to sort, use a relational database. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Aug 18, 2017, at 2:52 AM, Luca Dall'Osto > wrote:

Re: response time degradation with matchall queries / changin from SOLR 4.10 -> 6.x

Hi Erik, thanks for your reply. I made some deeper investigations to tackle the reason for the behavior but wasn't successful so far Answer to your questions: - yes I completely re-indexed the data - yes I'm running a collection of around 5.000 queries coming from our productive logs Now my

SplitShard Replica Placement

Hi all, I've run into an issue with solr 6.3.0 where the splitshard command placed both replicas of the new smaller shard on the same node, and I was curious as to whether the behavior should be expected or not. Without having dug into the source code, this is what I've observed splitshard doing

Re: response time degradation with matchall queries / changin from SOLR 4.10 -> 6.x

Hi Erik, thanks for your reply. I made some deeper investigations to tackle the reason for the behavior but wasn't successful so far Answer to your questions: - yes I completely re-indexed the data - yes I'm running a collection of around 5.000 queries coming from our productive logs Now my

Re: Solr Logs to ELK / AWS Firestream

Hey On Do, 2017-08-17 at 10:15 -0600, John Bickerstaff wrote: > I'm trying to get Solr logs into AWS Firestream. > > Not having a lot of luck. > > Does anyone out there have any experience getting Solr logs into an ELK > stack?  Or, better yet, getting Solr Logs into AWS Firestream? > > We

Re: Get results in multiple orders (multiple boosts)

Hello Tom, thanks for you reply. As say in last email, I made the custom function in JS.I posted in pastebin right now: https://pastebin.com/faXNi0fR What I have to do is create the same function in Solr... I will take a look to your link and try to create the custom function. Thanks Luca

Match with AND across multiple fields

In my index I have products that have multiple dimensions. I want the user to be able to search with /name/ + and up to 3 /dimensions/. So a query can occur like: /ProductX 10x20/, or: /ProductX 10x20x30/. Now I get to many results back, because matches are like: /name/ AND /dimension/ OR

Re: Get results in multiple orders (multiple boosts)

On Fri, Aug 18, 2017 at 8:21 AM, Luca Dall'Osto wrote: > > Yes, of course, and excuse me for the misunderstanding. > > > In my scenario I have to display a list with hundreds of documents. > An user can show this documents in a particular order, this order is

Solr cloud replica nodes missing some documents

Hello guys, I am having 5 nodes solr cloud setup with single shard. The solr version is 5.2.1. server1 (http://146.XXX.com:4001/solr/contracts_shard1_replica4)is the leader. A document with id '43e14a86cbdd422880cac22d9a15d3c0' was not replicated 3 nodes. Log shows that the

Re: Get results in multiple orders (multiple boosts)

Yes, of course, and excuse me for the misunderstanding. In my scenario I have to display a list with hundreds of documents. An user can show this documents in a particular order, this order is decided by user in a settings view. Order levels are for example: 1) Order by category, as most