Re: [CAUTION] Converting graph query to stream graph query

2019-10-15 Thread Natarajan, Rajeswari
I need to gather all the children of docid 1 . Root item has parent as null. (Sample data below) Tried as below nodes(graphtest, walk="1->parent", gather="docid", scatter="branches, leaves") Response : { "result-set": { "docs": [ { "node": "1",

Re: Solr-Cloud, join and collection collocation

2019-10-15 Thread Erick Erickson
You can certainly replicate the joined collection to every shard. It must fit in one shard and a replica of that shard must be co-located with every replica of the “to” collection. Have you looked at streaming and “streaming expressions"? It does not have the same problem, although it does

Solr-Cloud, join and collection collocation

2019-10-15 Thread Nicolas Paris
Hi I have several large collections that cannot fit in a standalone solr instance. They are split over multiple shards in solr-cloud mode. Those collections are supposed to be joined to an other collection to retrieve subset. Because I am using distributed collections, I am not able to use the

Document Update performances Improvement

2019-10-15 Thread Nicolas Paris
Hi I am looking for a way to faster the update of documents. In my context, the update replaces one of the many existing indexed fields, and keep the others as is. Right now, I am building the whole document, and replacing the existing one by id. I am wondering if **atomic update feature**

Solr JVM performance challenge with Updates

2019-10-15 Thread Ganesh Sethuraman
Hi Solr Users, We are using Solr 7.2.1 with 2 nodes (245GB RAM each) and 3 node ZK cluster in production. We are using Java 8 with default GC settings (with NewRatio=3) with 15GB heap, changed to 16 GB after the performance issue mentioned below. We have about 90 collections in this (~8 shards

Re: Position search

2019-10-15 Thread Tim Casey
If this is about a normalized query, I would put the normalization text into a specific field. The reason for this is you may want to search the overall text during any form of expansion phase of searching for data. That is, maybe you want to know the context of up to the 120th word. At least

Re: Position search

2019-10-15 Thread Alexandre Rafalovitch
Is the 100 words a hard boundary or a soft one? If it is a hard one (always 100 words), the easiest is probably copy field and in the (unstored) copy, trim off whatever you don't want to search. Possibly using regular expressions. Of course, "what's a word" is an important question here.

RE: [EXTERNAL] Re: High cpu usage when adding documents to v7.7 solr cloud

2019-10-15 Thread Peter Lancaster
Hi Oleksandr, Thanks very much for help. Yes that jira looks like exactly our problem. I'll give that a go tomorrow. Cheers, Peter. -Original Message- From: Oleksandr Drapushko [mailto:drapus...@gmail.com] Sent: 15 October 2019 19:52 To: solr-user@lucene.apache.org Subject: [EXTERNAL]

Converting graph query to stream graph query

2019-10-15 Thread Natarajan, Rajeswari
Hi, curl -XPOST -H 'Content-Type: application/json' 'http://localhost:8983/solr/ggg/update' --data-binary '{ "add" : { "doc" : { "id" : "a", "docid" : "1", "name" : "Root document one" } }, "add" : { "doc" : { "id" : "b", "docid" : "2", "name" : "Root document two" } }, "add" : { "doc" : {

RE: Position search

2019-10-15 Thread Markus Jelsma
Hello Adi, There is no SpanLastQuery or equivalent. But you could reverse the text and use SpanFirstQuery. Or, perhaps easier, add a bogus term to the end of the field and use PhraseQuery. Regards, Markus -Original message- > From:Kaminski, Adi > Sent: Tuesday 15th October 2019

Re: High cpu usage when adding documents to v7.7 solr cloud

2019-10-15 Thread Oleksandr Drapushko
Hi Peter, This bug was introduced in Solr 7.7.0. It is related to Java 8. And it was fixed in Solr 7.7.2. Here are the ways to deal with it: 1. Upgrade to Solr 7.7.2 2. Patch your Solr 7.7 3. Use Java 9+ You can read more on this here: https://issues.apache.org/jira/browse/SOLR-13349 Regards,

Re: solr 8.1.1 many time slower returning query results than solr 4.10.4 or solr 6.5.1

2019-10-15 Thread Russell Bahr
Hi Shawn, I included the wrong file for solr4 and did not realize until you pointed out the heap size. The correct file that is setting the Java environment is "Solr 4 tomcat setenv" I have uploaded that to the shared folder along with the requested screenshots "Solr 4 top screenshot","Solr 6 top

High cpu usage when adding documents to v7.7 solr cloud

2019-10-15 Thread Peter Lancaster
We have a solr cloud on v7.7.0 and we observe very high cpu usage when we're indexing new documents. The solr cloud in question has 50 shards and 2 replicas of each and we're using NRT. Obviously indexing takes some resources but we see pretty much 100% cpu usage when we're indexing documents

Atomic Updates with PreAnalyzedField

2019-10-15 Thread Oleksandr Drapushko
Hello Community, I've discovered data loss bug and couldn't find any mention of it. Please confirm this bug haven't been reported yet. Description: If you try to update non pre-analyzed fields in a document using atomic updates, data in pre-analyzed fields (if there is any) will be lost. The

Re: Metrics API - Documentation

2019-10-15 Thread Andrzej Białecki
We keep all essential user documentation (and some dev docs) in the Ref Guide. The source for the Ref Guide is checked-in under solr/solr-ref-guide, it uses a simple ASCII markup so adding some content should be easy. You should follow the same workflow as with the code (create a JIRA, and then

Re: Unable to log into Jira

2019-10-15 Thread Erick Erickson
I was once “Chris Erickson”, but infra straightened it out. > On Oct 15, 2019, at 11:59 AM, Christine Poerschke (BLOOMBERG/ LONDON) > wrote: > > Hi Richard, > > Sorry to hear you're experiencing log-in difficulties. I've opened > https://issues.apache.org/jira/browse/INFRA-19280 for this,

Re:Unable to log into Jira

2019-10-15 Thread Christine Poerschke (BLOOMBERG/ LONDON)
Hi Richard, Sorry to hear you're experiencing log-in difficulties. I've opened https://issues.apache.org/jira/browse/INFRA-19280 for this, hopefully it can be read without logging in. Regards, Christine From: solr-user@lucene.apache.org At: 10/15/19 16:31:36To: solr-user@lucene.apache.org

Re: Metrics API - Documentation

2019-10-15 Thread Richard Goodman
Many thanks both for your responses, they've been helpful. @Andrzej - Sorry I wasn't clear on the "A latency of 1mil" as I wasn't aware the image wouldn't come through. But following your bullet points helped me present a better unit for measurement in the axis. In regards to contributing, would

Unable to log into Jira

2019-10-15 Thread Richard Goodman
Hey, Sorry if this is the wrong group, I tried to email us...@infra.apache.org a few weeks ago but haven't heard anything. I am unable to log into my account, with it saying my password is incorrect. But what is more odd is my name on the account has changed from Richard Goodman to Alex Goodman.

Re: Re: Query on autoGeneratePhraseQueries

2019-10-15 Thread Rohan Kasat
Also check , pf , pf2 , pf3 ps , ps2, ps3 parameters for phrase searches. Regards, Rohan K On Tue, Oct 15, 2019 at 6:41 AM Audrey Lorberfeld - audrey.lorberf...@ibm.com wrote: > I'm not sure how your config file is setup, but I know that the way we do > multi-token synonyms is to have the sow

RE: Facet Advice

2019-10-15 Thread Moyer, Brett
Hello Shawn, thanks for reply. The results that come back are correct, but are we implementing the query correctly to filter by a selected facet? When I say wrong, it's more about the design/use of Facets in the Query. Is it proper to do fq=Tags:Retirement? Is using a Multivalued field correct

Problems with Wildcard Queries / Own Filter

2019-10-15 Thread Björn Keil
Hello, I am having a bit of a problem with Wildcard queries and I don't know how to pin it down yet. I have a suspect, but I kind find an error in it, one of the filters in the respective search field. The problem is that when I do a wildcard query: title:todesmä* it does return a result, but it

Re: Re: Query on autoGeneratePhraseQueries

2019-10-15 Thread Audrey Lorberfeld - audrey.lorberf...@ibm.com
I'm not sure how your config file is setup, but I know that the way we do multi-token synonyms is to have the sow (split on whitespace) parameter set to False while using the edismax parser. I'm not sure if this would work with PhraseQueries , but it might be worth a try! In our config file

Re: Solr 7.6 frequent OOM with Java 9, G1 and large heap sizes - any tests with Java 13 and the new ZGC?

2019-10-15 Thread Shawn Heisey
On 10/15/2019 2:49 AM, Vassil Velichkov (Sensika) wrote: I've reduced the JVM heap on one of the shards to 20GB and then simulated some heavy load to reproduce the issue in a faster way. The solr.log ROOT was set to TRACE level, but I can't really see anything meaningful, the solr.log ends @

Re: Query on autoGeneratePhraseQueries

2019-10-15 Thread Shubham Goswami
Hi kshitij Thanks for the reply! I tried to debug it and found that raw query(black company) has parsed as two separate queries black and company and returning the results based on black query instead of this it should have got parsed as a single phrase query like("black company") because i am

Re: Minimum Tomcat version that supports latest Solr version

2019-10-15 Thread Shawn Heisey
On 10/15/2019 12:42 AM, vikas shinde wrote: Dear Solr team, Which is the latest Tomcat version that supports the latest Solr version 8.2.0? Also provide details about previous Solr versions & their compatible Tomcat versions. Dominique is correct. We do not officially support running under

Re: Facet Advice

2019-10-15 Thread Shawn Heisey
On 10/14/2019 3:25 PM, Moyer, Brett wrote: Hello, looking for some advice, I have the suspicion we are doing Facets all wrong. We host financial information and recently "tagged" our pages with appropriate Facets. We have built a Flat design. Are we going at it the wrong way? In Solr we have

Re: Minimum Tomcat version that supports latest Solr version

2019-10-15 Thread Dominique Bejean
Hi, Solr is not tested with Tomcat since version 4. Why not using the embedded Jetty server ? Regards Dominique Le mar. 15 oct. 2019 à 10:44, vikas shinde a écrit : > Dear Solr team, > > Which is the latest Tomcat version that supports the latest Solr version > 8.2.0? > > Also provide

Re: solr 8.1.1 many time slower returning query results than solr 4.10.4 or solr 6.5.1

2019-10-15 Thread Shawn Heisey
On 10/14/2019 1:36 PM, Russell Bahr wrote: Backend replacement of solr4 and hopefully Frontend replacement as well. solr-spec 8.1.1 lucene-spec 8.1.1 Runtime Oracle Corporation OpenJDK 64-Bit Server VM 12 12+33 1 collection 6 shards 5 replicas per shard 17,919,889 current documents (35 days

RE: Position search

2019-10-15 Thread Kaminski, Adi
Hi Markus, Thanks for the guidance. Is there any official Solr documentation for that ? Tried some googling, only some Stackoverflow / Lucene posts are available. Also, will that approach work for the other use case of searching from end of documents ? For example if I need to perform some

RE: Solr 7.6 frequent OOM with Java 9, G1 and large heap sizes - any tests with Java 13 and the new ZGC?

2019-10-15 Thread Vassil Velichkov (Sensika)
Hi Shawn, I've reduced the JVM heap on one of the shards to 20GB and then simulated some heavy load to reproduce the issue in a faster way. The solr.log ROOT was set to TRACE level, but I can't really see anything meaningful, the solr.log ends @ 07:31:40.352 GMT, while the GC log shows later

Minimum Tomcat version that supports latest Solr version

2019-10-15 Thread vikas shinde
Dear Solr team, Which is the latest Tomcat version that supports the latest Solr version 8.2.0? Also provide details about previous Solr versions & their compatible Tomcat versions. Thanks & Regards. Vikas Shinde.

Re: Query on autoGeneratePhraseQueries

2019-10-15 Thread kshitij tyagi
Hi, Try debugging your solr query and understand how it gets parsed. Try using "debug=true" for the same On Tue, Oct 15, 2019 at 12:58 PM Shubham Goswami wrote: > *Hi all,* > > I am a beginner to solr framework and I am trying to implement > *autoGeneratePhraseQueries* property in a fieldtype

RE: Position search

2019-10-15 Thread Markus Jelsma
Hello Adi, Try SpanFirstQuery. It limits the search to within the Nth term in the field. Regards, Markus -Original message- > From:Kaminski, Adi > Sent: Tuesday 15th October 2019 8:25 > To: solr-user@lucene.apache.org > Subject: Position search > > Hi, > What's the recommended way

Query on autoGeneratePhraseQueries

2019-10-15 Thread Shubham Goswami
*Hi all,* I am a beginner to solr framework and I am trying to implement *autoGeneratePhraseQueries* property in a fieldtype of type=text_general, i kept the property value as true and restarted the solr server but still it is not taking my two words query like(Black company) as a phrase without

Position search

2019-10-15 Thread Kaminski, Adi
Hi, What's the recommended way to search in Solr (assuming 8.2 is used) for specific terms/phrases/expressions while limiting the search from position perspective. For example to search only in the first/last 100 words of the document ? Is there any built-in functionality for that ? Thanks in