Re: Solr operator and side car container

2023-12-14 Thread Joel Bernstein
We found the place to add the side car. Seems to be working as expected. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Dec 12, 2023 at 10:35 AM Joel Bernstein wrote: > Hi, > > Is there any facility in the Solr operator for adding a sidecar container > that runs along s

Solr operator and side car container

2023-12-12 Thread Joel Bernstein
Hi, Is there any facility in the Solr operator for adding a sidecar container that runs along side the solr container? I see there is an initContainer but I don't see anything for adding another container. Thanks, Joel

Re: Conditional query re-ranking

2023-12-01 Thread Joel Bernstein
Currently it does not. I'm assuming you mean conditions related to the main query result set. Can you give an example of what type of condition you have in mind? Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Nov 28, 2023 at 10:29 AM rajani m wrote: > Hi Solr Users, > > D

Re: Vector math with Streaming Expressions?

2023-11-09 Thread Joel Bernstein
hamber of Secrets"", fl="id,name,film_vector"), film_vector), b=col(a, film_vector), m=matrix(b), average=scalarDivide(length(b), sumColumns(m)) ) Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Nov 7, 2023 at 10:42 AM Eric Pugh wrote:

Re: Vector math with Streaming Expressions?

2023-10-15 Thread Joel Bernstein
This would in theory return the average of the vectors: let(a=select(search(...), film_vector), b=col(a, film_vector), m=matrix(valueAt(b, 0), valueAt(b, 1), valueAt(b, 2)), av=scalarDivide(3, sumColumns(m)) Joel Bernstein http://joelsolr.blogspot.com/ On Sat, Oct 14, 2023

Re: knn parser not working as expected

2023-08-07 Thread Joel Bernstein
Oh wait, I just saw your post above with the config. I suspect it's the hardcoded defType that is causing this problem. Try testing with this removed to confirm. Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Aug 7, 2023 at 8:32 PM Joel Bernstein wrote: > So, the knn query is be

Re: knn parser not working as expected

2023-08-07 Thread Joel Bernstein
So, the knn query is being parsed into an edismax query in the attached debug output. So something is definitely not working as expected. In your solrconfig.xml do you have a hardcoded qparser for the request handler? Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Aug 4, 2023 at 12:09 PM

Re: min date function - Solr Streaming

2023-06-06 Thread Joel Bernstein
Looks like this ticket never got documented: https://issues.apache.org/jira/browse/SOLR-10303 In your case you'll want to use the epoch function which returns unix time: select(search(collection), epoch(date_dt) as utime) I'll create a jira to document this feature. Joel

Re: Streaming expr as LTR feature

2023-06-01 Thread Joel Bernstein
supports that. Streaming expressions were never designed to perform at scale like that. So it would be hard to integrate streaming expressions into a system that required high qps and sub-second performance. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Jun 1, 2023 at 10:28 AM rajani m

Re: Streaming expr as LTR feature

2023-05-31 Thread Joel Bernstein
The select function is documented here: https://solr.apache.org/guide/solr/latest/query-guide/transform.html Joel Bernstein http://joelsolr.blogspot.com/ On Wed, May 31, 2023 at 1:51 PM Joel Bernstein wrote: > The array function doesn't operate in the way its being used here. H

Re: Streaming expr as LTR feature

2023-05-31 Thread Joel Bernstein
quot;p": 2.2449532856850816 }, { "p": 1.7212359783803421 }, { "p": 2.761290822044021 }, Joel Bernstein http://joelsolr.blogspot.com/ On Wed, May 31, 2023 at 12:59 PM Alessandro Benedetti wrote: > Hi, > we are working on contributing the possibility of having v

Re: join query parser performance

2023-05-25 Thread Joel Bernstein
One thing to understand about the topLevelDv approach is you'll need to warm both sides of the join. You can do this by adding a static warming query that facets on 'group_id_mv' and 'group_member_id' in both collections. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, May 25, 2023 at 9

Re: join query parser performance

2023-05-25 Thread Joel Bernstein
If you are using a recent version of Solr try adding the parameter method=topLevelDV Let us know how this effects performance in your use case. What matters most here is the number of documents the from side of join matches. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, May 25

Re: streaming expressions - sharding memory usage

2023-05-10 Thread Joel Bernstein
with scoring performs as well as it does. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, May 10, 2023 at 10:34 AM Sergio García Maroto wrote: > Thanks Joel for your answer. > > Actually what I need is to return scores from different collections and > make some calculations o

Re: streaming expressions - sharding memory usage

2023-05-10 Thread Joel Bernstein
you describe the exact use case you need to accomplish? For example, do you need to extract a large number of documents by joining streams of scored data? Or can you display just the top N documents of the joined streams? Joel Bernstein http://joelsolr.blogspot.com/ On Wed, May 10, 2023 at 6

Re: streaming expressions - sharding memory usage

2023-05-09 Thread Joel Bernstein
Can you share the expressions? Then we can discuss where the sharding comes into play. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, May 9, 2023 at 1:17 PM Sergio García Maroto wrote: > Hi, > > I am working currently on implementing sharding on current Solr Cloud > Clu

Re: Solr logs (hits value) and memory allocation

2023-05-04 Thread Joel Bernstein
It would also depend on the query. For example collapse keeps a Map of groups heads gathered during the query. A large result set and a high cardinality group field would result in more memory usage. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, May 3, 2023 at 3:11 PM Kevin Risden

Re: becoming a solr specialist

2023-05-04 Thread Joel Bernstein
I'll ping you on LinkedIn. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, May 4, 2023 at 1:53 PM Eric Pugh wrote: > Relevance Slack has an active #jobs channel as well. The magic invite > link is www.opensourceconnections.com/slack < > http://www.opensourceconnectio

Re: Suggestions to improve Star queries latencies

2023-04-19 Thread Joel Bernstein
To send the query to a single shard you can add the parameter "distrib=false" to the query and it will stay on that shard. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Apr 19, 2023 at 5:21 PM Joel Bernstein wrote: > You're hunting for a bottleneck. Here is how I w

Re: Suggestions to improve Star queries latencies

2023-04-19 Thread Joel Bernstein
at the bottleneck. If its very fast thats Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Apr 19, 2023 at 3:57 PM Rajani Maski wrote: > Thank you, Mikhail. > > > On Wed, Apr 19, 2023 at 7:59 AM Mikhail Khludnev wrote: > > > Hello, Rajani. > > I meant [SOLR-14765] optimize D

Re: Solr Node High CPU

2023-03-27 Thread Joel Bernstein
It was mentioned earlier but the thread dump is a good place to start looking. Typically you'd see a number of threads doing something CPU intensive. After that the heap dump will show you what parameters were used in the threads that running. Joel Bernstein http://joelsolr.blogspot.com

Re: Fastest way to index data to solr

2022-09-30 Thread Joel Bernstein
loading data without a soft commit, then set autoSoftCommit to an interval that balances load performance with memory retention. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Sep 30, 2022 at 12:37 PM Andy Lester wrote: > I can’t imagine a case where the speed in parsing the input d

Re: NullPointer Exception when using Cross Collection Join

2022-09-29 Thread Joel Bernstein
Can you share the stack trace? Also in the Solr log there will be a call to the /export handler. Can you get that from the log? Then we can isolate the call to the export handler and see if we can reproduce it. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Sep 29, 2022 at 3:01 PM

Re: NPE in collapse

2022-09-29 Thread Joel Bernstein
Oh, there are no segments... If this error is still occurring in the latest Solr version without top_fc hint then it's a bug. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Sep 29, 2022 at 3:27 PM Joel Bernstein wrote: > What version of Solr are you using? > > Try removing t

Re: NPE in collapse

2022-09-29 Thread Joel Bernstein
What version of Solr are you using? Try removing the top_fc hint, does the error still occur? Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Sep 29, 2022 at 12:47 PM 南拓弥 wrote: > Hello all, > > NPE in collapse hint=top_fc, > When there are no segments, using hint=top_fc

Re: Stream Expression Count matches

2022-09-22 Thread Joel Bernstein
It would also be good to map the null expression to a "tupleCount" function name so its more intuitive. I will create a ticket for this. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Sep 22, 2022 at 9:31 AM Joel Bernstein wrote: > There is an undocumented "null

Re: Stream Expression Count matches

2022-09-22 Thread Joel Bernstein
There is an undocumented "null" expression which will return the count: null(expr()) It would of course be good to document this expression. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Sep 20, 2022 at 10:08 AM Sergio García Maroto wrote: > Hi, > > I am trying t

Re: JDBC stream expression

2022-09-15 Thread Joel Bernstein
The subquery question: You can't use a sub-query, but perhaps you could use a join instead. For example: hashJoin(jdbc(), search()) Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Sep 13, 2022 at 2:36 PM James Greene wrote: > Houston, > > Placing the jdbc driver in the w

Re: JDBC stream expression

2022-09-11 Thread Joel Bernstein
I suspect this is a classloader issue. You may have to move the driver into the lib directory with the Solr core and Solrj libs so that its visible to both DIH and streaming expressions. Joel Bernstein http://joelsolr.blogspot.com/ On Sat, Sep 10, 2022 at 10:18 PM James Greene wrote

Re: JDBC stream expression

2022-09-09 Thread Joel Bernstein
You may be having classloader issues. I'm not sure where you have the driver jar, but it may need to be copied into the lib directory with core Solr jars. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Sep 9, 2022 at 4:58 PM Joel Bernstein wrote: > Can you post the stack trace f

Re: JDBC stream expression

2022-09-09 Thread Joel Bernstein
Can you post the stack trace from the log? It looks like the JDBCStream is registered, but the stack trace may provide some more info on why the driver didn't load. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Sep 9, 2022 at 12:51 PM James Greene wrote: > I'm still unable to

Re: JDBC stream expression

2022-09-08 Thread Joel Bernstein
is: org.apache.solr.client.solrj.io.stream.JDBCStream Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Sep 8, 2022 at 6:54 PM James Greene wrote: > I'm on solr 8.11.1 and trying to use the 'jdbc' streaming expression as > documented here: > https://solr.apache.org/guide/8_11/stre

Re: shards.tolerant support for stream handlers

2022-09-08 Thread Joel Bernstein
This is not something that has been tested yet with the stream handler unfortunately. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Sep 8, 2022 at 8:26 AM Chetan Rautela wrote: > Hi All, > > > While using stream handler API in Solr 7.4.0. > I am getting following e

Re: MultiLevel Collapse Supproted by Solr?

2022-09-08 Thread Joel Bernstein
/browse/SOLR-14329 Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Sep 8, 2022 at 1:21 AM Daxesh Parmar wrote: > Hii Joel, Thanks for the reply. > But I was not able to find any official documentation stating multiple > collapses. It would be helpful if you got any article about tha

Re: Results appear to be wrongly ordered after sorting

2022-09-07 Thread Joel Bernstein
The sort is on a tokenized field. You'll likely need to sort on a string or SortableTextField to get the correct ordering. https://solr.apache.org/guide/8_6/field-types-included-with-solr.html Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Sep 7, 2022 at 8:37 AM Netta Steinberg wrote

Re: using multiple level facets in streaming expressions

2022-09-07 Thread Joel Bernstein
You could upack the tree from the fields in the tuples but that would be extra work. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Sep 7, 2022 at 2:51 AM Shankar R wrote: > Thanks a lot Joel. if we need hierarchical facets then we cannot use > streaming expressions right? >

Re: using multiple level facets in streaming expressions

2022-09-06 Thread Joel Bernstein
multi-dimensional result. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Sep 6, 2022 at 8:19 AM t sornin wrote: > Joel send me this a while back: > > https://issues.apache.org/jira/browse/SOLR-9103 > > You can write your own plugin to make the multiple facet stream

Re: MultiLevel Collapse Supproted by Solr?

2022-09-06 Thread Joel Bernstein
on the collapse filter which will control the order of the collapse and select the expand field. There is also an undocumented expand parameter: expand.field This will allow you to specify which field to expand on. Collapse will support more than one collapse with versions Solr 8.5+. Joel Bernstein http

Re: using multiple level facets in streaming expressions

2022-09-05 Thread Joel Bernstein
The visual guide to streaming expressions and math expressions is here: https://solr.apache.org/guide/8_9/math-expressions.html Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Sep 5, 2022 at 5:14 PM Joel Bernstein wrote: > > There is an example here: > > https://solr.apac

Re: using multiple level facets in streaming expressions

2022-09-05 Thread Joel Bernstein
There is an example here: https://solr.apache.org/guide/8_9/search-sample.html#facet And also here: https://solr.apache.org/guide/8_9/search-sample.html#facet2d Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Sep 5, 2022 at 11:07 AM Shankar R wrote: > Hi, > Is it possible

Re: how to use solr streaming expressions without using zkhost

2022-09-04 Thread Joel Bernstein
This blog explains how to use the SolrStream to send a streaming expression to a Solr endpoint without zkhost information https://lucidworks.com/post/streaming-expressions-in-solrj/ Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Sep 2, 2022 at 1:40 PM Eric Pugh wrote: > I think i

Re: Weird KNN search error that happens periodically

2022-08-23 Thread Joel Bernstein
Can you post the stack trace from the logs? Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Aug 23, 2022 at 7:01 PM Derek C wrote: > Hi all, > > We have 3 X SOLR instances running SOLR cloud and I'm doing KNN > searches and KNN reranking queries across a couple of milli

Re: SOLR 9.0 - Mixing KNN and traditional queries - possible?

2022-08-22 Thread Joel Bernstein
This appears to be what you are looking for: https://issues.apache.org/jira/browse/SOLR-16246 Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Aug 19, 2022 at 6:10 AM Derek C wrote: > Hi all, > > I have a collection with about 2.5 Million documents. I've been > e

Re: Stream - substitute result from output of a stream expr as filter query to another expr

2022-08-22 Thread Joel Bernstein
If the sorts of the two streams are not the same you can use the hashJoin: https://solr.apache.org/guide/8_4/stream-decorator-reference.html#hashjoin Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Aug 22, 2022 at 12:00 PM Joel Bernstein wrote: > I believe what you're look

Re: Stream - substitute result from output of a stream expr as filter query to another expr

2022-08-22 Thread Joel Bernstein
I believe what you're looking for is an innerJoin: https://solr.apache.org/guide/8_4/stream-decorator-reference.html#innerjoin Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Aug 17, 2022 at 12:47 PM Susheel Kumar wrote: > Hello, > > How can we substitute an output from a

Re: ExternalFileField2, massively scalable external file fields

2022-07-28 Thread Joel Bernstein
been awhile! Hope you are doing well. > > > > > > The feature you are proposing reminded me of the Lucidworks work I was > > > involved with for Lowe's Canada and how to deal with changing product > > > prices... Whether in or out of Solr it seems like th

ExternalFileField2, massively scalable external file fields

2022-07-27 Thread Joel Bernstein
processing is a required data preparation step it may be outside of the scope of Solr. But it also might fit as a module. I'm curious how the community feels about this as a module. Joel Bernstein http://joelsolr.blogspot.com/

Re: SE - "EXCEPTION": "Unable to check eq(...) because a null value was found",

2022-07-06 Thread Joel Bernstein
I suspect the null related failure is occurring while initializing the second parameter. So the failure is occurring before the boolean logic is applied. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Jul 6, 2022 at 6:12 PM Kojo wrote: > Yes, I don't know if not short circuit for n

Re: Partial updates with Update Stream Decorator

2022-07-06 Thread Joel Bernstein
That would be quite tricky to create with existing functions. Did you find a way to inject a tuple into a field? Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Jul 6, 2022 at 12:10 PM Dan Rosher wrote: > Answering my own question, this IS possible but you need to send to the >

Re: SE - "EXCEPTION": "Unable to check eq(...) because a null value was found",

2022-07-06 Thread Joel Bernstein
So the AND didn't short circuit. That seems like it needs to be fixed. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Jul 6, 2022 at 3:20 PM Kojo wrote: > I have made it work, like this: > > > having( > having( > ANY STREAM > and(notNull(NULL FIELD), True)), >

Re: SE - "EXCEPTION": "Unable to check eq(...) because a null value was found",

2022-07-05 Thread Joel Bernstein
Here are some null handling examples: https://solr.apache.org/guide/solr/latest/query-guide/transform.html#handling-nulls Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Jul 5, 2022 at 4:44 PM Kojo wrote: > I found this e-mail archive > https://lists.apache.org/

Re: SE fetch function

2022-06-29 Thread Joel Bernstein
That is correct. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Jun 29, 2022 at 3:54 PM Kojo wrote: > Joel, > follow below solrconfig.xml. > > > > > explicit > 10 > 18 > > 1 > 0 > 15000 > 1

Re: SE fetch function

2022-06-28 Thread Joel Bernstein
if not overridden in the solrconfig. See this ticket for details: https://issues.apache.org/jira/browse/SOLR-11501. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Jun 28, 2022 at 7:59 PM Kojo wrote: > I am upgrading from Solr 6.6 to 9.0 > > I have some SE working fine on 6.6

Re: Semantic Knowledge Graph theoric question

2022-06-22 Thread Joel Bernstein
-parameters Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Jun 22, 2022 at 2:37 AM Danilo Tomasoni wrote: > Hello Dave, first of all thank you for your answer. > > I need to clarify that I've used separate (and quite good) NER algorithms > offline and the results w

Re: Facet counts for first N hits

2022-06-03 Thread Joel Bernstein
handler. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jun 3, 2022 at 8:24 AM Dave wrote: > Yeah, my first thought would be to have the first query with no facets and > a fl of just the id, limited to 1000, it’s a lot faster than you think if > you only return the id and

Re: Collapsing on a field works, but expand=true does nothing

2022-06-01 Thread Joel Bernstein
I just did a quick check on Solr 9 and expand / collapse was working. Here is the output: https://gist.github.com/joel-bernstein/6f7f3ee12d5375630f3311c5dbd693ee Is it possible that the expand component isn't registered in your deployment? The expand component is a default component but have you

Re: Solr Cloud and /export

2022-06-01 Thread Joel Bernstein
name. This function is also basically the same functionality as calling the "search" function with the param qt=/export. In Solr 11 there is also the drill function: https://solr.apache.org/guide/8_11/stream-source-reference.html#drill Joel Bernstein http://joelsolr.blogspot.com/ On Tue,

Re: Param Substitution in Stream Expressions?

2022-06-01 Thread Joel Bernstein
is as follows: -DStreamingExpressionMacros=true Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Jun 1, 2022 at 1:22 PM James Greene wrote: > My application relies on local parameter substitution pretty heavily due to > escaping issues and being able to re-use clauses. > > Is there an

Re: Snowflake vs Solr

2022-04-25 Thread Joel Bernstein
in Solr. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Apr 22, 2022 at 11:07 AM David Hastings < hastings.recurs...@gmail.com> wrote: > Those features are not database situations, no one goes from a search > engine to a DB, sometimes you go from a DB to search if you're

Re: How to boost documents containing as many foreign keys as provided?

2022-04-19 Thread Joel Bernstein
. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Apr 13, 2022 at 10:12 AM Marko Avlijas wrote: > Hello everyone, > > I am working on a music app. We have songs and songs can have > multiple genres. > We have a song - genre join table. > We are indexing genre ids for eac

Re: Representative filtering of very large result sets

2022-03-24 Thread Joel Bernstein
Yeah, that's a tricky problem. Keeping the result set small without losing results. I don't have an answer except as you already mentioned which would be to limit the query in some way. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Mar 24, 2022 at 8:24 AM Jeremy Buckley - IQ-C wrote

Re: Representative filtering of very large result sets

2022-03-23 Thread Joel Bernstein
documents with the same collapse key to the same shard. Then run the collapse query on the sharded collection. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Mar 23, 2022 at 9:42 PM Jeremy Buckley - IQ-C wrote: > The number of documents in the collection is about 90 mill

Re: Representative filtering of very large result sets

2022-03-23 Thread Joel Bernstein
It sounds like you are collapsing on a high cardinality field and/or faceting on high cardinality fields. Can you describe the cardinality of the fields so we can get an idea of how large the problem is? Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Mar 23, 2022 at 12:30 PM Jeremy

Re: solr-operator stale reads

2022-03-08 Thread Joel Bernstein
Based on this ticket I think it's Kube: https://github.com/kubernetes/kubernetes/issues/59848 Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Mar 8, 2022 at 7:35 PM Joel Bernstein wrote: > Hi, > > One of the things we've run into in our tests is that we appear to be > getting

solr-operator stale reads

2022-03-08 Thread Joel Bernstein
and see if stale reads are typical with Kube state or if this might be an issue with our application. Joel Bernstein http://joelsolr.blogspot.com/

Re: Storing logs in Apache Solr

2022-02-21 Thread Joel Bernstein
/8_11/logs.html http://joelsolr.blogspot.com/2021/02/driving-down-cloud-storage-costs-with.html https://solr.apache.org/guide/8_11/graph.html#temporal-graph-expressions Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Feb 21, 2022 at 10:52 AM Z0ltrix wrote: > No, we use a more gene

Re: unexpected wildcard results

2022-02-18 Thread Joel Bernstein
This is a great tool for understanding how analyzers are handling specific terms: https://solr.apache.org/guide/8_8/analysis-screen.html You'll be able to see how witchcraft was added to the index. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Feb 18, 2022 at 3:47 PM Matthew Roth wrote

Re: Retrieve Score Streaming Exppression

2022-02-10 Thread Joel Bernstein
The export handler doesn't currently export score. If you switch to the /select handler in the search function score will be available using the score field, but the query will be bounded (not a full export) Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Feb 9, 2022 at 9:02 AM Sergio

Re: ClassCastException in StreamingBinaryResponseParser when using export handler

2022-02-03 Thread Joel Bernstein
/ Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Feb 2, 2022 at 1:09 PM Ufuk YILMAZ wrote: > Solr version is 8.4 > > I'm trying to use the export handler through SolrJ: > > CloudSolrClient cloudSolrClient = ... > SolrQuery q = new SolrQuery(); > q.setParam(&q

Re: "Slow" Query performance with boosts.

2022-01-19 Thread Joel Bernstein
One other thing to check is the performance on each node. You can do this by running the query with the parameter distrib=false on each node. A distributed search is only as fast as the slowest node. So you'll want to rule out an underpowered node. Joel Bernstein http://joelsolr.blogspot.com

Re: "Slow" Query performance with boosts.

2022-01-19 Thread Joel Bernstein
the query. Gradually increase the result set size by adjusting the query. You then can get a feel for how result set size affects performance. This will give you an indication how much it will help to have more shards. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Jan 19, 2022 at 6:19

Re: Solrj BucketBasedJsonFacet.java not parsing "missing" counts

2022-01-18 Thread Joel Bernstein
this will be in a 9x release. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Jan 18, 2022 at 6:05 AM Michelle Allen wrote: > > I am using Solr 8.11.1 and json facets. > > My problem is that solrj does not provide a way, in the QueryResponse > getJsonFacetingResponse, to g

Re: Sudden increase in threads

2022-01-10 Thread Joel Bernstein
SolrStream will be created inside of the worker collection nodes. Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Jan 10, 2022 at 4:37 AM 123456780sss <123456780...@protonmail.com.invalid> wrote: > Sorry I forgot to add - we are using Solr 6.5.1 > > Sent with [ProtonMail](https://prot

Re: Non equi-joins with streaming expressions

2021-12-29 Thread Joel Bernstein
As you mentioned currently only the equi-join is supported. But you could pretty quickly adapt an existing join to do what you want. https://github.com/apache/solr/blob/main/solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/LeftOuterJoinStream.java Joel Bernstein http

Re: What happens when a streaming expression is sent to an alias

2021-12-29 Thread Joel Bernstein
parameter that is passed to the stream. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Dec 28, 2021 at 5:11 AM Ufuk YILMAZ wrote: > When you send an expression to an alias named "myAlias" pointing to N > number of collections, each having M number of replicas, how does it >

Re: No live solr servers & Invalid key request timestamp error

2021-12-09 Thread Joel Bernstein
These types of logs point to instability in the cluster. There are likely recoveries etc... to go along with them. If that is the case you'll want to get to the root cause of the instability. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Dec 7, 2021 at 11:00 PM Reej M wrote: > Hi

Re: Stream Query result Cacheable ?

2021-12-07 Thread Joel Bernstein
cache but does not use the query result cache like the /select handler. Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Dec 6, 2021 at 11:32 AM Michael Gibney wrote: > Could you share the specific requests/times that you're comparing? Oher > potentially-relevant details like inde

Re: solr-operator formatting of cpu resource

2021-11-23 Thread Joel Bernstein
I opened and closed an issue for this. It turns out this happens with standard Deployments objects as well. So, it's something other than the Solr-operator. Sorry for the noise. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Nov 23, 2021 at 2:17 PM Joel Bernstein wrote: > The mor

Re: solr-operator formatting of cpu resource

2021-11-23 Thread Joel Bernstein
, breaks that contract. I'll create an issue for this so we can discuss how to handle this. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Nov 23, 2021 at 1:13 PM Joel Bernstein wrote: > A little more testing has shown that the cpu value is only reformatted if > it's a multiple o

Re: solr-operator formatting of cpu resource

2021-11-23 Thread Joel Bernstein
A little more testing has shown that the cpu value is only reformatted if it's a multiple of 1000. I'm not sure if the solr-operator can control this or if this part of the go operator framework. But if it's controllable ideally we would probably not reformat the cpu. Joel Bernstein http

solr-operator formatting of cpu resource

2021-11-23 Thread Joel Bernstein
Format: format, } } Joel Bernstein http://joelsolr.blogspot.com/

Re: Solr tutorial in spanish in this web

2021-11-16 Thread Joel Bernstein
Looks great! Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Nov 16, 2021 at 7:11 AM Luis Calonge wrote: > > Hi. I consider Solr to be an excellent application, and I found that a > simple tutorial in Spanish about Solr was missing, which would make this > program known to

Re: Poor performance with many segments and indexing. [was: Is there a way to set max segment count...?]

2021-11-04 Thread Joel Bernstein
on disk by storing fewer fields. 2) Use stopwords to decrease the size of the positions stored in the index. 3) Use more shards. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Nov 4, 2021 at 8:35 AM Michael Conrad wrote: > Would either of these two settings possibly h

Re: 90% heap occupied by CompressingStoredFieldsReader

2021-10-28 Thread Joel Bernstein
Does this tie back to a Solr cache? I've seen situations where the document cache becomes a memory hog with large documents. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Oct 28, 2021 at 10:11 AM Amrit Sarkar wrote: > Hi everyone, > > I have a question wit

Re: boosting specific number of Products

2021-10-26 Thread Joel Bernstein
This may be what you're looking for: https://solr.apache.org/guide/8_8/query-re-ranking.html Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Oct 21, 2021 at 2:39 PM sachin gk wrote: > Thanks Dave, if I interpret correctly below expression will only boost top > 20 products

Re: Solr to get latest version of documents

2021-10-15 Thread Joel Bernstein
Solr will overwrite the documents as they are indexed. So if they are indexed in the order shown in your email you will get the correct versions. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Oct 15, 2021 at 1:02 AM krishna prasad < krishna.kittugenious...@gmail.com> wrote: &g

Re: Rolling restarts and the Solr Operator

2021-10-14 Thread Joel Bernstein
Thanks Houston, You are right, the main motivation for the SolrCloud per shard is auto-scaling. Here is the issue I created: https://github.com/apache/solr-operator/issues/348 Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Oct 14, 2021 at 2:09 PM Houston Putman wrote: > Ok, I fo

Re: Rolling restarts and the Solr Operator

2021-10-14 Thread Joel Bernstein
object in the cluster. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Oct 12, 2021 at 6:44 PM Joel Bernstein wrote: > Hi, > > I saw that the Solr operator takes into account collection topology when > performing rolling restarts. In a situation where there is one SolrCloud

Re: TopicStream and sorting?

2021-10-14 Thread Joel Bernstein
Hi Eric, The topic stream emulates a FIFO queue so it only knows about version. One thing that would be really nice would be a Streaming Expression wrapper around cursor mark queries, which supports sorting. The CursorStream would fit nicely along side the TopicStream and ExportStream. Joel

Re: Solr keeps creating new threads until it dies

2021-10-12 Thread Joel Bernstein
There is a thread dump on the Solr admin. You can use that to determine what all those threads are doing and where they are getting stuck. You can post parts of the thread dump back to this email thread as well. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Oct 12, 2021 at 11:15 AM

Rolling restarts and the Solr Operator

2021-10-12 Thread Joel Bernstein
Hi, I saw that the Solr operator takes into account collection topology when performing rolling restarts. In a situation where there is one SolrCloud object per-shard, I'm wondering how this will behave. In this case the Solr Operator would receive a different CR for each shard which would kick

Re: Solr 8 autocommit confirmation

2021-10-08 Thread Joel Bernstein
still in the logs and looks like this: 2019-12-16 19:00:23.931 INFO (searcherExecutor-66-thread-1) [ ] o.a.s.c.SolrCore [production1_shard35_replica_n1] Registered new searcher Searcher@16ef5fac[production_cv_month_201912_shard35_replica_n1] ... Joel Bernstein http://joelsolr.blogspot.com

Re: about filter cache in case of collapse

2021-09-28 Thread Joel Bernstein
Cache is set to false internally for the CollapsingQParserPlugin. So you don't need to specify cache=false. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Sep 28, 2021 at 5:57 AM Taisuke Miyazaki wrote: > Hi, > > Will filter cache be created if I specify fq={!collapse f

Re: Use case counting items in a field

2021-09-24 Thread Joel Bernstein
If you're looking for something closer to Jaccard similarity you could use https://nightlies.apache.org/solr/draft-guides/solr-reference-guide-main/other-parsers.html#minhash-query-parser Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Sep 24, 2021 at 4:20 PM Joel Bernstein wrote: >

Re: Use case counting items in a field

2021-09-24 Thread Joel Bernstein
that had the most papers in the field, instead of matching the query? Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Sep 22, 2021 at 10:16 AM Sergio García Maroto wrote: > Hi, > > I am wondering if the following use case can be solve with Solr. I have > been researching and c

Re: Advance filtering requirement to filter out based on the max value of a field

2021-09-24 Thread Joel Bernstein
Take a look at: https://solr.apache.org/guide/8_8/collapse-and-expand-results.html#collapsing-query-parser The "sort" parameter I believe will work for your use case. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Sep 24, 2021 at 1:55 PM ishara cooray wrote: > Hi, >

Re: Email alerts with streaming expressions

2021-09-07 Thread Joel Bernstein
or it can be parallelized. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Sep 7, 2021 at 6:32 AM Charlie Hull wrote: > Hi Dan, > > Yuval and my suggestions both rely on the same underlying code (Luwak, > now called Lucene Monitor). This lets you store a set of Lucene queri

Re: solr /export broken?

2021-08-02 Thread Joel Bernstein
Can you send the Solr log records for the request? That will show us what parameters were sent to to Solr. Joel Bernstein http://joelsolr.blogspot.com/ On Sat, Jul 31, 2021 at 8:47 AM Dave wrote: > Why does it feel like you have a ) in the wrong spot ? > > > On Jul 31, 2021

Re: Commit strategy for Heavy Bulk Indexing into solr

2021-07-23 Thread Joel Bernstein
Whether you use real-time-get or not you still need to soft commit to release the memory used to support real-time-get. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jul 23, 2021 at 3:39 PM Pratik Patel wrote: > Thanks for the response Joel. > > We do not use "Real-tim

Re: Commit strategy for Heavy Bulk Indexing into solr

2021-07-23 Thread Joel Bernstein
visible. Opening a new searcher will make these records visible and free the memory. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jul 23, 2021 at 2:54 PM Pratik Patel wrote: > Solr Cloud version is 8.5. I have also attached the solr log with gc > enabled and our app log which

  1   2   >