Re: Filter by sibling ?

2021-03-02 Thread Joel Bernstein
Solr's graph expressions can do this type of thing. It allows you to walk the relationships in a graph with filters: https://lucene.apache.org/solr/guide/8_6/graph-traversal.html Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Mar 2, 2021 at 9:00 AM Manoj Mokashi wrote: > Hi, >

Re: Idle timeout expired and Early Client Disconnect errors

2021-03-01 Thread Joel Bernstein
Also the parallel function builds hash partitioning filters that could lead to timeouts if they take too long to build. Try the query without the parallel function if you're still getting timeouts when making the query smaller. Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Mar 1, 2021

Re: Idle timeout expired and Early Client Disconnect errors

2021-03-01 Thread Joel Bernstein
parts would be a good strategy. Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Mar 1, 2021 at 3:30 PM ufuk yılmaz wrote: > Hello Mr. Bernstein, > > I’m using version 8.4. So, if I understand correctly, I can’t increase > timeouts and they are bound to happen in such a

Re: Idle timeout expired and Early Client Disconnect errors

2021-03-01 Thread Joel Bernstein
Oh wait, I misread your email. The idle timeout issue is configurable in: https://issues.apache.org/jira/browse/SOLR-14672 This unfortunately missed the 8.8 release and will be 8.9. This i Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Mar 1, 2021 at 2:56 PM Joel Bernstein wrote

Re: Idle timeout expired and Early Client Disconnect errors

2021-03-01 Thread Joel Bernstein
in the logs and were before the specific release. It might make sense to do a release that specifically suppresses these errors without backporting the full Solr 9.0 changes which impact the memory footprint of export. Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Mar 1, 2021 at 10:29 AM

Re: Congratulations to the new Apache Solr PMC Chair, Jan Høydahl!

2021-02-27 Thread Joel Bernstein
Congratulations Jan! Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Feb 22, 2021 at 2:41 AM Danilo Tomasoni wrote: > Congratulations Jan! > > Danilo Tomasoni > > Fondazione The Microsoft Research - University of Trento Centre for > Computational and Systems Biolog

Re: nodes() stream to infinite depth

2021-02-19 Thread Joel Bernstein
You could see if this meets you needs: https://lucene.apache.org/solr/guide/8_8/stream-source-reference.html#shortestpath Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Feb 19, 2021 at 2:45 PM Subhajit Das wrote: > Hi Joel, > > Thanks for response. But, is there any way to

Re: nodes() stream to infinite depth

2021-02-19 Thread Joel Bernstein
Nodes is designed for a stepwise graph walk. It doesn't do a full traversal. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Feb 19, 2021 at 4:47 AM Subhajit Das wrote: > > Hi, > > “{!graph ...}” goes to infinite depth by default. But “nodes()” stream > does not go to

Re: Significant terms expression giving error "length needs to be >= 1"

2021-02-16 Thread Joel Bernstein
Can you include the stack trace from the logs? Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Feb 15, 2021 at 3:53 PM ufuk yılmaz wrote: > We have a SolrCloud cluster, version 8.4 > > At the customer’s site there’s a collection with very few documents, > around 12. We

Re: Is there way to autowarm new searcher using recently ran queries

2021-01-27 Thread Joel Bernstein
searcher so you may need to change the auto-commit intervals. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Jan 27, 2021 at 5:30 PM Pushkar Raste wrote: > Hi, > > A rookie question. We have a Solr cluster that doesn't get too much > traffic. We see that our queries take long

Re: Streaming expressions, what is the effect of collection name inthe request url

2021-01-26 Thread Joel Bernstein
I have never tried this and didn't even know that you could have multiple collections in the URL. So, I'm really not sure what the behavior will be. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Jan 26, 2021 at 1:19 PM ufuk yılmaz wrote: > Does it have any ill side effects when

Re: NullPointerException in Graph Traversal nodes streaming expression

2021-01-26 Thread Joel Bernstein
be some exceptions before this, possibly on a different server if this has multiple servers involved. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Jan 21, 2021 at 5:46 PM Mike Drob wrote: > Can you provide a sample expression that would be able to reproduce this? > Are you able to try a

Re: Parallel streaming expression java.lang.IndexOutOfBoundsException

2021-01-26 Thread Joel Bernstein
Yes, this is not ideal. It means that the worker collection needs to have N shards rather than N replicas. Changing this should not be difficult if you'd like to provide a patch. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Jan 21, 2021 at 8:00 AM ufuk yılmaz wrote: > Loo

Re: Streaming expressions, what is the effect of collection name in the request url

2021-01-26 Thread Joel Bernstein
The URL path should be for one collection. This will be where the collection is compiled and run. It has no effect on what is actually being searched. That is specified in the expression themselves. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Jan 20, 2021 at 1:34 PM ufuk yılmaz wrote

Re: Steps to write a custom StreamingExpression

2021-01-26 Thread Joel Bernstein
I believe that would be the best path. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Jan 26, 2021 at 7:50 AM ufuk yılmaz wrote: > Should I create a java project with a dependency on solrj, or solr core ?, > then implement the Expressible interface > then build my project

Re: [Solr8.7] Performance of group.ngroups ?

2021-01-15 Thread Joel Bernstein
You can try collapse as well. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jan 15, 2021 at 4:51 AM Bruno Mannina wrote: > Hello, > > > > I found a temporary solution to my problem. > > > > I do a request without ngroups=true => result is quickly >

Re: Solr collapse & expand queries.

2020-11-30 Thread Joel Bernstein
of #1 and #2 above. If you remove group.ngroups grouping will usually be just as fast or faster then collapse. So in your testing, make sure you're testing the full data set with representative queries, and decide if group.ngroups is needed. Joel Bernstein http://joelsolr.blogspot.com

Re: Use stream result like a query (alternative to innerJoin)

2020-11-23 Thread Joel Bernstein
Here is the documentation for fetch: https://lucene.apache.org/solr/guide/8_4/stream-decorator-reference.html#fetch Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Nov 23, 2020 at 3:22 PM Joel Bernstein wrote: > There are two streams that behave like that. > > One is t

Re: Use stream result like a query (alternative to innerJoin)

2020-11-23 Thread Joel Bernstein
the main limitation is likely to be a blocker for you which is that it doesn't support one-to-many joins yet. Joel Bernstein http://joelsolr.blogspot.com/ On Sun, Nov 22, 2020 at 10:37 AM ufuk yılmaz wrote: > Hi all, > > I’m looking for a way to query two collections and find documents

Re: How to use the "eval" streaming expression?

2020-11-19 Thread Joel Bernstein
This blog gets more specific with some of the ideas behind the eval expression: https://joelsolr.blogspot.com/2017/04/having-talk-with-solr-using-new-echo.html Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Nov 19, 2020 at 12:21 PM Joel Bernstein wrote: > You could have a prog

Re: How to use the "eval" streaming expression?

2020-11-19 Thread Joel Bernstein
You could have a program that writes a Streaming Expression programmatically then use eval to run it. You can also save Streaming Expression data structures: tuple, list, array etc... and eval them into live streams that can be iterated. Joel Bernstein http://joelsolr.blogspot.com/ On Wed

Re: Using Multiple collections with streaming expressions

2020-11-12 Thread Joel Bernstein
The multiple collection syntax has been implemented for only a few stream sources: search, timeseries, facet and stats. Eventually it will be implemented for all stream sources. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Nov 10, 2020 at 12:32 PM ufuk yılmaz wrote: > Thanks ag

Re: Strange fetch streaming expression doesn't fetch fields sometimes?

2020-10-14 Thread Joel Bernstein
Yes, the docs mention one-to-one and many-to-one fetches, but one-to-many is not supported currently. I've never really been happy with fetch. It really needs to be replaced with a standard nested loop join that handles all scenarios. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Oct 13

Re: Using streaming expressions with shards filter

2020-10-06 Thread Joel Bernstein
Actually it's: .shards=shard1,shard2,shard3... Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Oct 6, 2020 at 2:38 PM Joel Bernstein wrote: > > There is a parameter in streaming expressions for this but it is not > available for use in every stream source. The search expressi

Re: Using streaming expressions with shards filter

2020-10-06 Thread Joel Bernstein
no-SolrCloud streaming expressions but was not fully realized yet. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Oct 1, 2020 at 11:31 AM Gael Jourdan-Weil < gael.jourdan-w...@kelkoogroup.com> wrote: > Hello, > > I am trying to use a Streaming Expression to query only a subset

Re: Loading JSON docs into Solr with Streaming Expressions?

2020-07-24 Thread Joel Bernstein
It's probably time to add JSON loading support to streaming expressions, but nothing yet. This ticket is almost done and paves the way for a suite of parseXYZ functions: https://issues.apache.org/jira/browse/SOLR-14673 Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jul 24, 2020 at 1:00

Re: Parallel SQL join on multivalue fields

2020-07-01 Thread Joel Bernstein
There isn't any real support for joins in Parallel SQL currently. I'm surprised that you're having some success doing them. Can you provide a sample SQL join that is working for you? Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jun 26, 2020 at 3:32 AM Piero Scrima wrote: >

Re: Use cases for the graph streams

2020-05-21 Thread Joel Bernstein
the correlations and relationships within the data set. Graph traversal is part of this broader goal which will get developed more over time. I'd be interested in hearing more about specific graph use cases that you're interested in solving. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, May

Re: REINDEXCOLLECTION not working on an alias

2020-05-19 Thread Joel Bernstein
I believe the issue is that under the covers this feature is using the "topic" streaming expressions which it was just reported doesn't work with aliases. This is something that will get fixed, but for the current release there isn't a workaround for this issue. Joel Bern

Re: using aliases in topic stream

2020-05-16 Thread Joel Bernstein
the alias would break the topic. But, I'm not sure that's a good enough reason not to support aliases with the topic if its note that aliases must be stable in the documentation. Joel Bernstein http://joelsolr.blogspot.com/ On Sat, May 16, 2020 at 11:10 AM Nightingale, Jonathan A (US

Re: Solr 8.1.5 Postlogs - Basic Authentication Error

2020-05-15 Thread Joel Bernstein
Right now this is not, but this would be fairly easy to add. I'll see if I can get that in for the next release. Joel Bernstein http://joelsolr.blogspot.com/ On Mon, May 11, 2020 at 5:03 PM Waheed, Imran wrote: > Is there a way to use bin/postllogs with basic authentication on? I

Re: using aliases in topic stream

2020-05-14 Thread Joel Bernstein
This is where the alias work was done: https://issues.apache.org/jira/browse/SOLR-9077 It could be though that there is a bug here. I'll see if I can reproduce it locally. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, May 14, 2020 at 6:24 PM Nightingale, Jonathan A (US

Re: using aliases in topic stream

2020-05-13 Thread Joel Bernstein
What version of Solr are you using? The topic stream in master seems to have the code in place to query aliases. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, May 13, 2020 at 12:33 PM Nightingale, Jonathan A (US) < jonathan.nighting...@baesystems.com> wrote: > Hi Everyone

Re: facets & docValues

2020-05-07 Thread Joel Bernstein
Bernstein http://joelsolr.blogspot.com/ On Tue, May 5, 2020 at 5:54 PM Revas wrote: > Hi joel, No, we have not, we have softCommit requirement of 2 secs. > > On Tue, May 5, 2020 at 3:31 PM Joel Bernstein wrote: > > > Have you configured static warming queries for the face

Re: facets & docValues

2020-05-05 Thread Joel Bernstein
Have you configured static warming queries for the facets? This will warm the cache structures for the facet fields. You just want to make sure you commits are spaced far enough apart that the warming completes before a new searcher starts warming. Joel Bernstein http://joelsolr.blogspot.com

Re: a new CLI tool bin/postlogs

2020-03-29 Thread Joel Bernstein
As long as the data is loading you are fine I believe. We can create a ticket to figure out that error, but it's not affecting the logic of the load in any way. Joel Bernstein http://joelsolr.blogspot.com/ On Sun, Mar 29, 2020 at 2:29 AM Kayak28 wrote: > Hello, Community: > >

Re: a new CLI tool bin/postlogs

2020-03-27 Thread Joel Bernstein
. So if there are different types of files in the directory it's pointed to it will have unexpected behavior. So you can run it on a single log file, or a directory containing only log files. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Mar 27, 2020 at 5:18 AM Kayak28 wrote: > He

Re: Stream InnerJoin to merge hierarchal data

2020-02-07 Thread Joel Bernstein
arentId but the sorts do not allow that as one of the joins is sorted on childid. One possible solution is to use fetch to retrieve the parent for the child: https://lucene.apache.org/solr/guide/8_0/stream-decorator-reference.html#fetch Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Feb

Re: Bug in scoreNodes function of streaming expressions?

2020-01-29 Thread Joel Bernstein
Here is the ticket: https://issues.apache.org/jira/browse/SOLR-14231 Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Jan 29, 2020 at 10:03 AM Joel Bernstein wrote: > Hi Pratik, > > I'll create the ticket now and report back. If you've got a fix please > post it to the tic

Re: Bug in scoreNodes function of streaming expressions?

2020-01-29 Thread Joel Bernstein
Hi Pratik, I'll create the ticket now and report back. If you've got a fix please post it to the ticket and I'll try to get this in for the next release. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Jan 28, 2020 at 11:52 AM pratik@semandex wrote: > Joel Bernstein wrote >

Re: SQL selectable fields

2020-01-24 Thread Joel Bernstein
Does "_nest_path_" come back in a normal search? I would expect that the fields that are returned by normal searches would also work in SQL. If that turns out to be the case you could derive the fields from performing a search and seeing what fields are returned. Joel Bern

Re: Select with concat function not working in 8.4.1

2020-01-24 Thread Joel Bernstein
going forward. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jan 24, 2020 at 5:16 AM Guilherme Nunes < guilherme.nu...@biologis.com> wrote: > Greetings. > > A follow-up to the below with my findings. > > The problem seems to be that t

Re: JSON Facet doesn't allow date range facets

2019-12-12 Thread Joel Bernstein
; > On 12/12/2019 12:11, Joel Bernstein wrote: > > So something like this should work: > > > > [2000-05-01T00:00:01Z TO 2019-06-02T00:00:01Z] > > > > Joel Bernstein > > http://joelsolr.blogspot.com/ > > > > > > On Thu, Dec 12, 2019 at 7:08

Re: JSON Facet doesn't allow date range facets

2019-12-12 Thread Joel Bernstein
So something like this should work: [2000-05-01T00:00:01Z TO 2019-06-02T00:00:01Z] Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Dec 12, 2019 at 7:08 AM Joel Bernstein wrote: > With the DatePointField you can still do the range query, but I believe > you'll need to specify th

Re: JSON Facet doesn't allow date range facets

2019-12-12 Thread Joel Bernstein
With the DatePointField you can still do the range query, but I believe you'll need to specify the full ISO date string: 2000-05-01T01:01:01Z Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Dec 12, 2019 at 6:46 AM Mel Mason wrote: > Unfortunately I need a date range field, e.g [2000

Re: JSON Facet doesn't allow date range facets

2019-12-12 Thread Joel Bernstein
There is a field type in the schema called pdate: This should work for you. The timeseries Streaming Expression uses the JSON facet API for range faceting and works really well. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Dec 12, 2019 at 6:28 AM Mel Mason wrote: > Hi, >

Re: Solr Admin Console hangs on Chrome

2019-12-10 Thread Joel Bernstein
Did a recent change to Chrome cause this? In Solr 8x, I'm not seeing slowness with Chrome on Mac. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Dec 10, 2019 at 8:26 PM SAGAR INGALE wrote: > I am also facing the same issue for v6.4.0 > > On Wed, 11 Dec, 2019, 5:37 AM Joel

Re: Solr Admin Console hangs on Chrome

2019-12-10 Thread Joel Bernstein
What version of Solr? Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Dec 10, 2019 at 5:58 PM Arnold Bronley wrote: > I am also facing similar issue. I have also switched to other browsers to > solve this issue. > > On Tue, Dec 10, 2019 at 2:22 PM Webster Homer &l

Re: How do I add my own Streaming Expressions?

2019-11-20 Thread Joel Bernstein
Yeah this not documented. Here are two links that will be helpful: https://issues.apache.org/jira/browse/SOLR-9103 Slide 40 Shows the solrconfig.xml approach to registering new streams: https://www.slideshare.net/lucidworks/creating-new-streams-presented-by-dennis-gove-bloomberg-lp Joel

Re: Solr 8.3 Solrj streaming expressions do not return all field values

2019-11-05 Thread Joel Bernstein
it's something going on below streaming expressions. If you are seeing the fields then it's the expressions themselves that are not handling the data as expected. Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Nov 4, 2019 at 9:09 AM Jörn Franke wrote: > Most likely this issue can bei a

Re: [CAUTION] Converting graph query to stream graph query

2019-10-18 Thread Joel Bernstein
I believe we were debugging why graph results were not being returned in a different thread. It looks like the same problem. Is your Solr instance a straight install or have you moved config files from an older version of Solr to a newer version of Solr. Joel Bernstein http

Re: Help with Stream Graph

2019-10-18 Thread Joel Bernstein
The query that is created to me looks looked good but it returns no results. Let's just do a basic query using the select handler: product_s:product1 If this brings back zero results then we know we have a problem with the data. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Oct 18

Re: Help with Stream Graph

2019-10-17 Thread Joel Bernstein
Can you show the logs from this request. There will be a Solr query that gets sent with product1 searched against the product_s field. Let's see how many documents that query returns. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Oct 17, 2019 at 1:41 AM Rajeswari Natarajan wrote: >

Re: The Visual Guide to Streaming Expressions and Math Expressions

2019-10-16 Thread Joel Bernstein
can be incorporated into the Solr code eventually but the test frameworks are quite different. I think some simple scripts can be included with the Solr to automated the downloads for Zeppelin and Zeppelin-Solr. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Oct 16, 2019 at 11:27 AM Pratik

The Visual Guide to Streaming Expressions and Math Expressions

2019-10-16 Thread Joel Bernstein
CSV files so that more time can be spent on analysis and visualization. https://github.com/apache/lucene-solr/blob/visual-guide/solr/solr-ref-guide/src/loading.adoc#loading-data Joel Bernstein

Re: igain query parser generating invalid output

2019-10-11 Thread Joel Bernstein
This sounds like a great patch. I can help with the review and commit after the jira is created. Thanks! Joel On Fri, Oct 11, 2019 at 1:06 AM Peter Davie < peter.da...@convergentsolutions.com.au> wrote: > Hi, > > I apologise in advance for the length of this email, but I want to share > my

Re: Trying to add model name to classify() output

2019-09-25 Thread Joel Bernstein
You can use the val function, which will just returns the string. val(CRIME) as expected Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Sep 23, 2019 at 10:00 PM Peter Davie < peter.da...@convergentsolutions.com.au> wrote: > Hi, > > I have trained a number of logi

Re: Incremental export of a huge collection

2019-09-12 Thread Joel Bernstein
This will do what you describe: https://lucene.apache.org/solr/guide/8_1/stream-source-reference.html#topic Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Sep 9, 2019 at 4:18 PM Mikhail Khludnev wrote: > Isn't _version_ a timestamp of insertion by default? > > On Mon, Sep 9,

Re: using let() with other streaming expressions

2019-08-16 Thread Joel Bernstein
is called "stream". Here is the syntax: let( a=search(techproducts, q="cat:electronics", fl="id, manu, price", sort="id asc"), b=search(techproducts, q="cat:electronics", fl="id, popularity, _version_", sort="id

Re: SQL equality predicate escaping single quotes

2019-08-09 Thread Joel Bernstein
It does appear that single quotes are being removed. If you want to provide a patch that allows single quotes to get passed through, I can help with testing and committing. On Thu, Aug 8, 2019 at 11:28 AM Kyle Lilly wrote: > Hi, > > When using the SQL handler is there any way to escape single

Re: Returning multiple fields in graph streaming expression response documents

2019-07-21 Thread Joel Bernstein
Good to hear. Joel Bernstein http://joelsolr.blogspot.com/ On Sun, Jul 21, 2019 at 5:21 PM Ahmed Adel wrote: > Yeah, it turned out to be related to the data. The “fetch” method works > fine as you described, it’s just the data distribution that caused name > field not to b

Re: Returning multiple fields in graph streaming expression response documents

2019-07-20 Thread Joel Bernstein
are returned. It could be a bug in the code or it could be something related to the data that's being fetched. Joel Bernstein http://joelsolr.blogspot.com/ On Sat, Jul 20, 2019 at 5:21 PM Ahmed Adel wrote: > To validate this, I indexed the datasets and ran the same query on Solr > 6.5.0 envir

Re: Returning multiple fields in graph streaming expression response documents

2019-07-19 Thread Joel Bernstein
is using edismax etc... This needs to be fixed. Can you check to see if the qparser is for the /select handler on your install? Anyway fetch needs to be reverted back to it's previous implementation before the above commit basically broke it. Joel Bernstein http://joelsolr.blogspot.com

Re: Returning multiple fields in graph streaming expression response documents

2019-07-19 Thread Joel Bernstein
e you have the fields reversed on the fetch. If that doesn't work, I'll investigate further. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jul 19, 2019 at 5:51 AM Ahmed Adel wrote: > Hi Joel, > > Thank you for your thoughts. I tried the fetch function, however, the > response

Re: Returning multiple fields in graph streaming expression response documents

2019-07-18 Thread Joel Bernstein
Hi Ahmed, Take a look at the fetch https://lucene.apache.org/solr/guide/8_0/stream-decorator-reference.html#fetch It probably makes sense to allow more field to be returned from a nodes expression as well. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Jul 17, 2019 at 3:12 AM Ahmed Adel

Re: Bug in scoreNodes function of streaming expressions?

2019-07-02 Thread Joel Bernstein
Ok, that sounds like a bug. I can create a ticket for this. On Mon, Jul 1, 2019 at 5:57 PM Pratik Patel wrote: > I think the problem was that my streaming expression was always returning > just one node. When I added more data so that I can have more than one > node, I started seeing the

Re: creating date facets

2019-06-20 Thread Joel Bernstein
You might find this useful. If makes creating time series aggregations a little easier. It uses JSON facets under the covers and is very fast. https://lucene.apache.org/solr/guide/7_6/stream-source-reference.html#timeseries Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Jun 19, 2019 at 1

Re: Issues with the handling of NULLs in Streaming Expressions

2019-06-06 Thread Joel Bernstein
is encountered it's treated as a string literal and doesn't preserve the null. So I think it's worth creating a ticket outlining your findings and we can think about solutions. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Jun 6, 2019 at 9:22 AM Oleksandr Chornyi wrote: > Hi guys! >

Re: NullPointerException with ExpandComponent on Collapsed Null Values

2019-06-04 Thread Joel Bernstein
This should be considered a bug. Feel free file jira for this. Joel Bernstein http://joelsolr.blogspot.com/ On Tue, Jun 4, 2019 at 9:16 AM aus...@3bx.org.INVALID wrote: > Just wanted to provide a bit more information on this issue after > experimenting a bit more. > > Th

Re: Solr-8.1.0 uses much more memory

2019-05-26 Thread Joel Bernstein
actually existed. Joel Bernstein http://joelsolr.blogspot.com/ On Sun, May 26, 2019 at 3:30 PM Shawn Heisey wrote: > On 5/26/2019 12:52 PM, Joe Doupnik wrote: > > I do queries while indexing, have done so for a long time, without > > difficulty nor memory usage spike

Re: How to use encrypted username password.

2019-05-20 Thread Joel Bernstein
Typically basic auth is encrypted using SSL. Joel Bernstein http://joelsolr.blogspot.com/ On Mon, May 20, 2019 at 6:49 PM Gangadhar Gangadhar wrote: > Hi, > >I’m trying to explore if there is any way to encrypt -basicauth or > encrypt username and password in -Dsolr.httpc

Re: Performance of /export requests

2019-05-12 Thread Joel Bernstein
impact? > Also, is the 30k sliding window configurable? In many cases we are > streaming back a few thousand, maybe up to 10k and then cutting off the > stream. If we could configure the size of that window, could that speed > things up some? > > Thanks again for the info. >

Re: Performance of /export requests

2019-05-11 Thread Joel Bernstein
Can you share the sort criteria and search query? The main strategy for improving performance of the export handler is adding more shards. This is different than with typical distributed search, where deep paging issues get worse as you add more shards. With the export handler if you double the

Re: Streaming Expression: get the value of the array at the specified position

2019-05-11 Thread Joel Bernstein
There actually is an undocumented function called valueAt. It works both for an array and for a matrix. For an array: let(echo="b", a=array(1,2,3,4,5), b=valueAt(a, 2)) should return 3. I have lot's of documentation still to do. Joel Bernstein http://joelsolr.blogspot.com

Re: Why did Solr stats min/max values were returned as float number for field of type="pint"?

2019-05-02 Thread Joel Bernstein
quot;, max(id_i), max(response_d)) Joel Bernstein http://joelsolr.blogspot.com/ On Thu, May 2, 2019 at 8:47 PM Wendy2 wrote: > Hi Solr users, > > I have a pint field: > indexed="true" stored="true"/> > > But Solr stats min/max values were re

Re: Pagination with streaming expressions

2019-05-02 Thread Joel Bernstein
into that category. Eventually though I will clear that ticket if someone else doesn't resolve it first. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, May 1, 2019 at 7:56 PM Erick Erickson wrote: > This sounds like an XY problem. You’re asking now to paginate, but not > explaining the p

Re: solr 7.x sql query returns null

2019-04-19 Thread Joel Bernstein
Ok I updated the ticket, we can move the discussion there. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Apr 19, 2019 at 7:44 AM David Barnett wrote: > Hi Joel > > BUG created in jira SOLR-13414 > > Please let me know if you need more info > > Thanks > > D

Re: solr 7.x sql query returns null

2019-04-18 Thread Joel Bernstein
a jira ticket and attach your schema we can track this down. I'll probably attach a special binary to the ticket which has additional logging so we can can find out what field is causing the problem. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Apr 18, 2019 at 1:38 PM David Barnett wrote

Re: solr 7.x sql query returns null

2019-04-18 Thread Joel Bernstein
This let's make sure the jdbc URL is correct. Reloading the collection shouldn't effect much unless the schema is different. But as Shawn mentioned the stack trace is not coming from Solr. Is there more in the logs beyond the Calcite exception? Joel Bernstein http://joelsolr.blogspot.com

Re: solr 7.x sql query returns null

2019-04-18 Thread Joel Bernstein
I ask this because SQL/JDBC may return a similar error if you try to run it on a non-Solr Cloud index. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Apr 18, 2019 at 10:16 AM Joel Bernstein wrote: > Was the original index a Solr Cloud index? > > > > Joel B

Re: solr 7.x sql query returns null

2019-04-18 Thread Joel Bernstein
Was the original index a Solr Cloud index? Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Apr 18, 2019 at 7:48 AM David Barnett wrote: > I have a large solr 7.3 collection 400m + documents. > > I’m trying to use the Solr JDBC driver to query the data

Re: Topic & Executor streaming expressions

2019-04-15 Thread Joel Bernstein
This blog covers this topic is some depth: https://joelsolr.blogspot.com/2017/01/deploying-solrs-new-parallel-executor.html Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Apr 15, 2019 at 11:19 AM Nazerke S wrote: > Hi everyone! > > Can anyone elaborate the topic and executor

Re: gatherNodes question. Is this a bug?

2019-04-12 Thread Joel Bernstein
t;: { "docs": [ { "node": "01/02444-7", "type_status_facet ": "Ongoing projetcs", "amount": 154620 }, { "node": "01/08149-7", "type_status_facet "

Re: gatherNodes question. Is this a bug?

2019-04-10 Thread Joel Bernstein
What you're trying to do should work. Possibly of you provide more detail like the full query with some sample outputs I might be able to see what the issue is. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Apr 10, 2019 at 10:55 AM Kojo wrote: > Hello everybody I have a question ab

Re: var, sttdev Streaming Evaluators.

2019-04-10 Thread Joel Bernstein
They currently are not. You can use describe() to get these values and getValue() if you want to use a specific value. let(arr=array(1,3,3), m=describe(a), s=getValue(m, stdev)) It makes sense to add these on there own as well. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Apr 10

Re: Gather Nodes Streaming

2019-03-21 Thread Joel Bernstein
gatherNodes requires single value fields in the tuples. In certain scenarios the cartesianProduct streaming expression can be used to explode a multi-value field into a single field stream. But in the scenario you describe this might not be possible. Joel Bernstein http://joelsolr.blogspot.com

Re: Re-read from CloudSolrStream

2019-02-20 Thread Joel Bernstein
It sounds like you just need to catch the exception? Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Feb 18, 2019 at 3:14 AM SOLR4189 wrote: > Hi all, > > Let's say I have a next code: > > http://joelsolr.blogspot.com/2015/04/the-streaming-api-solrjio-basics.

Re: Under-utilization during streaming expression execution

2019-02-15 Thread Joel Bernstein
Use large batches and fetch instead of hashjoin and lots of parallel workers. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Feb 15, 2019 at 7:48 PM Joel Bernstein wrote: > You can run in parallel and that should help quite a bit. But at a really > large batch job is better don

Re: Under-utilization during streaming expression execution

2019-02-15 Thread Joel Bernstein
You can run in parallel and that should help quite a bit. But at a really large batch job is better done like this: https://joelsolr.blogspot.com/2016/10/solr-63-batch-jobs-parallel-etl-and.html Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Feb 14, 2019 at 6:10 PM Gus Heck wrote: >

Re: Solr collapse result repeat in 6.6.5 cloud example techproducts.

2019-02-07 Thread Joel Bernstein
Do you have more then one shard? Collapse requires that all docs in the same collapse group be co-located on the same shard. Grouping I believe does not require this is some scenarios. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Feb 7, 2019 at 4:07 PM 刘正 wrote: > i try requ

Re: Single query to get the count for all individual collections

2019-01-21 Thread Joel Bernstein
quot; has been around for a while if you have an older version of Solr https://lucene.apache.org/solr/guide/7_6/stream-decorator-reference.html#list_expression Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Jan 21, 2019 at 12:53 PM Jens Brandt wrote: > Hi, > > ma

Re: Get MLT Interesting Terms for a set of documents corresponding to the query specified

2019-01-21 Thread Joel Bernstein
You find the significantTerms streaming expressions useful: https://lucene.apache.org/solr/guide/7_6/stream-source-reference.html#significantterms Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Jan 21, 2019 at 3:02 PM Pratik Patel wrote: > Aman, > > Thanks for the reply! &

Re: Error using collapse parser with /export

2019-01-21 Thread Joel Bernstein
) After exporting is it possible to unique the records using the unique Streaming Expression? Either of those cases would be the typical uses of these features. Joel Bernstein http://joelsolr.blogspot.com/ On Sun, Jan 20, 2019 at 10:13 PM Rahul Goswami wrote: > Hello, > > Following up on

Re: 6.3 -> 6.4 Sorting responseWriter renamed

2019-01-11 Thread Joel Bernstein
The functionality should be exactly the same. The config files though need to be changed. I would recommend adding any custom configs that you have to the new configs following the ExportWriter changes. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Jan 10, 2019 at 11:21 AM Raveendra

Re: Combine & Sort - SOLR and SQL Data

2018-12-17 Thread Joel Bernstein
olr/guide/7_6/stream-decorator-reference.html#merge Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Dec 17, 2018 at 8:57 AM Tech Support wrote: > Dear Team, > > Once per day, my data importing tool will collect data from SQL server and > add it into SOLRCloud. > > Current day da

Re: Shuffling Tuples - Parallel SQL

2018-12-07 Thread Joel Bernstein
in the field. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Dec 6, 2018 at 9:25 AM swathi wrote: > Hi, > > I am reading on Solr’s Parallel SQL. > > > How each replica partition the results by using HashQParser plugin and > shuffle the tuples with same values in the partit

Re: Streaming In Solr

2018-11-14 Thread Joel Bernstein
;let" expression to set variables and operate of vectors and matrices in memory. But basically you can think of it as decorators operating over streams of data. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Nov 14, 2018 at 3:26 AM Lucky Sharma wrote: > Hi Prakhar, > Than

Re: Median in Solr json facet api

2018-11-14 Thread Joel Bernstein
adjust the sample size to see how it effects the estimate. Here is the link to Solr Math Expressions in the User Guide: https://lucene.apache.org/solr/guide/7_5/math-expressions.html Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Nov 14, 2018 at 8:21 AM Toke Eskildsen wrote: >

Re: Unable to get Solr Graph Traversal working

2018-11-08 Thread Joel Bernstein
on Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Nov 7, 2018 at 1:22 PM Vidhya Kailash wrote: > I am unable to get even simple graph traversal expressions like the one > below to work in my environment (7.4 and 7.5 versions). They simply yield > no results, even though I know the da

Re: streaming expressions substring-evaluator

2018-10-31 Thread Joel Bernstein
The replace operator is going to be "replaced" :) Let's create an umbrella ticket for string operations and list out what would be nice to have. They can probably be added very quickly. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Oct 31, 2018 at 8:49 AM Gus Heck wrote:

  1   2   3   4   5   6   7   8   >