Re: Creating 100000 dynamic fields in solr

Thanks Jan! This helps a lot! Sai Vignan Malyala On Mon, May 11, 2020 at 5:07 PM Jan Høydahl wrote: > Sounds like you are looking for parent/child docs here, see > https://lucene.apache.org/solr/guide/8_5/indexing-nested-documents.html > > { > "type": "user", > "name": "user1",

Re: Creating 100000 dynamic fields in solr

Thank you Jan, Vincezo and Joe. This helps us a lot. On Mon, May 11, 2020 at 10:03 PM Joe Obernberger < joseph.obernber...@gmail.com> wrote: > Could you use a multi-valued field for user in each of your products? > > So productA and a field User that is a list of all the users that have >

Re: Unbalanced shard requests

Thanks Michael! Yes in each shard I have 10 Tlog replicas, no other type of replicas, and each Tlog replica is an individual solr instance on its own physical machine. In the jira you mentioned 'when "last place matches" == "first place matches" – e.g. when shards.preference specified matches

Re: Limiting random results set with facets.

If you can tag your filter query, you can exclude it when faceting. Your results will honor the filter query and you will get the N results back, and since faceting will exclude the filter, it will still give you facet count for the base query.

Solr 8.1.5 Postlogs - Basic Authentication Error

Is there a way to use bin/postllogs with basic authentication on? I am getting error if do not give username/password bin/postlogs http://localhost:8983/solr/logs server/logs/ server/logs Exception in thread "main"

Limiting random results set with facets.

I'm looking for a way if possible to run a query with random results, where I limit the number of results I want back, yet still have the facets accurately reflect the results I'm searching. When I run a search I use a filter query to randomize the results based on a modulo of a random seed. This

Re: Unbalanced shard requests

FYI: https://issues.apache.org/jira/browse/SOLR-14471 Wei, assuming you have only TLOG replicas, your "last place" matches (to which the random fallback ordering would not be applied -- see above issue) would be the same as the "first place" matches selected for executing distributed requests.

What is the logical order of applying sorts in SOLR?

Hi SOLR Community, What is the order of operations which SOLR applies to sorting? I've observed many times and across SOLR versions that a restrictive filter with a sort takes an extremely long time to return, suggesting to me that the SORT is applied before the filter. An example situation is

Re: Unbalanced shard requests

Wei, probably no need to answer my earlier questions; I think I see the problem here, and believe it is indeed a bug, introduced in 8.3. Will file an issue and submit a patch shortly. Michael On Mon, May 11, 2020 at 12:49 PM Michael Gibney wrote: > > Hi Wei, > > In considering this problem, I'm

Re: Unbalanced shard requests

Hi Wei, In considering this problem, I'm stumbling a bit on terminology (particularly, where you mention "nodes", I think you're referring to "replicas"?). Could you confirm that you have 10 TLOG replicas per shard, for each of 6 shards? How many *nodes* (i.e., running solr server instances) do

8.5.1 LogReplayer extremely slow

Hello, Our main Solr text search collection broke down last night (search was still working fine), every indexing action timed out with the Solr master spending most of its time in Java regex. One shard has only one replica left for queries and it stays like that. I have copied both shard's

Re: Creating 100000 dynamic fields in solr

Could you use a multi-valued field for user in each of your products? So productA and a field User that is a list of all the users that have productA.  Then you could do a search like: user:User1 AND Product_A_cost:[5 TO 10] user:(User1 User5...) AND Product_B_cost[0 TO 40] -Joe On

Re: solr payloads performance

Wei - Here's some details on the various payload capabilities and short-comings: https://lucidworks.com/post/solr-payloads/ SOLR-10541 is the main functional constraint (range faceting over functions). Erik > On May 8, 2020, at 7:26 PM, Wei wrote: > > Hi everyone, > > Have a

Unified highlighter- unable to get results - can get results with original and termvector highlighters

I am running Solr 8.4 and am attempting to use its highlighting feature. It appears to work well when I use the original highlighter or the term vector highlighter, but when I try to use the unified highlighter, I get no results returned. My Google searches so far have not revealed anybody

Re: Creating 100000 dynamic fields in solr

For in-place updates you should read this: https://lucene.apache.org/solr/guide/8_1/updating-parts-of-documents.html On Mon, May 11, 2020 at 2:49 PM Vincenzo D'Amore wrote: > But keep in mind that "With the exception of in-place updates, the whole > block must be updated or deleted together,

Re: solr payloads performance

Hi Wei, In order to use payload you have to use functions and that’s not cheap. In order to make it work fast, you could use it as post filter and filter on some summary field like minPrice/maxPrice/defaultPrice. HTH, Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr &

Re: Creating 100000 dynamic fields in solr

But keep in mind that "With the exception of in-place updates, the whole block must be updated or deleted together, not separately. For some applications this may result in tons of extra indexing and thus may be a deal-breaker." On Mon, May 11, 2020 at 1:37 PM Jan Høydahl wrote: > Sounds like

Problems when Upgrading from Solr 7.7.1 to 8.5.0

Hi all, we recently upgraded our SolrCloud cluster from version 7.7.1 to version 8.5.0 and ran into multiple problems. In the end we had to revert the upgrade and went back to Solr 7.7.1. In our company we are using Solr since Version 4 and so far, upgrading Solr to a newer version was possible

Re: Max docs and num docs are not matching after optimization

That’s odd, are you absolutely sure that there’s no indexing going on while the optimize is running? Optimizing only works on the closed segments that exist when the process _starts_, any updates that come in while the optimize is running will result in new segments that are not optimized and

Re: Max docs and num docs are not matching after optimization

Please help On Mon, 11 May, 2020, 11:51 AM Rajdeep Sahoo, wrote: > Hi all, > We are using solr 7.7.2 . After optimization the deleted docs count is > still showing as part of max docs. > As per my knowledge after optimization max docs and num docs count > should match. It is not happening

Re: Creating 100000 dynamic fields in solr

Creating that many dynamic fields is a bad idea, Solr isn’t built to handle that many fields. It works, but performance will decline and I’d guess that this app is sensitive to response time. So try Jan’s approach or find another would be my advice. Best, Erick > On May 11, 2020, at 7:37 AM,

Re: Creating 100000 dynamic fields in solr

Sounds like you are looking for parent/child docs here, see https://lucene.apache.org/solr/guide/8_5/indexing-nested-documents.html { "type": "user", "name": "user1", "products": [ { "id": "prod_A", "cost": 50}, { "id": "prod_B", "cost":

Re: Response Time Diff between Collection with low deletes

As detailed below. The collection where we have issues have 16 shards with 2 replica each. On Sun, May 10, 2020, 9:10 PM matthew sporleder wrote: > Why so many shards? > > > On May 10, 2020, at 9:09 PM, Ganesh Sethuraman > wrote: > > > > We are using dedicated host, Cent OS in EC2

Re: Creating 100000 dynamic fields in solr

I have around 1M products used by my clients. Client need a filter of these 1M products by their cost filters. Just like: User1 has 5 products (A,B,C,D,E) User2 has 3 products (D,E,F) User3 has 10 products (A,B,C,H,I,J,K,L,M,N,O) ...every customer has different sets. Now they want to search

Re: Creating 100000 dynamic fields in solr

Sounds like an anti pattern. Can you explain what search problem you are trying to solve with this many unique fields? Jan Høydahl > 11. mai 2020 kl. 07:51 skrev Vignan Malyala : > > Hi > Is it good idea to create 10 dynamic fields of time pint in solr? > I have that many fields to search

Re: solr core metrics & prometheus exporter - indexreader is closed

Hey Dwane, Thanks for your email, gah I should have mentioned that I had applied the patches from 8.x branches onto the exporter already *(such as the fixed thread pooling that you mentioned). *I still haven't gotten to the bottom of the IndexReader is closed issue, I found that if that was

Ranking issue when combining sorting and re-ranking on SolrCloud (multiple shards)

HI all, On our current master/slave setup (no cloud), we use a a custom sorting function to get the first pass results (using the sort param), and then we use LTR for re-ranking. This works fine, i.e. re-ranking is applied on the topN, after sorting has completed and the order is correct.

Max docs and num docs are not matching after optimization

Hi all, We are using solr 7.7.2 . After optimization the deleted docs count is still showing as part of max docs. As per my knowledge after optimization max docs and num docs count should match. It is not happening here.. Is there any way to troubleshoot this.

Why Solr query time is more in case the searched value frequency is more even if no sorting is applied, for the same number of rows?

Suppose I have two phone numbers P1 and P2 and the number of records with P1 are X and with P2 are 2X (2 times X) respectively. If I query for R rows for P1 and P2, the QTime in case of P2 is more. I am not specifying any sort parameter and the number of rows I'm asking for is same in both the