date:20200516

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

2020-05-16 Thread vishal patel

Thanks for reply.

I know Query field value is large. But same thing is working fine in Solr 6.1.0 
and query executed within 300 milliseconds. Schema.xml and Solrconfig.xml are 
same. Why is it taking lots of time for execution in Solr 8.3.0?

Is there any changes in Solr 8.3.0?

Regards,
Vishal Patel

From: Mikhail Khludnev 
Sent: Saturday, May 16, 2020 6:55 PM
To: solr-user 
Subject: Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

It seems this thread is doing heavy work, mind the bottom line.

202.8013ms
124.8008ms
qtp153245266-156 (156)
org.apache.lucene.search.similarities.BM25Similarity$BM25Scorer.(BM25Similarity.java:219)
org.apache.lucene.search.similarities.BM25Similarity.scorer(BM25Similarity.java:192)
org.apache.lucene.search.similarities.PerFieldSimilarityWrapper.scorer(PerFieldSimilarityWrapper.java:47)
org.apache.lucene.search.TermQuery$TermWeight.(TermQuery.java:74)
org.apache.lucene.search.TermQuery.createWeight(TermQuery.java:205)
org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:726)
org.apache.lucene.search.BooleanWeight.(BooleanWeight.java:63)
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:231)
org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:726)
org.apache.lucene.search.TopFieldCollector.populateScores(TopFieldCollector.java:531)
org.apache.solr.search.grouping.distributed.command.TopGroupsFieldCommand.postCollect(TopGroupsFieldCommand.java:178)
org.apache.solr.search.grouping.CommandHandler.execute(CommandHandler.java:168)
org.apache.solr.handler.component.QueryComponent.doProcessGroupedDistributedSearchSecondPhase(QueryComponent.java:1403)
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:387)
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:328)
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
org.apache.solr.core.SolrCore.execute(SolrCore.java:2596)


It seems like it ranks groups by query score, that doubtful thing to do.

>From the log. Here's how to recognize query running 25 sec "QTime=25063"


Query itself q=+msg_id:(10519539+10519540+10523575+10523576+ ... is
not what search engines are made for. They are purposed for short
query.

You may

1. leverage {!terms} query parser which might handle such long terms
list more efficiently

2. make sure you don't enable unnecessary grouping features, eg group
ranking in the stack above makes no sense for this kind of query


It's worth to revamp an overall approach in favor of query time
{!join} or index time join see {!parent}/nested docs.



On Sat, May 16, 2020 at 1:46 PM vishal patel 
wrote:

> Thanks for reply.
>
> I have taken a thread dump at the time of query execution. I do not know
> the thread name so send the All threads. I have also send the logs so you
> can get idea.
>
> Thread Dump All Stack Trace:
> https://drive.google.com/file/d/1N4rVXJoaAwNvPIY2aw57gKA9mb4vRTMR/view
> Solr 8.3 shard 1 log:
> https://drive.google.com/file/d/1h5d_eZfQvYET7JKzbNKZwhZ_RmaX7hWf/view
> Solr 8.3 shard 2 log:
> https://drive.google.com/file/d/19CRflzQ7n5BZBNaaC7EFszgzKKlPfIVl/view
>
> I have some questions regarding the thread dump
> - How can I know the my thread name from thread dump? can I get from the
> log?
> - When do I take a thread dump? on query execution or after query
> execution?
>
> Note: I got a thread name from log and checked in thread dump on query
> execution time and after query executed. Both time thread stack trace got
> different.
>
> If any other things are required then let me know I will send.
>
> Regards,
> Vishal Patel
> 
> From: Mikhail Khludnev 
> Sent: Saturday, May 16, 2020 2:23 PM
> To: solr-user 
> Subject: Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1
>
> Can you check Thread Dump in Solr Admin while Solr 8.3 crunches query for
> 34 seconds? Please share the deepest thread stack. This might give a clue
> what's going on there.
>
> On Sat, May 16, 2020 at 11:46 AM vishal patel <
> vishalpatel200...@outlook.com>
> wrote:
>
> > Any one is looking my issue? Please help me.
> >
> > Sent from Outlook
> > 
> > From: vishal patel 
> > Sent: Friday, May 15, 2020 3:06 PM
> > To: solr-user@lucene.apache.org 
> > Subject: Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1
> >
> > I have result of query debug for both version so It will helpful.
> >
> > Solr 6.1 query debug URL
> > https://drive.google.com/file/d/1ixqpgAXsVLDZA-aUobJLrMOOefZX2NL1/view
> > Solr 8.3.1 query debug URL
> > https://drive.google.com/file/d/1MOKVE-iPZFuzRnDZhY9V6OsAKFT38U5r/view
> >
> > I indexed same data in both version.
> >
> > I found score=1.0 in result of Solr 8.3.0 and score=0.016147947 in result
> > of Solr 8.6.1. Is there any impact of score in query execution? why is
> > score=1.0 in result of Solr

Re: using aliases in topic stream

2020-05-16 Thread Joel Bernstein

I think you probably found the problem. The persisting of checkpoints
should not use aliases but in theory the aliases could work for the main
collection. It may be that the decision was made not to support aliases for
the topic stream because the checkpoints for each shard are saved and
changing the alias would break the topic. But, I'm not sure that's a good
enough reason not to support aliases with the topic if its note that
aliases must be stable in the documentation.


Joel Bernstein
http://joelsolr.blogspot.com/


On Sat, May 16, 2020 at 11:10 AM Nightingale, Jonathan A (US) <
jonathan.nighting...@baesystems.com> wrote:

> Joel,
> I looked at the commits from that enhancement briefly. It looks like the
> work to resolve te slices was put into a central method in CLoudSolrStream.
> But to check aliases the Boolean flag needs to be true.
>
> Everywhere this is used in TopicStream its set to false (3 places). The
> first two are for retrieving and persisting the topi checkpoints, bringing
> up the question, should aliases be allowed for that collection as well?
>
> The third is probably what I'm running into.
>
>   protected void constructStreams() throws IOException {
> try {
>   ZkStateReader zkStateReader = cloudSolrClient.getZkStateReader();
>   Slice[] slices = CloudSolrStream.getSlices(this.collection,
> zkStateReader, false);
>
> Anyways, this was my 5 minute investigation. I haven't taken the effort to
> try and change it. Do you think this could be it?
>
> Jonathan
> -Original Message-
> From: Joel Bernstein 
> Sent: Thursday, May 14, 2020 6:32 PM
> To: solr-user@lucene.apache.org
> Subject: Re: using aliases in topic stream
>
> *** WARNING ***
> EXTERNAL EMAIL -- This message originates from outside our organization.
>
>
> This is where the alias work was done:
>
> https://issues.apache.org/jira/browse/SOLR-9077
>
> It could be though that there is a bug here. I'll see if I can reproduce
> it locally.
>
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> On Thu, May 14, 2020 at 6:24 PM Nightingale, Jonathan A (US) <
> jonathan.nighting...@baesystems.com> wrote:
>
> > I'm looking on master on git hub, the solrj tests assume never use
> > aliases Just as an example. that’s all over the place in the tests
> >
> >
> > https://github.com/apache/lucene-solr/blob/master/solr/solrj/src/test/
> > org/apache/solr/client/solrj/io/stream/StreamDecoratorTest.java
> >
> > @Test
> >   public void testTerminatingDaemonStream() throws Exception {
> > Assume.assumeTrue(!useAlias);
> >
> > -Original Message-
> > From: Joel Bernstein 
> > Sent: Wednesday, May 13, 2020 1:11 PM
> > To: solr-user@lucene.apache.org
> > Subject: Re: using aliases in topic stream
> >
> > *** WARNING ***
> > EXTERNAL EMAIL -- This message originates from outside our organization.
> >
> >
> > What version of Solr are you using? The topic stream in master seems
> > to have the code in place to query aliases.
> >
> > Joel Bernstein
> > http://joelsolr.blogspot.com/
> >
> >
> > On Wed, May 13, 2020 at 12:33 PM Nightingale, Jonathan A (US) <
> > jonathan.nighting...@baesystems.com> wrote:
> >
> > > Hi Everyone,
> > >
> > > I'm trying to run this stream and I get the following error
> > >
> > > topic(topics,collection1,
> > > q="classes:GXP/INDEX",fl="uuid",id="feed-8",initialCheckpoint=0,chec
> > > kp
> > > ointEvery=-1)
> > >
> > > {
> > >   "result-set": {
> > > "docs": [
> > >   {
> > > "EXCEPTION": "Slices not found for collection1",
> > > "EOF": true,
> > > "RESPONSE_TIME": 6
> > >   }
> > > ]
> > >   }
> > > }
> > >
> > > "collection1" is an alias. I can search using the alias perfectly
> > > fine. In fact the search stream operation works fine with the alias.
> > > It's just this topic one I've seen so far. Does anyone know why this
> is?
> > >
> > > Thanks!
> > > Jonathan Nightingale
> > >
> > >
> >
>

Re: Unbalanced shard requests

2020-05-16 Thread Tomás Fernández Löbbe

I just backported Michael’s fix to be released in 8.5.2

On Fri, May 15, 2020 at 6:38 AM Michael Gibney 
wrote:

> Hi Wei,
> SOLR-14471 has been merged, so this issue should be fixed in 8.6.
> Thanks for reporting the problem!
> Michael
>
> On Mon, May 11, 2020 at 7:51 PM Wei  wrote:
> >
> > Thanks Michael!  Yes in each shard I have 10 Tlog replicas,  no other
> type
> > of replicas, and each Tlog replica is an individual solr instance on its
> > own physical machine.  In the jira you mentioned 'when "last place
> matches"
> > == "first place matches" – e.g. when shards.preference specified matches
> > *all* available replicas'.   My setting is
> > shards.preference=replica.location:local,replica.type:TLOG,
> > I also tried just shards.preference=replica.location:local and it still
> has
> > the issue. Can you explain a bit more?
> >
> > On Mon, May 11, 2020 at 12:26 PM Michael Gibney <
> mich...@michaelgibney.net>
> > wrote:
> >
> > > FYI: https://issues.apache.org/jira/browse/SOLR-14471
> > > Wei, assuming you have only TLOG replicas, your "last place" matches
> > > (to which the random fallback ordering would not be applied -- see
> > > above issue) would be the same as the "first place" matches selected
> > > for executing distributed requests.
> > >
> > >
> > > On Mon, May 11, 2020 at 1:49 PM Michael Gibney
> > >  wrote:
> > > >
> > > > Wei, probably no need to answer my earlier questions; I think I see
> > > > the problem here, and believe it is indeed a bug, introduced in 8.3.
> > > > Will file an issue and submit a patch shortly.
> > > > Michael
> > > >
> > > > On Mon, May 11, 2020 at 12:49 PM Michael Gibney
> > > >  wrote:
> > > > >
> > > > > Hi Wei,
> > > > >
> > > > > In considering this problem, I'm stumbling a bit on terminology
> > > > > (particularly, where you mention "nodes", I think you're referring
> to
> > > > > "replicas"?). Could you confirm that you have 10 TLOG replicas per
> > > > > shard, for each of 6 shards? How many *nodes* (i.e., running solr
> > > > > server instances) do you have, and what is the replica placement
> like
> > > > > across those nodes? What, if any, non-TLOG replicas do you have per
> > > > > shard (not that it's necessarily relevant, but just to get a
> complete
> > > > > picture of the situation)?
> > > > >
> > > > > If you're able without too much trouble, can you determine what the
> > > > > behavior is like on Solr 8.3? (there were different changes
> introduced
> > > > > to potentially relevant code in 8.3 and 8.4, and knowing whether
> the
> > > > > behavior you're observing manifests on 8.3 would help narrow down
> > > > > where to look for an explanation).
> > > > >
> > > > > Michael
> > > > >
> > > > > On Fri, May 8, 2020 at 7:34 PM Wei  wrote:
> > > > > >
> > > > > > Update:  after I remove the shards.preference parameter from
> > > > > > solrconfig.xml,  issue is gone and internal shard requests are
> now
> > > > > > balanced. The same parameter works fine with solr 7.6.  Still not
> > > sure of
> > > > > > the root cause, but I observed a strange coincidence: the nodes
> that
> > > are
> > > > > > most frequently picked for shard requests are the first node in
> each
> > > shard
> > > > > > returned from the CLUSTERSTATUS api.  Seems something wrong with
> > > shuffling
> > > > > > equally compared nodes when shards.preference is set.  Will
> report
> > > back if
> > > > > > I find more.
> > > > > >
> > > > > > On Mon, Apr 27, 2020 at 5:59 PM Wei  wrote:
> > > > > >
> > > > > > > Hi Eric,
> > > > > > >
> > > > > > > I am measuring the number of shard requests, and it's for query
> > > only, no
> > > > > > > indexing requests.  I have an external load balancer and see
> each
> > > node
> > > > > > > received about the equal number of external queries. However
> for
> > > the
> > > > > > > internal shard queries,  the distribution is uneven:6 nodes
> > > (one in
> > > > > > > each shard,  some of them are leaders and some are non-leaders
> )
> > > gets about
> > > > > > > 80% of the shard requests, the other 54 nodes gets about 20% of
> > > the shard
> > > > > > > requests.   I checked a few other parameters set:
> > > > > > >
> > > > > > > -Dsolr.disable.shardsWhitelist=true
> > > > > > > shards.preference=replica.location:local,replica.type:TLOG
> > > > > > >
> > > > > > > Nothing seems to cause the strange behavior.  Any suggestions
> how
> > > to
> > > > > > > debug this?
> > > > > > >
> > > > > > > -Wei
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Apr 27, 2020 at 5:42 PM Erick Erickson <
> > > erickerick...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Wei:
> > > > > > >>
> > > > > > >> How are you measuring utilization here? The number of incoming
> > > requests
> > > > > > >> or CPU?
> > > > > > >>
> > > > > > >> The leader for each shard are certainly handling all of the
> > > indexing
> > > > > > >> requests since they’re TLOG replicas, so that’s one thing that
> > > might
> > > > > > >> skewing your measurements.
>

Re: What is the logical order of applying sorts in SOLR?

2020-05-16 Thread David Hastings

the bq parameter, heres a SO thread for it:

https://stackoverflow.com/questions/45150856/how-to-know-when-to-use-solr-bq-vs-bf-and-how-to-apply-query-logic

On Sat, May 16, 2020 at 6:27 PM Stephen Lewis Bianamara <
stephen.bianam...@gmail.com> wrote:

> Hi Paras,
>
> I'm not sure I follow. How would one replace sort by id with boosting?
>
> Thanks,
> Stephen
>
> On Fri, May 15, 2020, 7:43 AM Paras Lehana
>  wrote:
>
> > As a workaround, can you try field boosting?
> >
> > On Tue, 12 May 2020 at 00:45, Stephen Lewis Bianamara <
> > stephen.bianam...@gmail.com> wrote:
> >
> > > Hi SOLR Community,
> > >
> > > What is the order of operations which SOLR applies to sorting? I've
> > > observed many times and across SOLR versions that a restrictive filter
> > with
> > > a sort takes an extremely long time to return, suggesting to me that
> the
> > > SORT is applied before the filter.
> > >
> > > An example situation is querying for fq:Foo=Bar vs querying for
> > fq:Foo=Bar
> > > sort by Id desc. I've observed over many SOLR versions and collections
> > that
> > > the former is orders of magnitude cheaper and quicker to respond, even
> > when
> > > the result set is tiny (10-100).
> > >
> > > Does anyone in this forum know whether this is the default behavior and
> > > whether there is any way through the API or SOLR configuration to apply
> > > sorts after filters?
> > >
> > > Thanks,
> > > Stephen
> > >
> >
> >
> > --
> > --
> > Regards,
> >
> > *Paras Lehana* [65871]
> > Development Engineer, *Auto-Suggest*,
> > IndiaMART InterMESH Ltd,
> >
> > 11th Floor, Tower 2, Assotech Business Cresterra,
> > Plot No. 22, Sector 135, Noida, Uttar Pradesh, India 201305
> >
> > Mob.: +91-9560911996
> > Work: 0120-4056700 | Extn:
> > *1196*
> >
> > --
> > *
> > *
> >
> >  
> >
>

Re: What is the logical order of applying sorts in SOLR?

2020-05-16 Thread Stephen Lewis Bianamara

Hi Paras,

I'm not sure I follow. How would one replace sort by id with boosting?

Thanks,
Stephen

On Fri, May 15, 2020, 7:43 AM Paras Lehana
 wrote:

> As a workaround, can you try field boosting?
>
> On Tue, 12 May 2020 at 00:45, Stephen Lewis Bianamara <
> stephen.bianam...@gmail.com> wrote:
>
> > Hi SOLR Community,
> >
> > What is the order of operations which SOLR applies to sorting? I've
> > observed many times and across SOLR versions that a restrictive filter
> with
> > a sort takes an extremely long time to return, suggesting to me that the
> > SORT is applied before the filter.
> >
> > An example situation is querying for fq:Foo=Bar vs querying for
> fq:Foo=Bar
> > sort by Id desc. I've observed over many SOLR versions and collections
> that
> > the former is orders of magnitude cheaper and quicker to respond, even
> when
> > the result set is tiny (10-100).
> >
> > Does anyone in this forum know whether this is the default behavior and
> > whether there is any way through the API or SOLR configuration to apply
> > sorts after filters?
> >
> > Thanks,
> > Stephen
> >
>
>
> --
> --
> Regards,
>
> *Paras Lehana* [65871]
> Development Engineer, *Auto-Suggest*,
> IndiaMART InterMESH Ltd,
>
> 11th Floor, Tower 2, Assotech Business Cresterra,
> Plot No. 22, Sector 135, Noida, Uttar Pradesh, India 201305
>
> Mob.: +91-9560911996
> Work: 0120-4056700 | Extn:
> *1196*
>
> --
> *
> *
>
>  
>

RE: using aliases in topic stream

2020-05-16 Thread Nightingale, Jonathan A (US)

Joel,
I looked at the commits from that enhancement briefly. It looks like the work 
to resolve te slices was put into a central method in CLoudSolrStream. But to 
check aliases the Boolean flag needs to be true.

Everywhere this is used in TopicStream its set to false (3 places). The first 
two are for retrieving and persisting the topi checkpoints, bringing up the 
question, should aliases be allowed for that collection as well?

The third is probably what I'm running into.

  protected void constructStreams() throws IOException {
try {
  ZkStateReader zkStateReader = cloudSolrClient.getZkStateReader();
  Slice[] slices = CloudSolrStream.getSlices(this.collection, 
zkStateReader, false);

Anyways, this was my 5 minute investigation. I haven't taken the effort to try 
and change it. Do you think this could be it?

Jonathan
-Original Message-
From: Joel Bernstein  
Sent: Thursday, May 14, 2020 6:32 PM
To: solr-user@lucene.apache.org
Subject: Re: using aliases in topic stream

*** WARNING ***
EXTERNAL EMAIL -- This message originates from outside our organization.


This is where the alias work was done:

https://issues.apache.org/jira/browse/SOLR-9077

It could be though that there is a bug here. I'll see if I can reproduce it 
locally.



Joel Bernstein
http://joelsolr.blogspot.com/


On Thu, May 14, 2020 at 6:24 PM Nightingale, Jonathan A (US) < 
jonathan.nighting...@baesystems.com> wrote:

> I'm looking on master on git hub, the solrj tests assume never use 
> aliases Just as an example. that’s all over the place in the tests
>
>
> https://github.com/apache/lucene-solr/blob/master/solr/solrj/src/test/
> org/apache/solr/client/solrj/io/stream/StreamDecoratorTest.java
>
> @Test
>   public void testTerminatingDaemonStream() throws Exception {
> Assume.assumeTrue(!useAlias);
>
> -Original Message-
> From: Joel Bernstein 
> Sent: Wednesday, May 13, 2020 1:11 PM
> To: solr-user@lucene.apache.org
> Subject: Re: using aliases in topic stream
>
> *** WARNING ***
> EXTERNAL EMAIL -- This message originates from outside our organization.
>
>
> What version of Solr are you using? The topic stream in master seems 
> to have the code in place to query aliases.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> On Wed, May 13, 2020 at 12:33 PM Nightingale, Jonathan A (US) < 
> jonathan.nighting...@baesystems.com> wrote:
>
> > Hi Everyone,
> >
> > I'm trying to run this stream and I get the following error
> >
> > topic(topics,collection1,
> > q="classes:GXP/INDEX",fl="uuid",id="feed-8",initialCheckpoint=0,chec
> > kp
> > ointEvery=-1)
> >
> > {
> >   "result-set": {
> > "docs": [
> >   {
> > "EXCEPTION": "Slices not found for collection1",
> > "EOF": true,
> > "RESPONSE_TIME": 6
> >   }
> > ]
> >   }
> > }
> >
> > "collection1" is an alias. I can search using the alias perfectly 
> > fine. In fact the search stream operation works fine with the alias.
> > It's just this topic one I've seen so far. Does anyone know why this is?
> >
> > Thanks!
> > Jonathan Nightingale
> >
> >
>

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

2020-05-16 Thread Mikhail Khludnev

It seems this thread is doing heavy work, mind the bottom line.

202.8013ms
124.8008ms
qtp153245266-156 (156)
org.apache.lucene.search.similarities.BM25Similarity$BM25Scorer.(BM25Similarity.java:219)
org.apache.lucene.search.similarities.BM25Similarity.scorer(BM25Similarity.java:192)
org.apache.lucene.search.similarities.PerFieldSimilarityWrapper.scorer(PerFieldSimilarityWrapper.java:47)
org.apache.lucene.search.TermQuery$TermWeight.(TermQuery.java:74)
org.apache.lucene.search.TermQuery.createWeight(TermQuery.java:205)
org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:726)
org.apache.lucene.search.BooleanWeight.(BooleanWeight.java:63)
org.apache.lucene.search.BooleanQuery.createWeight(BooleanQuery.java:231)
org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:726)
org.apache.lucene.search.TopFieldCollector.populateScores(TopFieldCollector.java:531)
org.apache.solr.search.grouping.distributed.command.TopGroupsFieldCommand.postCollect(TopGroupsFieldCommand.java:178)
org.apache.solr.search.grouping.CommandHandler.execute(CommandHandler.java:168)
org.apache.solr.handler.component.QueryComponent.doProcessGroupedDistributedSearchSecondPhase(QueryComponent.java:1403)
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:387)
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:328)
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:211)
org.apache.solr.core.SolrCore.execute(SolrCore.java:2596)


It seems like it ranks groups by query score, that doubtful thing to do.

>From the log. Here's how to recognize query running 25 sec "QTime=25063"


Query itself q=+msg_id:(10519539+10519540+10523575+10523576+ ... is
not what search engines are made for. They are purposed for short
query.

You may

1. leverage {!terms} query parser which might handle such long terms
list more efficiently

2. make sure you don't enable unnecessary grouping features, eg group
ranking in the stack above makes no sense for this kind of query


It's worth to revamp an overall approach in favor of query time
{!join} or index time join see {!parent}/nested docs.



On Sat, May 16, 2020 at 1:46 PM vishal patel 
wrote:

> Thanks for reply.
>
> I have taken a thread dump at the time of query execution. I do not know
> the thread name so send the All threads. I have also send the logs so you
> can get idea.
>
> Thread Dump All Stack Trace:
> https://drive.google.com/file/d/1N4rVXJoaAwNvPIY2aw57gKA9mb4vRTMR/view
> Solr 8.3 shard 1 log:
> https://drive.google.com/file/d/1h5d_eZfQvYET7JKzbNKZwhZ_RmaX7hWf/view
> Solr 8.3 shard 2 log:
> https://drive.google.com/file/d/19CRflzQ7n5BZBNaaC7EFszgzKKlPfIVl/view
>
> I have some questions regarding the thread dump
> - How can I know the my thread name from thread dump? can I get from the
> log?
> - When do I take a thread dump? on query execution or after query
> execution?
>
> Note: I got a thread name from log and checked in thread dump on query
> execution time and after query executed. Both time thread stack trace got
> different.
>
> If any other things are required then let me know I will send.
>
> Regards,
> Vishal Patel
> 
> From: Mikhail Khludnev 
> Sent: Saturday, May 16, 2020 2:23 PM
> To: solr-user 
> Subject: Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1
>
> Can you check Thread Dump in Solr Admin while Solr 8.3 crunches query for
> 34 seconds? Please share the deepest thread stack. This might give a clue
> what's going on there.
>
> On Sat, May 16, 2020 at 11:46 AM vishal patel <
> vishalpatel200...@outlook.com>
> wrote:
>
> > Any one is looking my issue? Please help me.
> >
> > Sent from Outlook
> > 
> > From: vishal patel 
> > Sent: Friday, May 15, 2020 3:06 PM
> > To: solr-user@lucene.apache.org 
> > Subject: Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1
> >
> > I have result of query debug for both version so It will helpful.
> >
> > Solr 6.1 query debug URL
> > https://drive.google.com/file/d/1ixqpgAXsVLDZA-aUobJLrMOOefZX2NL1/view
> > Solr 8.3.1 query debug URL
> > https://drive.google.com/file/d/1MOKVE-iPZFuzRnDZhY9V6OsAKFT38U5r/view
> >
> > I indexed same data in both version.
> >
> > I found score=1.0 in result of Solr 8.3.0 and score=0.016147947 in result
> > of Solr 8.6.1. Is there any impact of score in query execution? why is
> > score=1.0 in result of Solr 8.3.0?
> >
> > Regards,
> > Vishal Patel
> > 
> > From: vishal patel 
> > Sent: Thursday, May 14, 2020 7:39 PM
> > To: solr-user@lucene.apache.org 
> > Subject: Performance issue in Query execution in Solr 8.3.0 and 8.5.1
> >
> > I am upgrading Solr 6.1.0 to Solr 8.3.0 or Solr 8.5.1.
> >
> > I get performance issue for query execution in Solr 8.3.0 or Solr 8.5.1
> > when values of one field is large in query and group field is apply.
> >
> > My Solr

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

2020-05-16 Thread vishal patel

Thanks for reply.

I have taken a thread dump at the time of query execution. I do not know the 
thread name so send the All threads. I have also send the logs so you can get 
idea.

Thread Dump All Stack Trace:
https://drive.google.com/file/d/1N4rVXJoaAwNvPIY2aw57gKA9mb4vRTMR/view
Solr 8.3 shard 1 log:
https://drive.google.com/file/d/1h5d_eZfQvYET7JKzbNKZwhZ_RmaX7hWf/view
Solr 8.3 shard 2 log:
https://drive.google.com/file/d/19CRflzQ7n5BZBNaaC7EFszgzKKlPfIVl/view

I have some questions regarding the thread dump
- How can I know the my thread name from thread dump? can I get from the log?
- When do I take a thread dump? on query execution or after query execution?

Note: I got a thread name from log and checked in thread dump on query 
execution time and after query executed. Both time thread stack trace got 
different.

If any other things are required then let me know I will send.

Regards,
Vishal Patel

From: Mikhail Khludnev 
Sent: Saturday, May 16, 2020 2:23 PM
To: solr-user 
Subject: Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

Can you check Thread Dump in Solr Admin while Solr 8.3 crunches query for
34 seconds? Please share the deepest thread stack. This might give a clue
what's going on there.

On Sat, May 16, 2020 at 11:46 AM vishal patel 
wrote:

> Any one is looking my issue? Please help me.
>
> Sent from Outlook
> 
> From: vishal patel 
> Sent: Friday, May 15, 2020 3:06 PM
> To: solr-user@lucene.apache.org 
> Subject: Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1
>
> I have result of query debug for both version so It will helpful.
>
> Solr 6.1 query debug URL
> https://drive.google.com/file/d/1ixqpgAXsVLDZA-aUobJLrMOOefZX2NL1/view
> Solr 8.3.1 query debug URL
> https://drive.google.com/file/d/1MOKVE-iPZFuzRnDZhY9V6OsAKFT38U5r/view
>
> I indexed same data in both version.
>
> I found score=1.0 in result of Solr 8.3.0 and score=0.016147947 in result
> of Solr 8.6.1. Is there any impact of score in query execution? why is
> score=1.0 in result of Solr 8.3.0?
>
> Regards,
> Vishal Patel
> 
> From: vishal patel 
> Sent: Thursday, May 14, 2020 7:39 PM
> To: solr-user@lucene.apache.org 
> Subject: Performance issue in Query execution in Solr 8.3.0 and 8.5.1
>
> I am upgrading Solr 6.1.0 to Solr 8.3.0 or Solr 8.5.1.
>
> I get performance issue for query execution in Solr 8.3.0 or Solr 8.5.1
> when values of one field is large in query and group field is apply.
>
> My Solr URL :
> https://drive.google.com/file/d/1UqFE8I6M451Z1wWAu5_C1dzqYEOGjuH2/view
> My Solr config and schema :
> https://drive.google.com/drive/folders/1pJBxL0OOwAJSEC5uK_87ikaHEVGdDEEn<
> https://drive.google.com/drive/folders/1pJBxL0OOwAJSEC5uK_87ikaHEVGdDEEn>
>
> It takes 34 seconds in Solr 8.3.0 or Solr 8.5.1. Same URL takes 1.5
> seconds in Solr 6.1.0.
>
> Is there any changes or issue related to grouping in Solr 8.3.0 or 8.5.1?
>
>
> Regards,
> Vishal Patel
>
>

--
Sincerely yours
Mikhail Khludnev

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

2020-05-16 Thread Mikhail Khludnev

Can you check Thread Dump in Solr Admin while Solr 8.3 crunches query for
34 seconds? Please share the deepest thread stack. This might give a clue
what's going on there.

On Sat, May 16, 2020 at 11:46 AM vishal patel 
wrote:

> Any one is looking my issue? Please help me.
>
> Sent from Outlook
> 
> From: vishal patel 
> Sent: Friday, May 15, 2020 3:06 PM
> To: solr-user@lucene.apache.org 
> Subject: Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1
>
> I have result of query debug for both version so It will helpful.
>
> Solr 6.1 query debug URL
> https://drive.google.com/file/d/1ixqpgAXsVLDZA-aUobJLrMOOefZX2NL1/view
> Solr 8.3.1 query debug URL
> https://drive.google.com/file/d/1MOKVE-iPZFuzRnDZhY9V6OsAKFT38U5r/view
>
> I indexed same data in both version.
>
> I found score=1.0 in result of Solr 8.3.0 and score=0.016147947 in result
> of Solr 8.6.1. Is there any impact of score in query execution? why is
> score=1.0 in result of Solr 8.3.0?
>
> Regards,
> Vishal Patel
> 
> From: vishal patel 
> Sent: Thursday, May 14, 2020 7:39 PM
> To: solr-user@lucene.apache.org 
> Subject: Performance issue in Query execution in Solr 8.3.0 and 8.5.1
>
> I am upgrading Solr 6.1.0 to Solr 8.3.0 or Solr 8.5.1.
>
> I get performance issue for query execution in Solr 8.3.0 or Solr 8.5.1
> when values of one field is large in query and group field is apply.
>
> My Solr URL :
> https://drive.google.com/file/d/1UqFE8I6M451Z1wWAu5_C1dzqYEOGjuH2/view
> My Solr config and schema :
> https://drive.google.com/drive/folders/1pJBxL0OOwAJSEC5uK_87ikaHEVGdDEEn<
> https://drive.google.com/drive/folders/1pJBxL0OOwAJSEC5uK_87ikaHEVGdDEEn>
>
> It takes 34 seconds in Solr 8.3.0 or Solr 8.5.1. Same URL takes 1.5
> seconds in Solr 6.1.0.
>
> Is there any changes or issue related to grouping in Solr 8.3.0 or 8.5.1?
>
>
> Regards,
> Vishal Patel
>
>

-- 
Sincerely yours
Mikhail Khludnev

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

2020-05-16 Thread vishal patel

Any one is looking my issue? Please help me.

Sent from Outlook

From: vishal patel 
Sent: Friday, May 15, 2020 3:06 PM
To: solr-user@lucene.apache.org 
Subject: Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

I have result of query debug for both version so It will helpful.

Solr 6.1 query debug URL
https://drive.google.com/file/d/1ixqpgAXsVLDZA-aUobJLrMOOefZX2NL1/view
Solr 8.3.1 query debug URL
https://drive.google.com/file/d/1MOKVE-iPZFuzRnDZhY9V6OsAKFT38U5r/view

I indexed same data in both version.

I found score=1.0 in result of Solr 8.3.0 and score=0.016147947 in result of 
Solr 8.6.1. Is there any impact of score in query execution? why is score=1.0 
in result of Solr 8.3.0?

Regards,
Vishal Patel

From: vishal patel 
Sent: Thursday, May 14, 2020 7:39 PM
To: solr-user@lucene.apache.org 
Subject: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

I am upgrading Solr 6.1.0 to Solr 8.3.0 or Solr 8.5.1.

I get performance issue for query execution in Solr 8.3.0 or Solr 8.5.1 when 
values of one field is large in query and group field is apply.

My Solr URL : 
https://drive.google.com/file/d/1UqFE8I6M451Z1wWAu5_C1dzqYEOGjuH2/view
My Solr config and schema : 
https://drive.google.com/drive/folders/1pJBxL0OOwAJSEC5uK_87ikaHEVGdDEEn

It takes 34 seconds in Solr 8.3.0 or Solr 8.5.1. Same URL takes 1.5 seconds in 
Solr 6.1.0.

Is there any changes or issue related to grouping in Solr 8.3.0 or 8.5.1?

Regards,
Vishal Patel

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

Re: using aliases in topic stream

Re: Unbalanced shard requests

Re: What is the logical order of applying sorts in SOLR?

Re: What is the logical order of applying sorts in SOLR?

RE: using aliases in topic stream

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

Re: Performance issue in Query execution in Solr 8.3.0 and 8.5.1

10 matches

Site Navigation

Mail list logo

Footer information