Re: How to check when a search exceeds the threshold of timeAllowed parameter
Great I must have missed that. On Wed, Dec 23, 2015 at 9:41 AM, Jeff Warteswrote: > Looks like it’ll set partialResults=true on your results if you hit the > timeout. > > https://issues.apache.org/jira/browse/SOLR-502 > > https://issues.apache.org/jira/browse/SOLR-5986 > > > > > > > On 12/22/15, 5:43 PM, "Vincenzo D'Amore" wrote: > > >Well... I can write everything, but really all this just to understand > >when timeAllowed > >parameter trigger a partial answer? I mean, isn't there anything set in > the > >response when is partial? > > > >On Wed, Dec 23, 2015 at 2:38 AM, Walter Underwood > >wrote: > > > >> We need to know a LOT more about your site. Number of documents, size of > >> index, frequency of updates, length of queries approximate size of > server > >> (CPUs, RAM, type of disk), version of Solr, version of Java, and > features > >> you are using (faceting, highlighting, etc.). > >> > >> After that, we’ll have more questions. > >> > >> wunder > >> Walter Underwood > >> wun...@wunderwood.org > >> http://observer.wunderwood.org/ (my blog) > >> > >> > >> > On Dec 22, 2015, at 4:58 PM, Vincenzo D'Amore > >> wrote: > >> > > >> > Hi All, > >> > > >> > my website is under pressure, there is a big number of concurrent > >> searches. > >> > When the connected users are too many, the searches becomes so slow > that > >> in > >> > some cases users have to wait many seconds. > >> > The queue of searches becomes so long that, in same cases, servers are > >> > blocked trying to serve all these requests. > >> > As far as I know because some searches are very expensive, and when > many > >> > expensive searches clog the queue server becomes unresponsive. > >> > > >> > In order to quickly workaround this herd effect, I have added a > >> > default timeAllowed to 15 seconds, and this seems help a lot. > >> > > >> > But during stress tests but I'm unable to understand when and what > >> requests > >> > are affected by timeAllowed parameter. > >> > > >> > Just be clear, I have configure timeAllowed parameter in a SolrCloud > >> > environment, given that partial results may be returned (if there are > >> any), > >> > how can I know when this happens? When the timeAllowed parameter > trigger > >> a > >> > partial answer? > >> > > >> > Best regards, > >> > Vincenzo > >> > > >> > > >> > > >> > -- > >> > Vincenzo D'Amore > >> > email: v.dam...@gmail.com > >> > skype: free.dev > >> > mobile: +39 349 8513251 > >> > >> > > > > > >-- > >Vincenzo D'Amore > >email: v.dam...@gmail.com > >skype: free.dev > >mobile: +39 349 8513251 > -- Bill Bell billnb...@gmail.com cell 720-256-8076
Re: How to check when a search exceeds the threshold of timeAllowed parameter
Looks like it’ll set partialResults=true on your results if you hit the timeout. https://issues.apache.org/jira/browse/SOLR-502 https://issues.apache.org/jira/browse/SOLR-5986 On 12/22/15, 5:43 PM, "Vincenzo D'Amore"wrote: >Well... I can write everything, but really all this just to understand >when timeAllowed >parameter trigger a partial answer? I mean, isn't there anything set in the >response when is partial? > >On Wed, Dec 23, 2015 at 2:38 AM, Walter Underwood >wrote: > >> We need to know a LOT more about your site. Number of documents, size of >> index, frequency of updates, length of queries approximate size of server >> (CPUs, RAM, type of disk), version of Solr, version of Java, and features >> you are using (faceting, highlighting, etc.). >> >> After that, we’ll have more questions. >> >> wunder >> Walter Underwood >> wun...@wunderwood.org >> http://observer.wunderwood.org/ (my blog) >> >> >> > On Dec 22, 2015, at 4:58 PM, Vincenzo D'Amore >> wrote: >> > >> > Hi All, >> > >> > my website is under pressure, there is a big number of concurrent >> searches. >> > When the connected users are too many, the searches becomes so slow that >> in >> > some cases users have to wait many seconds. >> > The queue of searches becomes so long that, in same cases, servers are >> > blocked trying to serve all these requests. >> > As far as I know because some searches are very expensive, and when many >> > expensive searches clog the queue server becomes unresponsive. >> > >> > In order to quickly workaround this herd effect, I have added a >> > default timeAllowed to 15 seconds, and this seems help a lot. >> > >> > But during stress tests but I'm unable to understand when and what >> requests >> > are affected by timeAllowed parameter. >> > >> > Just be clear, I have configure timeAllowed parameter in a SolrCloud >> > environment, given that partial results may be returned (if there are >> any), >> > how can I know when this happens? When the timeAllowed parameter trigger >> a >> > partial answer? >> > >> > Best regards, >> > Vincenzo >> > >> > >> > >> > -- >> > Vincenzo D'Amore >> > email: v.dam...@gmail.com >> > skype: free.dev >> > mobile: +39 349 8513251 >> >> > > >-- >Vincenzo D'Amore >email: v.dam...@gmail.com >skype: free.dev >mobile: +39 349 8513251
Re: How to check when a search exceeds the threshold of timeAllowed parameter
On 12/23/2015 01:42 AM, William Bell wrote: I agree that when using timeAllowed in the header info there should be an entry that indicates timeAllowed triggered. If I'm not mistaken, there is => partialResults:true "responseHeader":{ "partialResults":true } // This is the only reason why we have not used timeAllowed. So this is a great suggestion. Something like: 1 ?? That would be great. 0 1 107 *:* 1000 On Tue, Dec 22, 2015 at 6:43 PM, Vincenzo D'Amorewrote: Well... I can write everything, but really all this just to understand when timeAllowed parameter trigger a partial answer? I mean, isn't there anything set in the response when is partial? On Wed, Dec 23, 2015 at 2:38 AM, Walter Underwood wrote: We need to know a LOT more about your site. Number of documents, size of index, frequency of updates, length of queries approximate size of server (CPUs, RAM, type of disk), version of Solr, version of Java, and features you are using (faceting, highlighting, etc.). After that, we’ll have more questions. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) On Dec 22, 2015, at 4:58 PM, Vincenzo D'Amore wrote: Hi All, my website is under pressure, there is a big number of concurrent searches. When the connected users are too many, the searches becomes so slow that in some cases users have to wait many seconds. The queue of searches becomes so long that, in same cases, servers are blocked trying to serve all these requests. As far as I know because some searches are very expensive, and when many expensive searches clog the queue server becomes unresponsive. In order to quickly workaround this herd effect, I have added a default timeAllowed to 15 seconds, and this seems help a lot. But during stress tests but I'm unable to understand when and what requests are affected by timeAllowed parameter. Just be clear, I have configure timeAllowed parameter in a SolrCloud environment, given that partial results may be returned (if there are any), how can I know when this happens? When the timeAllowed parameter trigger a partial answer? Best regards, Vincenzo -- Vincenzo D'Amore email: v.dam...@gmail.com skype: free.dev mobile: +39 349 8513251 -- Vincenzo D'Amore email: v.dam...@gmail.com skype: free.dev mobile: +39 349 8513251
Re: How to check when a search exceeds the threshold of timeAllowed parameter
We need to know a LOT more about your site. Number of documents, size of index, frequency of updates, length of queries approximate size of server (CPUs, RAM, type of disk), version of Solr, version of Java, and features you are using (faceting, highlighting, etc.). After that, we’ll have more questions. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Dec 22, 2015, at 4:58 PM, Vincenzo D'Amorewrote: > > Hi All, > > my website is under pressure, there is a big number of concurrent searches. > When the connected users are too many, the searches becomes so slow that in > some cases users have to wait many seconds. > The queue of searches becomes so long that, in same cases, servers are > blocked trying to serve all these requests. > As far as I know because some searches are very expensive, and when many > expensive searches clog the queue server becomes unresponsive. > > In order to quickly workaround this herd effect, I have added a > default timeAllowed to 15 seconds, and this seems help a lot. > > But during stress tests but I'm unable to understand when and what requests > are affected by timeAllowed parameter. > > Just be clear, I have configure timeAllowed parameter in a SolrCloud > environment, given that partial results may be returned (if there are any), > how can I know when this happens? When the timeAllowed parameter trigger a > partial answer? > > Best regards, > Vincenzo > > > > -- > Vincenzo D'Amore > email: v.dam...@gmail.com > skype: free.dev > mobile: +39 349 8513251
How to check when a search exceeds the threshold of timeAllowed parameter
Hi All, my website is under pressure, there is a big number of concurrent searches. When the connected users are too many, the searches becomes so slow that in some cases users have to wait many seconds. The queue of searches becomes so long that, in same cases, servers are blocked trying to serve all these requests. As far as I know because some searches are very expensive, and when many expensive searches clog the queue server becomes unresponsive. In order to quickly workaround this herd effect, I have added a default timeAllowed to 15 seconds, and this seems help a lot. But during stress tests but I'm unable to understand when and what requests are affected by timeAllowed parameter. Just be clear, I have configure timeAllowed parameter in a SolrCloud environment, given that partial results may be returned (if there are any), how can I know when this happens? When the timeAllowed parameter trigger a partial answer? Best regards, Vincenzo -- Vincenzo D'Amore email: v.dam...@gmail.com skype: free.dev mobile: +39 349 8513251
Re: How to check when a search exceeds the threshold of timeAllowed parameter
Well... I can write everything, but really all this just to understand when timeAllowed parameter trigger a partial answer? I mean, isn't there anything set in the response when is partial? On Wed, Dec 23, 2015 at 2:38 AM, Walter Underwoodwrote: > We need to know a LOT more about your site. Number of documents, size of > index, frequency of updates, length of queries approximate size of server > (CPUs, RAM, type of disk), version of Solr, version of Java, and features > you are using (faceting, highlighting, etc.). > > After that, we’ll have more questions. > > wunder > Walter Underwood > wun...@wunderwood.org > http://observer.wunderwood.org/ (my blog) > > > > On Dec 22, 2015, at 4:58 PM, Vincenzo D'Amore > wrote: > > > > Hi All, > > > > my website is under pressure, there is a big number of concurrent > searches. > > When the connected users are too many, the searches becomes so slow that > in > > some cases users have to wait many seconds. > > The queue of searches becomes so long that, in same cases, servers are > > blocked trying to serve all these requests. > > As far as I know because some searches are very expensive, and when many > > expensive searches clog the queue server becomes unresponsive. > > > > In order to quickly workaround this herd effect, I have added a > > default timeAllowed to 15 seconds, and this seems help a lot. > > > > But during stress tests but I'm unable to understand when and what > requests > > are affected by timeAllowed parameter. > > > > Just be clear, I have configure timeAllowed parameter in a SolrCloud > > environment, given that partial results may be returned (if there are > any), > > how can I know when this happens? When the timeAllowed parameter trigger > a > > partial answer? > > > > Best regards, > > Vincenzo > > > > > > > > -- > > Vincenzo D'Amore > > email: v.dam...@gmail.com > > skype: free.dev > > mobile: +39 349 8513251 > > -- Vincenzo D'Amore email: v.dam...@gmail.com skype: free.dev mobile: +39 349 8513251
Re: How to check when a search exceeds the threshold of timeAllowed parameter
I agree that when using timeAllowed in the header info there should be an entry that indicates timeAllowed triggered. This is the only reason why we have not used timeAllowed. So this is a great suggestion. Something like: 1 ?? That would be great. 0 1 107 *:* 1000 On Tue, Dec 22, 2015 at 6:43 PM, Vincenzo D'Amorewrote: > Well... I can write everything, but really all this just to understand > when timeAllowed > parameter trigger a partial answer? I mean, isn't there anything set in the > response when is partial? > > On Wed, Dec 23, 2015 at 2:38 AM, Walter Underwood > wrote: > > > We need to know a LOT more about your site. Number of documents, size of > > index, frequency of updates, length of queries approximate size of server > > (CPUs, RAM, type of disk), version of Solr, version of Java, and features > > you are using (faceting, highlighting, etc.). > > > > After that, we’ll have more questions. > > > > wunder > > Walter Underwood > > wun...@wunderwood.org > > http://observer.wunderwood.org/ (my blog) > > > > > > > On Dec 22, 2015, at 4:58 PM, Vincenzo D'Amore > > wrote: > > > > > > Hi All, > > > > > > my website is under pressure, there is a big number of concurrent > > searches. > > > When the connected users are too many, the searches becomes so slow > that > > in > > > some cases users have to wait many seconds. > > > The queue of searches becomes so long that, in same cases, servers are > > > blocked trying to serve all these requests. > > > As far as I know because some searches are very expensive, and when > many > > > expensive searches clog the queue server becomes unresponsive. > > > > > > In order to quickly workaround this herd effect, I have added a > > > default timeAllowed to 15 seconds, and this seems help a lot. > > > > > > But during stress tests but I'm unable to understand when and what > > requests > > > are affected by timeAllowed parameter. > > > > > > Just be clear, I have configure timeAllowed parameter in a SolrCloud > > > environment, given that partial results may be returned (if there are > > any), > > > how can I know when this happens? When the timeAllowed parameter > trigger > > a > > > partial answer? > > > > > > Best regards, > > > Vincenzo > > > > > > > > > > > > -- > > > Vincenzo D'Amore > > > email: v.dam...@gmail.com > > > skype: free.dev > > > mobile: +39 349 8513251 > > > > > > > -- > Vincenzo D'Amore > email: v.dam...@gmail.com > skype: free.dev > mobile: +39 349 8513251 > -- Bill Bell billnb...@gmail.com cell 720-256-8076
Re: How to check when a search exceeds the threshold of timeAllowed parameter
timeAllowed was designed to handle queries that by themselves consume lots of resources, not to try to handle situations with large numbers of requests that starve other requests from accessing CPU and I/O resources. The usual technique for handling large numbers of requests is replication, making more copies of the index that can each be searched in parallel. How long do queries take when the site is operating normally? Make sure that you have enough system memory to cache the index, otherwise the machine wish be thrashing with lots of I/O for competing requests. -- Jack Krupansky On Tue, Dec 22, 2015 at 8:43 PM, Vincenzo D'Amorewrote: > Well... I can write everything, but really all this just to understand > when timeAllowed > parameter trigger a partial answer? I mean, isn't there anything set in the > response when is partial? > > On Wed, Dec 23, 2015 at 2:38 AM, Walter Underwood > wrote: > > > We need to know a LOT more about your site. Number of documents, size of > > index, frequency of updates, length of queries approximate size of server > > (CPUs, RAM, type of disk), version of Solr, version of Java, and features > > you are using (faceting, highlighting, etc.). > > > > After that, we’ll have more questions. > > > > wunder > > Walter Underwood > > wun...@wunderwood.org > > http://observer.wunderwood.org/ (my blog) > > > > > > > On Dec 22, 2015, at 4:58 PM, Vincenzo D'Amore > > wrote: > > > > > > Hi All, > > > > > > my website is under pressure, there is a big number of concurrent > > searches. > > > When the connected users are too many, the searches becomes so slow > that > > in > > > some cases users have to wait many seconds. > > > The queue of searches becomes so long that, in same cases, servers are > > > blocked trying to serve all these requests. > > > As far as I know because some searches are very expensive, and when > many > > > expensive searches clog the queue server becomes unresponsive. > > > > > > In order to quickly workaround this herd effect, I have added a > > > default timeAllowed to 15 seconds, and this seems help a lot. > > > > > > But during stress tests but I'm unable to understand when and what > > requests > > > are affected by timeAllowed parameter. > > > > > > Just be clear, I have configure timeAllowed parameter in a SolrCloud > > > environment, given that partial results may be returned (if there are > > any), > > > how can I know when this happens? When the timeAllowed parameter > trigger > > a > > > partial answer? > > > > > > Best regards, > > > Vincenzo > > > > > > > > > > > > -- > > > Vincenzo D'Amore > > > email: v.dam...@gmail.com > > > skype: free.dev > > > mobile: +39 349 8513251 > > > > > > > -- > Vincenzo D'Amore > email: v.dam...@gmail.com > skype: free.dev > mobile: +39 349 8513251 >