Re: Unable to write response, client closed connection or we are shutting down

[Nawab Zada Asad Iqbal]

So, I tried few things and it seems like there are more page faults after
the solr6 upgrade. Even when there is no update or query activity (Except
the periodic commit), the pagefaults are little higher than they used to be
.


Any suggestions in this area?

Thanks
Nawab

On Tue, Aug 15, 2017 at 4:09 PM, Nawab Zada Asad Iqbal 
wrote:

> Hi Rick
>
> My software is not very sophisticated. I have picked some queries from
> production logs, which I am replaying against this solr installation. It is
> not a SolrCloud but i specify "shards="  in the query to gather results
> from all shards.
>
> I found some values to tweak e.g.
> 1500
> 15
>
> After doing this, the "Unable to write response, client closed connection
> or we are shutting down" error is mostly gone.
>
> However, the query perf is bad. The server is not using all the assigned
> memory. However CPU usage is reaching 80%.
> The query response time is 50+ times worse (e.g., 1400 msec vs 20 msec for
> 75th percentile) .
> What can I do to use more memory and hopefully alleviate some of this bad
> performance?
>
> My cache settings are identical to older setup.
>
>
> Thanks
> Nawab
>
>
>
>
>
> On Mon, Aug 14, 2017 at 9:01 AM, Rick Leir  wrote:
>
>> Nawab
>> What test software do you use? What else is happening when the exception
>> occurs?
>> Cheers -- Rick
>>
>> On August 12, 2017 1:48:19 PM EDT, Nawab Zada Asad Iqbal <
>> khi...@gmail.com> wrote:
>> >Hi,
>> >
>> >I am executing a query performance test against my solr 6.6 setup and I
>> >noticed following exception every now and then. What do I need to do?
>> >
>> >Aug 11, 2017 08:40:07 AM INFO  (qtp761960786-250) [   x:filesearch]
>> >o.a.s.s.HttpSolrCall Unable to write response, client closed connection
>> >or
>> >we are shutting down
>> >org.eclipse.jetty.io.EofException
>> >at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:199)
>> >at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:420)
>> >at org.eclipse.jetty.io.WriteFlusher.completeWrite(
>> >WriteFlusher.java:375)
>> >at org.eclipse.jetty.io.SelectChannelEndPoint$3.run(
>> >SelectChannelEndPoint.java:107)
>> >at org.eclipse.jetty.io.SelectChannelEndPoint.onSelected(
>> >SelectChannelEndPoint.java:193)
>> >at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.
>> >processSelected(ManagedSelector.java:283)
>> >at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(
>> >ManagedSelector.java:181)
>> >at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.
>> >executeProduceConsume(ExecuteProduceConsume.java:249)
>> >at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.
>> >produceConsume(ExecuteProduceConsume.java:148)
>> >   at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(
>> >ExecuteProduceConsume.java:136)
>> >at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
>> >QueuedThreadPool.java:671)
>> >at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
>> >QueuedThreadPool.java:589)
>> >at java.lang.Thread.run(Thread.java:748)
>> >Caused by: java.io.IOException: Broken pipe
>> >at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
>> >at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
>> >at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
>> >at sun.nio.ch.IOUtil.write(IOUtil.java:51)
>> >at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
>> >at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:177)
>> >
>> >
>> >
>> >Apart from that, I also noticed that the query response time is longer
>> >than
>> >I expected, while the memory utilization stays <= 35%. I thought that
>> >somewhere I have set maxThreads (Jetty) to a very low number, however I
>> >am
>> >falling back on default which is 1 (so that shouldn't be a
>> >problem).
>> >
>> >
>> >Thanks
>> >Nawab
>>
>> --
>> Sorry for being brief. Alternate email is rickleir at yahoo dot com
>
>
>

Re: Help with Query/Function for conditional boost

[Ahmet Arslan]

Hi Shamik,

I belive 5-args map function can be used here. Here is a link which may inspire 
you.
http://www.cominvent.com/2012/01/25/super-flexible-autocomplete-with-solr/

Ahmet





On Wednesday, August 16, 2017, 11:06:28 PM GMT+3, Shamik Bandopadhyay 
 wrote:


Hi,

  I'm trying to create a function to boost dynamically boost a field based
on specific values for a searchable field. Here's an example:

I've the following query fields with default boost.

qf=text^2 title^4 command^8

Also, there's a default boost on the source field.

bq=source:help^10 source:forum^5

Among the searchable fields, command gets the highest preference. To add to
that,I would like to see boost results from source help further when a
query term exists in command field. With my current setting, documents from
forum are appearing at the top when a search term is found in command
field. Increasing the boost to source:help didn't make any difference.

Just wondering if it's possible to write a function which will
conditionally boost command field for documents tagged with source=help

if(termfreq(source,'help'), command^8, command^1)

The above function is just for reference to show what I'm trying to achieve.

Any pointers will be helpful.

Thanks,
Shamik

Help with Query/Function for conditional boost

[Shamik Bandopadhyay]

Hi,

   I'm trying to create a function to boost dynamically boost a field based
on specific values for a searchable field. Here's an example:

I've the following query fields with default boost.

qf=text^2 title^4 command^8

Also, there's a default boost on the source field.

bq=source:help^10 source:forum^5

Among the searchable fields, command gets the highest preference. To add to
that,I would like to see boost results from source help further when a
query term exists in command field. With my current setting, documents from
forum are appearing at the top when a search term is found in command
field. Increasing the boost to source:help didn't make any difference.

Just wondering if it's possible to write a function which will
conditionally boost command field for documents tagged with source=help

if(termfreq(source,'help'), command^8, command^1)

The above function is just for reference to show what I'm trying to achieve.

Any pointers will be helpful.

Thanks,
Shamik

entities, keepwords, and confusion

[John Blythe]

hi all,

i'm attempting to define some entities in a keepwords file that a group of
fields will use as other fields are copied into them.

my index analyzer is defined with a KeyWord tokenizer followed by the
Keepword filter and a LowerCase for the finale.

my query analyzer uses WhiteSpace for the tokenizer and then users a
shingling and lowercase filter.

if i use the phrase `Big Mac` i'll get what i expect in the Analysis tool.
if i actually query for "Big Mac" (enquoted in the query), however, it
produces this

"parsedquery":"(+DisjunctionMaxQuery((entiites_vendors:big mac)))/no_coord",
> "parsedquery_toString":"+(entiites_vendors:big mac)",



wrapping the phrase parenthetically gives the following:

> "parsedquery":"(+(DisjunctionMaxQuery((entities_vendors:(big))
> DisjunctionMaxQuery((entities_vendors:mac ())/no_coord",
> "parsedquery_toString":"+((entities_vendors:(big)
> (entities_vendors:mac))) ()",



fwiw, i did double check via facets and a *:* search that my big mac was in
the data set to begin with.

i think there is a foundational category or concept i'm missing when
approaching some problems like this. i had an issue earlier in the year
that i never resolved that dealt with synonyms but was surrounded by much
of the same confusion. hopefully i can get less n00by w some help in
figuring this out.

thanks-

Stack Overflow in FuzzyLookupFactory

[Walter Underwood]

Just got a stack overflow in the Lucene automata code. Is there a way to save 
out the FSA for a bug report?

This is in 6.5.1, so it may be related to 
https://issues.apache.org/jira/browse/SOLR-9458 


wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

Re: Breaking down QTime for debugging performance

[Erick Erickson]

Try adding =true to the URL.

On Tue, Aug 15, 2017 at 5:18 PM, Nawab Zada Asad Iqbal  wrote:
> Hi all
>
> For a given solr host and shard, is there any way to get a breakdown on
> QTime to see where is the time being spent?
>
>
> Thanks
> Nawab

Re: Issue with delta import

[bhargava ravali koganti]

I'm getting error like this

Exception while processing: test document : SolrInputDocument(fields:
[]):org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
execute query: SELECT * FROM person WHERE 'false' != 'false' OR
last_modified > '2017-08-09 15:48:02' Processing Document # 1
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:69)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.(JdbcDataSource.java:318)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:279)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:54)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:244)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:416)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:480)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:461)
Caused by: com.datastax.driver.core.exceptions.SyntaxError: line 1:27 no
viable alternative at input 'false' (SELECT * FROM person WHERE ['fals]...)
at
com.datastax.driver.core.exceptions.SyntaxError.copy(SyntaxError.java:58)
at
com.datastax.driver.core.exceptions.SyntaxError.copy(SyntaxError.java:24)
at
com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
at
com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
at
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
at
com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:43)
at
com.dbschema.CassandraPreparedStatement.executeQuery(CassandraPreparedStatement.java:73)
at
com.dbschema.CassandraPreparedStatement.execute(CassandraPreparedStatement.java:114)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.executeStatement(JdbcDataSource.java:338)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.(JdbcDataSource.java:313)
... 12 more
Caused by: com.datastax.driver.core.exceptions.SyntaxError: line 1:27 no
viable alternative at input 'false' (SELECT * FROM person WHERE ['fals]...)
at 
com.datastax.driver.core.Responses$Error.asException(Responses.java:144)
at
com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:179)
at
com.datastax.driver.core.RequestHandler.setFinalResult(RequestHandler.java:186)
at
com.datastax.driver.core.RequestHandler.access$2500(RequestHandler.java:50)
at
com.datastax.driver.core.RequestHandler$SpeculativeExecution.setFinalResult(RequestHandler.java:817)
at
com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:651)
at
com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1077)
at
com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1000)
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
at
io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
at
io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:153)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:157)
at
io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:254)
at
io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
at
io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:153)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:157)
at
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at
io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:83)
at
io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:153)
at

Re: Issue with delta import

[bhargava ravali koganti]

Does this way even handle the deletes?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Issue-with-delta-import-tp4347680p4350726.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: QueryParser changes query by itself

[Yonik Seeley]

The queryCache shouldn't be involved, this is somehow an issue in
parsing (and Solr doesn't currently cache parsing).
Perhaps there is something shared in your SynonymQParser instances
that isn't quite thread safe?
It could also be something in the text analysis in lucene as well
(related to the new graph stuff?)

-Yonik


On Wed, Aug 16, 2017 at 7:32 AM, Bernd Fehling
 wrote:
> My class SynonymQParser which calls SolrQueryParserBase.parse :
>
> class SynonymQParser extends QParser {
> protected SolrQueryParser sqparser;
> ...
> @Override
> public Query parse() throws SyntaxError {
> ...
> sqparser = new SolrQueryParser(this, defaultField);
> sqparser.setEnableGraphQueries(false);
> sqparser.setEnablePositionIncrements(false);
> ...
> Query synquery = sqparser.parse(qstr);
> ...
>
> And this is SolrQueryParserBase with method parse:
>
> public abstract class SolrQueryParserBase extends QueryBuilder {
> ...
> public Query parse(String query) throws SyntaxError {
> ReInit(new FastCharStream(new StringReader(query)));
> try {
>   // TopLevelQuery is a Query followed by the end-of-input (EOF)
>   Query res = TopLevelQuery(null);  // pass null so we can tell later 
> if an explicit field was provided or not
>   return res!=null ? res : newBooleanQuery().build();
> }
> ...
>
>
> The String variable "query" going into parse method is always 
> "textth:waffenhandel" !!!
> Having a breakpoint at "return", the Query variable "res" changes sometimes to
> TermQuery with term "textth:rss" instead of being a SynonymQuery.
>
> This is strange!!!
>
> What is ReInit right before try doing, is that a cahe lookup?
>
> Or is the problem in TopLevelQuery?
>
> Regards
> Bernd
>
>
> Am 16.08.2017 um 09:06 schrieb Bernd Fehling:
>> Hi Ahmet,
>>
>> thank you for your reply. I was also targeting towards QueryCache but
>> with your hint about LUCENE-3758 I have a better point to start with.
>>
>> If the system is under high load and the the QueryCache is filled I have
>> a higher rate of changed queries.
>> In debug mode the "timing-->process-->query" of changed queries is always 
>> "0" zero.
>>
>> The query parser "SynonymQParser" is self developed which uses QParserPlugin.
>> There is no caching inside and works for years.
>> Only compiled against recent Lucene/Solr and some modifications like
>> using Builder with newer Lucene versions.
>>
>> I will test without query cache.
>> Wich one should be disabled, Query Result Cache?
>>
>> Regards
>> Bernd
>>
>>
>> Am 15.08.2017 um 19:07 schrieb Ahmet Arslan:
>>> Hi Bernd,
>>>
>>> In LUCENE-3758, a new member field added into ComplexPhraseQuery class. But 
>>> we didn't change its hashCode method accordingly. This caused anomalies in 
>>> Solr, and Yonik found the bug and fixed hashCode. Your e-mail somehow 
>>> reminded me this.
>>> Could it be the QueryCache and hashCode method/implementation of Query 
>>> subclasses.
>>> May be your good and bad example is producing same hashCode? And this is 
>>> confusing query cache in solr?
>>> Can you disable the query cache, to test it?
>>> By the way, which query parser are you using? I believe SynonymQuery is 
>>> produced by BM25 similarity, right?
>>>
>>> Ahmet
>>>
>>>
>>> On Friday, August 11, 2017, 2:48:07 PM GMT+3, Bernd Fehling 
>>>  wrote:
>>>
>>>
>>> We just noticed a very strange problem with Solr 6.4.2 QueryParser.
>>> The QueryParser changes the query by itself from time to time.
>>> This happens if doing a search request reload several times at higher rate.
>>>
>>> Good example:
>>> ...
>>> textth:waffenhandel
>>>   
>>> ...
>>> textth:waffenhandel
>>> textth:waffenhandel
>>>   +SynonymQuery(Synonym(textth:"arms sales" 
>>> textth:"arms trade"...
>>>   +Synonym(textth:"arms sales" 
>>> textth:"arms trade"...
>>>
>>>
>>> Bad example:
>>> ...
>>> textth:waffenhandel
>>>   
>>> ...
>>> textth:waffenhandel
>>> textth:waffenhandel
>>>   +textth:rss
>>>   +textth:rss
>>>
>>> As you can see in the bad example after several reloads the parsedquery 
>>> changed to term "rss".
>>> But the original querystring has no "rss" substring at all. That is really 
>>> strange.
>>>
>>> Anyone seen this before?
>>>
>>> Single index, Solr 6.4.2.
>>>
>>> Regards
>>> Bernd
>>>

Re: QueryParser changes query by itself

[Bernd Fehling]

My class SynonymQParser which calls SolrQueryParserBase.parse :

class SynonymQParser extends QParser {
protected SolrQueryParser sqparser;
...
@Override
public Query parse() throws SyntaxError {
...
sqparser = new SolrQueryParser(this, defaultField);
sqparser.setEnableGraphQueries(false);
sqparser.setEnablePositionIncrements(false);
...
Query synquery = sqparser.parse(qstr);
...

And this is SolrQueryParserBase with method parse:

public abstract class SolrQueryParserBase extends QueryBuilder {
...
public Query parse(String query) throws SyntaxError {
ReInit(new FastCharStream(new StringReader(query)));
try {
  // TopLevelQuery is a Query followed by the end-of-input (EOF)
  Query res = TopLevelQuery(null);  // pass null so we can tell later 
if an explicit field was provided or not
  return res!=null ? res : newBooleanQuery().build();
}
...


The String variable "query" going into parse method is always 
"textth:waffenhandel" !!!
Having a breakpoint at "return", the Query variable "res" changes sometimes to
TermQuery with term "textth:rss" instead of being a SynonymQuery.

This is strange!!!

What is ReInit right before try doing, is that a cahe lookup?

Or is the problem in TopLevelQuery?

Regards
Bernd


Am 16.08.2017 um 09:06 schrieb Bernd Fehling:
> Hi Ahmet,
> 
> thank you for your reply. I was also targeting towards QueryCache but
> with your hint about LUCENE-3758 I have a better point to start with.
> 
> If the system is under high load and the the QueryCache is filled I have
> a higher rate of changed queries.
> In debug mode the "timing-->process-->query" of changed queries is always "0" 
> zero.
> 
> The query parser "SynonymQParser" is self developed which uses QParserPlugin.
> There is no caching inside and works for years.
> Only compiled against recent Lucene/Solr and some modifications like
> using Builder with newer Lucene versions.
> 
> I will test without query cache.
> Wich one should be disabled, Query Result Cache?
> 
> Regards
> Bernd
> 
> 
> Am 15.08.2017 um 19:07 schrieb Ahmet Arslan:
>> Hi Bernd,
>>
>> In LUCENE-3758, a new member field added into ComplexPhraseQuery class. But 
>> we didn't change its hashCode method accordingly. This caused anomalies in 
>> Solr, and Yonik found the bug and fixed hashCode. Your e-mail somehow 
>> reminded me this.
>> Could it be the QueryCache and hashCode method/implementation of Query 
>> subclasses.
>> May be your good and bad example is producing same hashCode? And this is 
>> confusing query cache in solr?
>> Can you disable the query cache, to test it?
>> By the way, which query parser are you using? I believe SynonymQuery is 
>> produced by BM25 similarity, right?
>>
>> Ahmet
>>
>>
>> On Friday, August 11, 2017, 2:48:07 PM GMT+3, Bernd Fehling 
>>  wrote:
>>
>>
>> We just noticed a very strange problem with Solr 6.4.2 QueryParser.
>> The QueryParser changes the query by itself from time to time.
>> This happens if doing a search request reload several times at higher rate.
>>
>> Good example:
>> ...
>> textth:waffenhandel
>>   
>> ...
>> textth:waffenhandel
>> textth:waffenhandel
>>   +SynonymQuery(Synonym(textth:"arms sales" 
>> textth:"arms trade"...
>>   +Synonym(textth:"arms sales" textth:"arms 
>> trade"...
>>
>>
>> Bad example:
>> ...
>> textth:waffenhandel
>>   
>> ...
>> textth:waffenhandel
>> textth:waffenhandel
>>   +textth:rss
>>   +textth:rss
>>
>> As you can see in the bad example after several reloads the parsedquery 
>> changed to term "rss".
>> But the original querystring has no "rss" substring at all. That is really 
>> strange.
>>
>> Anyone seen this before?
>>
>> Single index, Solr 6.4.2.
>>
>> Regards
>> Bernd
>>

Connection rest by peer error

[Srinivas Kashyap]

Hello,

In one of our tomcat based solr 5.2.1 deployment environment, we are 
experiencing the below error which is occurring recursively.

When the app is restarted, the error won't show up until sometime. Max allowed 
connections in tomcat context xml and JVM memory of solr is sufficient enough.

14-Aug-2017 09:59:34.515 SEVERE [http-nio--exec-7] 
org.apache.solr.common.SolrException.log 
null:org.apache.catalina.connector.ClientAbortException: java.io.IOException: 
Connection reset by peer
at 
org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:393)
at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:426)
at 
org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:342)
at 
org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:317)
at 
org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.java:110)
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:297)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
at org.apache.solr.util.FastWriter.flush(FastWriter.java:137)
at 
org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:54)
at 
org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:727)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
at 
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at 
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616)
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:522)
at 
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1095)
at 
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:672)
at 
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1502)
at 
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1458)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at 
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
at sun.nio.ch.IOUtil.write(IOUtil.java:65)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:124)
at 
org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:101)
at 
org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:172)
at 
org.apache.coyote.http11.InternalNioOutputBuffer.writeToSocket(InternalNioOutputBuffer.java:139)
at 
org.apache.coyote.http11.InternalNioOutputBuffer.addToBB(InternalNioOutputBuffer.java:197)
at 
org.apache.coyote.http11.InternalNioOutputBuffer.access$000(InternalNioOutputBuffer.java:41)
at 
org.apache.coyote.http11.InternalNioOutputBuffer$SocketOutputBuffer.doWrite(InternalNioOutputBuffer.java:320)
at 
org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:118)
at 
org.apache.coyote.http11.AbstractOutputBuffer.doWrite(AbstractOutputBuffer.java:256)
at org.apache.coyote.Response.doWrite(Response.java:501)
at 
org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:388)
... 31 more

Please guide me what should I be looking at.


DISCLAIMER: E-mails and attachments from Bamboo Rose, LLC are confidential. If 
you 

Re: Get results in multiple orders (multiple boosts)

[Rick Leir]

Luca
Can you give me an example? If category_id is 9500, what would you want to sort 
on? -- Rick

On August 16, 2017 5:19:56 AM EDT, Luca Dall'Osto 
 wrote:
>Hello Rick,
>I have no algorithm: user choose the order of the categories, and the
>sources, For example:
>
>
>For Category:
>- at position 0 category_id 9500.
>- at position 1 category_id 10.
>- at position 2 category_ud 555.
>(etc...)
>
>
>For Source:
>
>- at position 0 source_id 12.
>- at position 1 source_id 30.
>
>- at position 2 source_id 3.
>
>(etc...)
>
>
>After that, user decide what kind of sort apply.
>For example: fist by DATE, then by CATEGORY, then by SOURCE and then by
>NAME.
>I have 1 array with all category Ids with correct order decided by user
>and another one with source Ids in correct order decided by user.
>Then I have another array that specify the sort type (like example,
>user should ask documents ordered by DATE first, then by CATEGORY,
>etc... )
>Natural sort order could be fine only for DATE and NAME, but for
>CATEGORY and SOURCE I have to use the array with ids sorted by user.
>Thanks!
>
>Luca 
>
>On Tuesday, August 8, 2017 6:54 PM, Rick Leir 
>wrote:
> 
>
> Luca,
>What is the algorithm for the custom sort order?  -- Rick
>
>On August 7, 2017 6:38:49 AM EDT, Luca Dall'Osto
> wrote:
>>Hello Rick,
>>thanks for your answer.
>>Yes, I compose solr query from frontend request, but I'm not able to
>>sort by a custom order, only by natural order (for example:
>>sort=category desc, source desc, /*...*/ ).
>>How do you set a custom sort order in solr?
>>Thanks
>>
>>Luca
>>
>>
>> 
>>
>>On Friday, August 4, 2017 7:41 PM, Rick Leir 
>>wrote:
>> 
>>
>> Luca
>>I hope you have a web app in front of Solr. It could accept parameters
>>from the browser, then construct the query as necessary to do your
>>sorting. Cheers -- Rick
>>
>>On August 4, 2017 5:32:31 AM EDT, Luca Dall'Osto
>> wrote:
>>>Hello,
>>>sorry for the late, I was out of my home.
>>>
>>>
>>>In response to Rick: 
>>>I can't do that because: 
>>>1) each user should have multiple sort (for example user "A" can sort
>>>by date and then by category and then by name ...) .
>>>2) the sort is not natural sort: user has a custom order for a field
>>>(for example sorting category field of user "A" could be category 10
>>at
>>>position 1, category 2 at position 2, category 9500 at position 3,
>>>category 40 at position 5 ...).
>>>
>>>
>>>In response to Susheel:have you done an example? 
>>>
>>>Thanks for your response.
>>>
>>>Luca
>>>
>>> 
>>>
>>>On Thursday, July 20, 2017 4:10 AM, Susheel Kumar
>>> wrote:
>>> 
>>>
>>> Let me try to put an example for custom sort.
>>>
>>>On Wed, Jul 19, 2017 at 6:34 AM, Rick Leir 
>wrote:
>>>
 Luca,
 You can pass a sort parameter in the query. User A could
>>>sort=date%20desc
 and user b could sort=foofield%20asc.

 Maybe query functions can also help with this. Cheers -- Rick

 On July 19, 2017 4:39:59 AM EDT, Luca Dall'Osto
  wrote:
 >Hello,The problem of build an index is that each user has a custom
 >source order and category order: are not static orders (for
>example
 >user X could have category:5 as most important category but user Y
 >could have category:9 as most important).
 >Has anyone ever written a custom sort function in solr?Maybe a
>link
>>>of
 >a tutorial or an example could be very helpful. Thanks
 >
 >Luca
 >
 >On Tuesday, July 18, 2017 4:18 PM, alessandro.benedetti
 > wrote:
 >
 >
 >"I have different "sort preferences", so I can't build a index and
>>>use
 >for
 >sorting.Maybe I have to sort by category then by source and by
>>>language
 >or
 >by source, then by category and by date"
 >
 >I would like to focus on this bit.
 >It is ok to go for a custom function and sort at query time, but I
>>>am
 >curious to explore why an index time solution should not be ok.
 >You can have these distinct fields :
 >source_priority
 >language_priority
 >category_priority
 >ect
 >
 >This values can be assigned at the documents at indexing time (
>>>using
 >for
 >example a custom update request processor).
 >Then at query time you can easily sort on those values in a multi
 >layered
 >approach :
 >sort:source_priority desc, category_priority  desc
 >Of course, if the priority for a source changes quite often or if
>>>it's
 >user
 >dependent, a query time solution would be preferred.
 >
 >
 >
 >
 >
 >-
 >---
 >Alessandro Benedetti
 >Search Consultant, R Software Engineer, Director
 >Sease Ltd. - www.sease.io
 >--
 >View this message in context:
 

Re: Get results in multiple orders (multiple boosts)

[Luca Dall'Osto]

Hello Rick,
I have no algorithm: user choose the order of the categories, and the sources, 
For example:


For Category:
- at position 0 category_id 9500.
- at position 1 category_id 10.
- at position 2 category_ud 555.
(etc...)


For Source:

- at position 0 source_id 12.
- at position 1 source_id 30.

- at position 2 source_id 3.

(etc...)


After that, user decide what kind of sort apply.
For example: fist by DATE, then by CATEGORY, then by SOURCE and then by NAME.
I have 1 array with all category Ids with correct order decided by user and 
another one with source Ids in correct order decided by user.
Then I have another array that specify the sort type (like example, user should 
ask documents ordered by DATE first, then by CATEGORY, etc... )
Natural sort order could be fine only for DATE and NAME, but for CATEGORY and 
SOURCE I have to use the array with ids sorted by user.
Thanks!

Luca 

On Tuesday, August 8, 2017 6:54 PM, Rick Leir  wrote:
 

 Luca,
What is the algorithm for the custom sort order?  -- Rick

On August 7, 2017 6:38:49 AM EDT, Luca Dall'Osto 
 wrote:
>Hello Rick,
>thanks for your answer.
>Yes, I compose solr query from frontend request, but I'm not able to
>sort by a custom order, only by natural order (for example:
>sort=category desc, source desc, /*...*/ ).
>How do you set a custom sort order in solr?
>Thanks
>
>Luca
>
>
> 
>
>On Friday, August 4, 2017 7:41 PM, Rick Leir 
>wrote:
> 
>
> Luca
>I hope you have a web app in front of Solr. It could accept parameters
>from the browser, then construct the query as necessary to do your
>sorting. Cheers -- Rick
>
>On August 4, 2017 5:32:31 AM EDT, Luca Dall'Osto
> wrote:
>>Hello,
>>sorry for the late, I was out of my home.
>>
>>
>>In response to Rick: 
>>I can't do that because: 
>>1) each user should have multiple sort (for example user "A" can sort
>>by date and then by category and then by name ...) .
>>2) the sort is not natural sort: user has a custom order for a field
>>(for example sorting category field of user "A" could be category 10
>at
>>position 1, category 2 at position 2, category 9500 at position 3,
>>category 40 at position 5 ...).
>>
>>
>>In response to Susheel:have you done an example? 
>>
>>Thanks for your response.
>>
>>Luca
>>
>> 
>>
>>On Thursday, July 20, 2017 4:10 AM, Susheel Kumar
>> wrote:
>> 
>>
>> Let me try to put an example for custom sort.
>>
>>On Wed, Jul 19, 2017 at 6:34 AM, Rick Leir  wrote:
>>
>>> Luca,
>>> You can pass a sort parameter in the query. User A could
>>sort=date%20desc
>>> and user b could sort=foofield%20asc.
>>>
>>> Maybe query functions can also help with this. Cheers -- Rick
>>>
>>> On July 19, 2017 4:39:59 AM EDT, Luca Dall'Osto
>>>  wrote:
>>> >Hello,The problem of build an index is that each user has a custom
>>> >source order and category order: are not static orders (for example
>>> >user X could have category:5 as most important category but user Y
>>> >could have category:9 as most important).
>>> >Has anyone ever written a custom sort function in solr?Maybe a link
>>of
>>> >a tutorial or an example could be very helpful. Thanks
>>> >
>>> >Luca
>>> >
>>> >On Tuesday, July 18, 2017 4:18 PM, alessandro.benedetti
>>> > wrote:
>>> >
>>> >
>>> >"I have different "sort preferences", so I can't build a index and
>>use
>>> >for
>>> >sorting.Maybe I have to sort by category then by source and by
>>language
>>> >or
>>> >by source, then by category and by date"
>>> >
>>> >I would like to focus on this bit.
>>> >It is ok to go for a custom function and sort at query time, but I
>>am
>>> >curious to explore why an index time solution should not be ok.
>>> >You can have these distinct fields :
>>> >source_priority
>>> >language_priority
>>> >category_priority
>>> >ect
>>> >
>>> >This values can be assigned at the documents at indexing time (
>>using
>>> >for
>>> >example a custom update request processor).
>>> >Then at query time you can easily sort on those values in a multi
>>> >layered
>>> >approach :
>>> >sort:source_priority desc, category_priority  desc
>>> >Of course, if the priority for a source changes quite often or if
>>it's
>>> >user
>>> >dependent, a query time solution would be preferred.
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >-
>>> >---
>>> >Alessandro Benedetti
>>> >Search Consultant, R Software Engineer, Director
>>> >Sease Ltd. - www.sease.io
>>> >--
>>> >View this message in context:
>>> >http://lucene.472066.n3.nabble.com/Get-results-in-
>>> multiple-orders-multiple-boosts-tp4346304p4346559.html
>>> >Sent from the Solr - User mailing list archive at Nabble.com.
>>> >
>>> >
>>> >
>>>
>>> --
>>> Sorry for being brief. Alternate email is rickleir at yahoo dot com
>>
>>
>>  
>
>-- 
>Sorry for being brief. Alternate email is rickleir at yahoo dot com
>
>  

-- 
Sorry 

Re: Issue with delta import

[vrindavda]

yes.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Issue-with-delta-import-tp4347680p4350734.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: QueryParser changes query by itself

[Bernd Fehling]

Hi Ahmet,

thank you for your reply. I was also targeting towards QueryCache but
with your hint about LUCENE-3758 I have a better point to start with.

If the system is under high load and the the QueryCache is filled I have
a higher rate of changed queries.
In debug mode the "timing-->process-->query" of changed queries is always "0" 
zero.

The query parser "SynonymQParser" is self developed which uses QParserPlugin.
There is no caching inside and works for years.
Only compiled against recent Lucene/Solr and some modifications like
using Builder with newer Lucene versions.

I will test without query cache.
Wich one should be disabled, Query Result Cache?

Regards
Bernd


Am 15.08.2017 um 19:07 schrieb Ahmet Arslan:
> Hi Bernd,
> 
> In LUCENE-3758, a new member field added into ComplexPhraseQuery class. But 
> we didn't change its hashCode method accordingly. This caused anomalies in 
> Solr, and Yonik found the bug and fixed hashCode. Your e-mail somehow 
> reminded me this.
> Could it be the QueryCache and hashCode method/implementation of Query 
> subclasses.
> May be your good and bad example is producing same hashCode? And this is 
> confusing query cache in solr?
> Can you disable the query cache, to test it?
> By the way, which query parser are you using? I believe SynonymQuery is 
> produced by BM25 similarity, right?
> 
> Ahmet
> 
> 
> On Friday, August 11, 2017, 2:48:07 PM GMT+3, Bernd Fehling 
>  wrote:
> 
> 
> We just noticed a very strange problem with Solr 6.4.2 QueryParser.
> The QueryParser changes the query by itself from time to time.
> This happens if doing a search request reload several times at higher rate.
> 
> Good example:
> ...
> textth:waffenhandel
>   
> ...
> textth:waffenhandel
> textth:waffenhandel
>   +SynonymQuery(Synonym(textth:"arms sales" 
> textth:"arms trade"...
>   +Synonym(textth:"arms sales" textth:"arms 
> trade"...
> 
> 
> Bad example:
> ...
> textth:waffenhandel
>   
> ...
> textth:waffenhandel
> textth:waffenhandel
>   +textth:rss
>   +textth:rss
> 
> As you can see in the bad example after several reloads the parsedquery 
> changed to term "rss".
> But the original querystring has no "rss" substring at all. That is really 
> strange.
> 
> Anyone seen this before?
> 
> Single index, Solr 6.4.2.
> 
> Regards
> Bernd
>