Re: Autoscaling with core properties

[Shalin Shekhar Mangar]

Hi Jim,

Very interesting scenario that we didn't anticipate. I think this is a
limitation of the MoveReplica API which does not move core properties.

But it also raises questions such as whether to always move all core
properties? I personally think core properties are an artifact that was
suitable for non-cloud Solr but does not lend well to cloud environments.
We also have replica properties in Solr and corresponding APIs to
add/update/delete them. See
https://lucene.apache.org/solr/guide/7_4/collections-api.html#addreplicaprop
(however, I think even these are not carried forward by move replica
today). Do you mind opening a Jira to discuss how we can fix the current
behavior?

Would using the Config API to set user properties work for your use-case?
See
https://lucene.apache.org/solr/guide/7_4/configuring-solrconfig-xml.html#substituting-properties-in-solr-config-files
and
https://lucene.apache.org/solr/guide/7_4/config-api.html#commands-for-user-defined-properties

We can improve autoscaling actions such as ComputePlanAction to add custom
core properties to any add replica or move replica command. That is
probably worth another Jira as well.


On Wed, Sep 5, 2018 at 11:54 PM James Strassburg 
wrote:

> Hello,
>
> We're creating a SolrCloud in AWS and attempting to use autoscaling to add
> replicas during nodeAdded/nodeLost. This was working fine for us until we
> started creating collections specifying core properties (e.g.
>
> /solr/admin/collections?action=CREATE_datasource=REDACTED).
> It seems that when the nodeLost/Added trigger fires the properties don't
> manifest in the core create invocation and we get errors like the
> following:
>
> products_20180904200015_shard1_replica_n39:
> org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> Could not load conf for core products_20180904200015_shard1_replica_n39:
> Can't load schema schema.xml: No system property or default value specified
> for synonyms_datasource value:jdbc/${synonyms_datasource}
>
> The autoscaling API also doesn't appear to have a way to set the properties
> when we create the triggers.
>
> Are we missing something or is this not supported at this time? I couldn't
> find a relevant JIRA or other documentation or solr-user discussion on
> this.
>
> thanks,
>
> JiM
>


-- 
Regards,
Shalin Shekhar Mangar.

Re: Null Pointer Exception without details on Update in schemaless 7.4

[deniz]

server is also 7.4 

and your assumption/check on null values on input doc seems legit... I have
added some checks before pushing the doc to solr and replaced null values
with some default values, and updates seem going through w/o problem...
though having a little bit explanatory logs on server side might be
useful...

thanks a lot for pointing out the null fields 



-
Zeki ama calismiyor... Calissa yapar...
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Null Pointer Exception without details on Update in schemaless 7.4

[Shawn Heisey]

On 9/5/2018 8:32 PM, deniz wrote:

I have set up a schemaless solr (cloud) and have been trying to test the
updates. as DIH is not going through field guessing, I have wrote a small
piece of code to query data from db and push the docs to solr...

Once the client pushes the docs to solr, on server there are npe logs as
below:
  
o.a.s.h.RequestHandlerBase java.lang.NullPointerException

   at
org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$AddSchemaFieldsUpdateProcessor.mapValueClassesToFieldType(AddSchemaFieldsUpdateProcessorFactory.java:509)


Your client-side log indicated the SolrJ version, but you did not say 
which version of Solr you're running on the server.


If the server version is also 7.4.0, then my examination of the source 
code will be accurate.  It seems that when the 
AddSchemaFieldsUpdateProcessorFactory class is looping through the 
fields that it found in the input document, one of them is the java 
'null' value instead of an instance of the SolrInputField class. Thus 
when the code tries to call the "getValues()" method (line 509) on the 
field object, Java throws NullPointerException.


I do not know what could result in a null value like that, or whether 
SolrInputDocument needs some kind of safety check to make sure a null 
SolrInputField is not possible.


If the server version isn't 7.4.0, then I cannot be 100 percent certain 
about what's happening, because the code could be different in that version.


Thanks,
Shawn

Re: Contextual Synonym Filter

[Ryan Yacyshyn]

Hi Luca,

I wonder if payloads might be able to solve your case for contextual
synonyms?

http://jorgelbg.github.io/posts/solr-contextual-synonyms

Ryan





On Mon, 3 Sep 2018 at 21:59 Andrea Gazzarini  wrote:

> Hi Luca,
> I believe this is not an easy task to do passing through Solr/Lucene
> internals; did you try to use what Solr offers out of the box?
> For example, you could define several fields associated where each
> corresponding field type uses a different synonym set. So you would have
>
>   * F1 -> FT1 -> SYNSET1
>   * F2 -> FT2 -> SYNSET2
>   * ...
>
> if you query using *fq=F1:something* then the *FT1* (and the *SYNSET1*)
> will be used, if you query using *fq=F2:something* then the *FT2* (and
> the *SYNSET2*) will be used, and so on
>
> I don't know your context so my suggestion could be absolutely in the
> wrong path
>
> Best,
> Andrea
>
> On 03/09/2018 15:41, Vergantini Luca wrote:
> >
> > I need to create a contextual Synonym Filter:
> >
> > I need that the Synonym Filter load different synonym configuration
> > based on the fq query parameter.
> >
> > I’ve already modified the SynonymGraphFilterFactory to load from DB
> > (this is another requirement) but I can’t understand how to make the
> > fq parameter arrive to the Factory.
> >
> > Maybe I need a Query Parser plugin?
> >
> > Please help
> >
> >
> >
> > Luca Vergantini
> >
> > Whitehall Reply
> > Via del Giorgione, 59
> > 00147 - Roma - ITALY
> > phone: +39 06 844341 <+39%2006%20844341>
> > l.vergant...@reply.it 
> > www.reply.it
> >
> > Whitehall Reply
>
>

Re: Null Pointer Exception without details on Update in schemaless 7.4

[deniz]

nope, the data i am pushing is stuff like string, int, etc etc

i have have checked further and made bunch of trial and error, here are the
things I was able to figure out:

 - If a date value from database is null, then it is breaking the update
with "-00-00" is not a valid date string error. 
 - There are some column names with suffixes which is triggering dynamic
field creation with incorrect data.

But above cases have proper error logs unlike the one I have initially
posted. 




-
Zeki ama calismiyor... Calissa yapar...
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Null Pointer Exception without details on Update in schemaless 7.4

[David Hastings]

Are you trying to push a java hash to solr?

On Sep 5, 2018, at 10:32 PM, deniz 
mailto:denizdurmu...@gmail.com>> wrote:

I have set up a schemaless solr (cloud) and have been trying to test the
updates. as DIH is not going through field guessing, I have wrote a small
piece of code to query data from db and push the docs to solr...

Once the client pushes the docs to solr, on server there are npe logs as
below:


o.a.s.h.RequestHandlerBase java.lang.NullPointerException
 at
org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$AddSchemaFieldsUpdateProcessor.mapValueClassesToFieldType(AddSchemaFieldsUpdateProcessorFactory.java:509)
 at
org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$AddSchemaFieldsUpdateProcessor.processAdd(AddSchemaFieldsUpdateProcessorFactory.java:396)
 at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
 at
org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)
 at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
 at
org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)
 at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
 at
org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)
 at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
 at
org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)
 at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
 at
org.apache.solr.update.processor.FieldNameMutatingUpdateProcessorFactory$1.processAdd(FieldNameMutatingUpdateProcessorFactory.java:75)
 at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
 at
org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)
 at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
 at
org.apache.solr.update.processor.AbstractDefaultValueUpdateProcessorFactory$DefaultValueUpdateProcessor.processAdd(AbstractDefaultValueUpdateProcessorFactory.java:92)
 at
org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:98)
 at
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:188)
 at
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:144)
 at
org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:311)
 at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256)
 at
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:130)
 at
org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:276)
 at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256)
 at
org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:178)
 at
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:195)
 at
org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:109)
 at
org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:55)
 at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
 at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
 at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
 at org.apache.solr.core.SolrCore.execute(SolrCore.java:2539)
 at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:709)
 at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:515)
 at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
 at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
 at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
 at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
 at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
 at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
 at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
 at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
 at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
 at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
 at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
 at

Null Pointer Exception without details on Update in schemaless 7.4

[deniz]

I have set up a schemaless solr (cloud) and have been trying to test the
updates. as DIH is not going through field guessing, I have wrote a small
piece of code to query data from db and push the docs to solr...

Once the client pushes the docs to solr, on server there are npe logs as
below:

 
o.a.s.h.RequestHandlerBase java.lang.NullPointerException
  at
org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$AddSchemaFieldsUpdateProcessor.mapValueClassesToFieldType(AddSchemaFieldsUpdateProcessorFactory.java:509)
  at
org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$AddSchemaFieldsUpdateProcessor.processAdd(AddSchemaFieldsUpdateProcessorFactory.java:396)
  at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
  at
org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)
  at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
  at
org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)
  at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
  at
org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)
  at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
  at
org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)
  at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
  at
org.apache.solr.update.processor.FieldNameMutatingUpdateProcessorFactory$1.processAdd(FieldNameMutatingUpdateProcessorFactory.java:75)
  at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
  at
org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)
  at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
  at
org.apache.solr.update.processor.AbstractDefaultValueUpdateProcessorFactory$DefaultValueUpdateProcessor.processAdd(AbstractDefaultValueUpdateProcessorFactory.java:92)
  at
org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:98)
  at
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:188)
  at
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:144)
  at
org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:311)
  at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256)
  at
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:130)
  at
org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:276)
  at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256)
  at
org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:178)
  at
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:195)
  at
org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:109)
  at
org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:55)
  at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
  at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
  at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
  at org.apache.solr.core.SolrCore.execute(SolrCore.java:2539)
  at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:709)
  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:515)
  at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
  at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
  at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
  at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
  at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
  at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
  at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
  at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
  at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
  at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
  at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
  at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
  at

Re: Streaming timeseries() and buckets with no docs

[Jan Høydahl]

Checked git history for TimeSeriesStream on master, and I cannot see any 
commits related to this?

SOLR-11914: Deprecated some SolrParams methods. * toSolrParams(nl) moved to a 
NamedList method, which is more natural. David Smiley 23.04.2018, 19:26
SOLR-11629: Add new CloudSolrClient.Builder ctors Jason Gerlowski 10.03.2018, 
15:30
SOLR-11799: Fix NPE and class cast exceptions in the TimeSeriesStream Joel 
Bernstein 28.12.2017, 17:14
SOLR-11490: Add missing @since tags To all descendants of TupleStream Alexandre 
Rafalovitch 19.10.2017, 03:38
SOLR-10770: Fix precommit Joel Bernstein 30.05.2017, 20:51
SOLR-10770: Add date formatting to timeseries Streaming Expression Joel 
Bernstein 30.05.2017, 20:38
SOLR-10566: Fix error handling Joel Bernstein 01.05.2017, 18:06
SEARCH-313: Handled unescaped plus sign in gap Joel Bernstein 27.04.2017, 04:34
SOLR-10566: Fix precommit Joel Bernstein 26.04.2017, 17:17
SOLR-10566: Add timeseries Streaming Expression Joel Bernstein 26.04.2017, 16:57

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 5. sep. 2018 kl. 16:12 skrev Jan Høydahl :
> 
> I have tested this with latest released ver 7.4.0
> 
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com 
> 
>> 4. sep. 2018 kl. 16:32 skrev Joel Bernstein > >:
>> 
>> Which version are you using?
>> 
>> I remember addressing this issue, but it may have been in Alfresco's
>> version of Solr and never got ported back.
>> 
>> I do agree that in a time series a null value is not what people want. It
>> is a very small change to populate with zeros if it has not already been
>> done in the latest versions.
>> 
>> Joel Bernstein
>> http://joelsolr.blogspot.com/ 
>> 
>> 
>> On Mon, Sep 3, 2018 at 8:58 AM Jan Høydahl  wrote:
>> 
>>> Hi
>>> 
>>> We have a timeseries expression with gap="+1DAY" and a sum(imps_l) to
>>> aggregate sums of an integer for each bucket.
>>> Now, some day buckets do not contain any documents at all, and instead of
>>> returning a tuple with value 0, it returns
>>> a tuple with no entry at all for the sum, see the bucket for date_dt
>>> 2018-06-22 below:
>>> 
>>> {
>>>  "result-set": {
>>>"docs": [
>>>  {
>>>"sum(imps_l)": 0,
>>>"date_dt": "2018-06-21",
>>>"count(*)": 5
>>>  },
>>>  {
>>>"date_dt": "2018-06-22",
>>>"count(*)": 0
>>>  },
>>>  {
>>>"EOF": true,
>>>"RESPONSE_TIME": 3
>>>  }
>>>]
>>>  }
>>> }
>>> 
>>> 
>>> Now when we want to convert this into a column using col(a,'sum(imps_l)')
>>> then that array will get mostly numbers
>>> but also some string entries 'sum(imps_l)' which is the key name. I need
>>> purely integers in the column.
>>> 
>>> Should the timeseries() have output values for all functions even if there
>>> are no documents in the bucket?
>>> Or is there something similar to the select() expression that can take a
>>> stream of tuples not originating directly
>>> from search() and replace values? Or is there perhaps a function that can
>>> loop through the column produced by col()
>>> and replace non-numeric values with 0?
>>> 
>>> --
>>> Jan Høydahl, search solution architect
>>> Cominvent AS - www.cominvent.com
>>> 
>>> 
> 

Autoscaling with core properties

[James Strassburg]

Hello,

We're creating a SolrCloud in AWS and attempting to use autoscaling to add
replicas during nodeAdded/nodeLost. This was working fine for us until we
started creating collections specifying core properties (e.g.
/solr/admin/collections?action=CREATE_datasource=REDACTED).
It seems that when the nodeLost/Added trigger fires the properties don't
manifest in the core create invocation and we get errors like the following:

products_20180904200015_shard1_replica_n39:
org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
Could not load conf for core products_20180904200015_shard1_replica_n39:
Can't load schema schema.xml: No system property or default value specified
for synonyms_datasource value:jdbc/${synonyms_datasource}

The autoscaling API also doesn't appear to have a way to set the properties
when we create the triggers.

Are we missing something or is this not supported at this time? I couldn't
find a relevant JIRA or other documentation or solr-user discussion on
this.

thanks,

JiM

Re: Accessing multiple datasource from an entity

[Shawn Heisey]

On 9/5/2018 11:36 AM, Monique Monteiro wrote:

I need to add a deltaQuery to a DIH configuration, but it should reference
a different datasource.  Example:


 
 

 
 *
*  ...*

Is it possible?


As far as I am aware, each entity can only reference one dataSource.  
Both of your queries are in the same entity, so they're going to use the 
same dataSource.


DIH is a fairly simple tool.  It's highly configurable, but has limitations.

To do what you're trying, you're going to need to build your own 
indexing software that can bridge between your data sources and Solr.  
There are Solr clients available for many programming languages, 
although the only official client is the one for Java.  For all other 
languages, the clients are third-party.


Thanks,
Shawn

Accessing multiple datasource from an entity

[Monique Monteiro]

Hi all,

I need to add a deltaQuery to a DIH configuration, but it should reference
a different datasource.  Example:






*
*  ...*

Is it possible?

Thanks in advance.

-- 
Monique Monteiro
Twitter: http://twitter.com/monilouise

Re: 7.3 appears to leak

[Tomás Fernández Löbbe]

I created SOLR-12743 to track this.

On Mon, Jul 16, 2018 at 12:30 PM Markus Jelsma 
wrote:

> Hello Thomas,
>
> To be absolutely sure you suffer from the same problem as one of our
> collections, can you confirm that your Solr cores are leaking a
> SolrIndexSearcher instance on each commit? If not, there may be a second
> problem.
>
> Also, do you run any custom plugins or apply patches to your Solr
> instances? Or is your Solr a 100 % official build?
>
> Thanks,
> Markus
>
>
>
> -Original message-
> > From:Thomas Scheffler 
> > Sent: Monday 16th July 2018 13:39
> > To: solr-user@lucene.apache.org
> > Subject: Re: 7.3 appears to leak
> >
> > Hi,
> >
> > we noticed the same problems here in a rather small setup. 40.000
> metadata documents with nearly as much files that have „literal.*“ fields
> with it. While 7.2.1 has brought some tika issues the real problems started
> to appear with version 7.3.0 which are currently unresolved in 7.4.0.
> Memory consumption is out-of-roof. Where previously 512MB heap was enough,
> now 6G aren’t enough to index all files.
> >
> > kind regards,
> >
> > Thomas
> >
> > > Am 04.07.2018 um 15:03 schrieb Markus Jelsma <
> markus.jel...@openindex.io>:
> > >
> > > Hello Andrey,
> > >
> > > I didn't think of that! I will try it when i have the courage again,
> probably next week or so.
> > >
> > > Many thanks,
> > > Markus
> > >
> > >
> > > -Original message-
> > >> From:Kydryavtsev Andrey 
> > >> Sent: Wednesday 4th July 2018 14:48
> > >> To: solr-user@lucene.apache.org
> > >> Subject: Re: 7.3 appears to leak
> > >>
> > >> If it is not possible to find a resource leak by code analysis and
> there is no better ideas, I can suggest a brute force approach:
> > >> - Clone Solr's sources from appropriate branch
> https://github.com/apache/lucene-solr/tree/branch_7_3
> > >> - Log every searcher's holder increment/decrement operation in a way
> to catch every caller name (use Thread.currentThread().getStackTrace() or
> something)
> https://github.com/apache/lucene-solr/blob/branch_7_3/solr/core/src/java/org/apache/solr/util/RefCounted.java
> > >> - Build custom artefacts and upload them on prod
> > >> - After memory leak happened - analyse logs to see what part of
> functionality doesn't decrement searcher after counter was incremented. If
> searchers are leaked - there should be such code I guess.
> > >>
> > >> This is not something someone would like to do, but it is what it is.
> > >>
> > >>
> > >>
> > >> Thank you,
> > >>
> > >> Andrey Kudryavtsev
> > >>
> > >>
> > >> 03.07.2018, 14:26, "Markus Jelsma" :
> > >>> Hello Erick,
> > >>>
> > >>> Even the silliest ideas may help us, but unfortunately this is not
> the case. All our Solr nodes run binaries from the same source from our
> central build server, with the same libraries thanks to provisioning. Only
> schema and config are different, but the  directive is the same all
> over.
> > >>>
> > >>> Are there any other ideas, speculations, whatever, on why only our
> main text collection leaks a SolrIndexSearcher instance on commit since
> 7.3.0 and every version up?
> > >>>
> > >>> Many thanks?
> > >>> Markus
> > >>>
> > >>> -Original message-
> >   From:Erick Erickson 
> >   Sent: Friday 29th June 2018 19:34
> >   To: solr-user 
> >   Subject: Re: 7.3 appears to leak
> > 
> >   This is truly puzzling then, I'm clueless. It's hard to imagine
> this
> >   is lurking out there and nobody else notices, but you've eliminated
> >   the custom code. And this is also very peculiar:
> > 
> >   * it occurs only in our main text search collection, all other
> >   collections are unaffected;
> >   * despite what i said earlier, it is so far unreproducible outside
> >   production, even when mimicking production as good as we can;
> > 
> >   Here's a tedious idea. Restart Solr with the -v option, I _think_
> that
> >   shows you each and every jar file Solr loads. Is it "somehow"
> possible
> >   that your main collection is loading some jar from somewhere that's
> >   different than you expect? 'cause silly ideas like this are all I
> can
> >   come up with.
> > 
> >   Erick
> > 
> >   On Fri, Jun 29, 2018 at 9:56 AM, Markus Jelsma
> >    wrote:
> >   > Hello Erick,
> >   >
> >   > The custom search handler doesn't interact with
> SolrIndexSearcher, this is really all it does:
> >   >
> >   >   public void handleRequestBody(SolrQueryRequest req,
> SolrQueryResponse rsp) throws Exception {
> >   > super.handleRequestBody(req, rsp);
> >   >
> >   > if (rsp.getToLog().get("hits") instanceof Integer) {
> >   >   rsp.addHttpHeader("X-Solr-Hits",
> String.valueOf((Integer)rsp.getToLog().get("hits")));
> >   > }
> >   > if (rsp.getToLog().get("hits") instanceof Long) {
> >   >   rsp.addHttpHeader("X-Solr-Hits",
> 

Re: Heap Memory Problem after Upgrading to 7.4.0

[Tomás Fernández Löbbe]

I think this is pretty bad. I created
https://issues.apache.org/jira/browse/SOLR-12743. Feel free to add any more
details you have there.

On Mon, Sep 3, 2018 at 1:50 PM Markus Jelsma 
wrote:

> Hello Björn,
>
> Take great care, 7.2.1 cannot read an index written by 7.4.0, so you
> cannot roll back but need to reindex!
>
> Andrey Kudryavtsev made a good suggestion in the thread on how to find the
> culprit, but it will be a tedious task. I have not yet had the time or
> courage to venture there.
>
> Hope it helps,
> Markus
>
>
>
> -Original message-
> > From:Björn Häuser 
> > Sent: Monday 3rd September 2018 22:28
> > To: solr-user@lucene.apache.org
> > Subject: Re: Heap Memory Problem after Upgrading to 7.4.0
> >
> > Hi Markus,
> >
> > this reads exactly like what we have. Where you able to figure out
> anything? Currently thinking about rollbacking to 7.2.1.
> >
> >
> >
> > > On 3. Sep 2018, at 21:54, Markus Jelsma 
> wrote:
> > >
> > > Hello,
> > >
> > > Getting an OOM plus the fact you are having a lot of IndexSearcher
> instances rings a familiar bell. One of our collections has the same issue
> [1] when we attempted an upgrade 7.2.1 > 7.3.0. I managed to rule out all
> our custom Solr code but had to keep our Lucene filters in the schema, the
> problem persisted.
> > >
> > > The odd thing, however, is that you appear to have the same problem,
> but not with 7.3.0? Since you shortly after 7.3.0 upgraded to 7.4.0, can
> you confirm the problem is not also in 7.3.0?
> > >
> >
> > We had very similar problems with 7.3.0 but never analyzed them and just
> updated to 7.4.0 because I thought thats the bug we hit:
> https://issues.apache.org/jira/browse/SOLR-11882 <
> https://issues.apache.org/jira/browse/SOLR-11882>
> >
> >
> > > You should see the instance count for IndexSearcher increase by one
> for each replica on each commit.
> >
> >
> > Sorry, where can I find this? ;) Sorry, did not find anything.
> >
> > Thanks
> > Björn
> >
> > >
> > > Regards,
> > > Markus
> > >
> > > [1]
> http://lucene.472066.n3.nabble.com/RE-7-3-appears-to-leak-td4396232.html
> > >
> > >
> > >
> > > -Original message-
> > >> From:Erick Erickson 
> > >> Sent: Monday 3rd September 2018 20:49
> > >> To: solr-user 
> > >> Subject: Re: Heap Memory Problem after Upgrading to 7.4.0
> > >>
> > >> I would expect at least 1 IndexSearcher per replica, how many total
> > >> replicas hosted in your JVM?
> > >>
> > >> Plus, if you're actively indexing, there may temporarily be 2
> > >> IndexSearchers open while the new searcher warms.
> > >>
> > >> And there may be quite a few caches, at least queryResultCache and
> > >> filterCache and documentCache, one of each per replica and maybe two
> > >> (for queryResultCache and filterCache) if you have a background
> > >> searcher autowarming.
> > >>
> > >> At a glance, your autowarm counts are very high, so it may take some
> > >> time to autowarm leading to multiple IndexSearchers and caches open
> > >> per replica when you happen to hit a commit point. I usually start
> > >> with 16-20 as an autowarm count, the benefit decreases rapidly as you
> > >> increase the count.
> > >>
> > >> I'm not quite sure why it would be different in 7x .vs. 6x. How much
> > >> heap do you allocate to the JVM? And do you see similar heap dumps in
> > >> 6.6?
> > >>
> > >> Best,
> > >> Erick
> > >> On Mon, Sep 3, 2018 at 10:33 AM Björn Häuser 
> wrote:
> > >>>
> > >>> Hello,
> > >>>
> > >>> we recently upgraded our solrcloud (5 nodes, 25 collections, 1 shard
> each, 4 replicas each) from 6.6.0 to 7.3.0 and shortly after to 7.4.0. We
> are running Zookeeper 4.1.13.
> > >>>
> > >>> Since the upgrade to 7.3.0 and also 7.4.0 we encountering heap space
> exhaustion. After obtaining a heap dump it looks like that we have a lot of
> IndexSearchers open for our largest collection.
> > >>>
> > >>> The dump contains around ~60 IndexSearchers, and each containing
> around ~40mb heap. Another 500MB of heap is the fieldcache, which is
> expected in my opinion.
> > >>>
> > >>> The current config can be found here:
> https://gist.github.com/bjoernhaeuser/327a65291ac9793e744b87f0a561e844 <
> https://gist.github.com/bjoernhaeuser/327a65291ac9793e744b87f0a561e844>
> > >>>
> > >>> Analyzing the heap dump eclipse MAT says this:
> > >>>
> > >>> Problem Suspect 1
> > >>>
> > >>> 91 instances of "org.apache.solr.search.SolrIndexSearcher", loaded
> by "org.eclipse.jetty.webapp.WebAppClassLoader @ 0x6807d1048" occupy
> 1.981.148.336 (38,26%) bytes.
> > >>>
> > >>> Biggest instances:
> > >>>
> > >>>• org.apache.solr.search.SolrIndexSearcher @ 0x6ffd47ea8 -
> 70.087.272 (1,35%) bytes.
> > >>>• org.apache.solr.search.SolrIndexSearcher @ 0x79ea9c040 -
> 65.678.264 (1,27%) bytes.
> > >>>• org.apache.solr.search.SolrIndexSearcher @ 0x6855ad680 -
> 63.050.600 (1,22%) bytes.
> > >>>
> > >>>
> > >>> Problem Suspect 2
> > >>>
> > >>> 223 instances of "org.apache.solr.util.ConcurrentLRUCache", loaded
> by 

Re: Need to connect solr with solrj from AWS lambda

[Erick Erickson]

To unsubscribe:

Please follow the instructions here:
http://lucene.apache.org/solr/community.html#mailing-lists-irc. You
must use the _exact_ same e-mail as you used to subscribe.

If the initial try doesn't work and following the suggestions at the
"problems" link doesn't work for you, let us know. But note you need
to show us the _entire_ return header to allow anyone to diagnose the
problem.

Best,
Erick
On Tue, Sep 4, 2018 at 11:48 AM Srinu Lella  wrote:
>
> How to unsubscribe from this list?
>
> On Fri 31 Aug, 2018, 11:08 PM nalsrini,  wrote:
>
> > Hi,
> > I need to connect solr with solrj from AWS java lambda. I use solr 5.3.
> >
> > I get the client object like this:
> > SolrClient client = new
> > HttpSolrClient(System.getenv(SysEnvConstants.SOLR_HOST));
> >
> > I neither get an error nor a response when I call these(for example) from
> > the lambda:
> >
> > SolrDocument sorld = client.getById(id);
> >
> > OR
> >
> > UpdateResponse ur = client.deleteByQuery(sb.toString());
> >
> > thanks
> > Srini
> >
> >
> >
> > --
> > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
> >

Re: Solr 6.4.1: : SolrException:nfs no locks available

[Erick Erickson]

Here's what I copied from an explanation from Uwe Schindler, 'cause I
believe most anything he has to say on this subject:

It is just simple, Lucene locking and commits do not work correct on
NFS file systems because they are not fully POSIX conformant. Because
of this you may also produce corrupt indexes, as commits don't work
and corrupt concurrently open files. Also you may see JVM crushes if
memory mapped files are unmapped because of network failures and cause
SIGSEGV.

If you want to use Lucene on NFS mounts, you have 2 possibilities:
- Change to CIFS/Samba mounts (CIFS conforms to POSIX standards like
delete on last close and also supports correct locking with
NativeFSLockFactory) -- or move to local disks!
- Use a special deletion policy (https://lucene.apache.org/) to make
the commits not corrupt you open IndexSearchers because of suddenly
disappearing files (Lucene deletes files while they are open, as POSIX
has delete-on-last-close) and use SimpleFSLockFactory. But
SimpleFSLockFactory may hit stale lock files issues on killed JVMs.
Also don't use MMapDirectory for file storage as this wil likely crush
your JVM on network problems!

Some background: The original and recommended lock system works
correct with killed VMs, as the existence of the lock file has nothing
to do with the "state" of being locked. The lock file is just a
placeholder to actually have a file instance to do the locking. There
is no solution for mixed NFS and non-NFS directories. So get either
rid of them or add your own logic to choose right lock and deletion
policy depending on the file system. You may use Java 7+'s Path/Files
API to get all mount points. Memory mapping is risky with NFS, as a
no-longer reachable file may suddenly unmap the cache buffers from
process space and the next access will segmentation fault your JVM.
The Snapshot deletion policy keeps the last commits available on disk,
so the "delete-on-last-close" behaviour by POSIX is not required. But
you have to take care to delete snapshots when you have closed all
readers.

On Wed, Sep 5, 2018 at 6:59 AM Shawn Heisey  wrote:
>
> On 9/5/2018 6:55 AM, Imran Rajjad wrote:
> > I am using Solr Cloud 6.4.1. After a hard restart the solr nodes are
> > constantly showing to be in DOWN state and would not go into recovery. I
> > have also deleted the write.lock files from all the replica folders, but
> > the problem would not go away. The error displayed at web console is : no
> > locks available
> >
> > My replica folders reside in an nfs mount, I am using RHEL 6/CentOS6.8. Has
> > anyone ever faced this issue?
>
> Lucene-based software (including Solr) does NOT work well on NFS.  NFS
> does not provide all the locking functionality that Lucene tries to use
> by default.
>
> You're probably going to need to change the lock factory, and might even
> need to completely disable locking.  If you do disable locking, you have
> to be VERY careful to never allow more than one core or more than one
> Solr instance to try and open a core directory.  Doing so will likely
> corrupt the index.
>
> I strongly recommend NOT using NFS storage for Solr.  In addition to
> locking problems, it also tends to be extremely slow.Use a local filesystem.
>
> Thanks,
> Shawn
>

Re: ZooKeeper issues with AWS

[Erick Erickson]

Jack:

Thanks for letting us know, that provides evidence that will help
prioritize upgrading ZK.

Erick
On Wed, Sep 5, 2018 at 7:15 AM Jack Schlederer
 wrote:
>
> Ah, yes. We use ZK 3.4.13 for our ZK server nodes, but we never thought to
> upgrade the ZK JAR within Solr. We included that in our Solr image, and
> it's working like a charm, re-resolving DNS names when new ZKs come up with
> different IPs. Thanks for the help guys!
>
> --Jack
>
> On Sat, Sep 1, 2018 at 9:41 AM Shawn Heisey  wrote:
>
> > On 9/1/2018 3:42 AM, Björn Häuser wrote:
> > > as far as I can see the required fix for this is finally in 3.4.13:
> > >
> > > - https://issues.apache.org/jira/browse/ZOOKEEPER-2184 <
> > https://issues.apache.org/jira/browse/ZOOKEEPER-2184>
> > >
> > > Would be great to have this in the next solr update.
> >
> > Issue created.
> >
> > https://issues.apache.org/jira/browse/SOLR-12727
> >
> > Note that you can actually do this upgrade yourself on your Solr
> > install.  In server/solr-webapp/webapp/WEB-INF/lib, just delete the
> > current zookeeper jar, copy the 3.4.13 jar into the directory, then
> > restart Solr.  If you're on Windows, you'll need to stop Solr before you
> > can do that.  Windows doesn't allow deleting a file that is open.
> >
> > I expect that if you do this upgrade yourself, Solr should work without
> > problems.  Typically in the past when a new ZK version is included, no
> > code changes are required.
> >
> > Thanks,
> > Shawn
> >
> >

Re: Storing PID below /run

[ahubold]

I have now opened https://issues.apache.org/jira/browse/SOLR-12737 for this.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

about precision preference in autoscaling API

[Kudrettin Güleryüz]

Hi,

Can you please clarify "precision" preference definition? For instance, is
freedisk "precision" is honored when comparing the max and the min freedisk
of all nodes in the sortedNodes list? Or is it between any consecutive
nodes in the sortedNodes list? I think it may be the latter but then the
worst case difference between the min and max freedisk could be as high as
num_solr_nodes * precision value.

Thank you,

Re: node sort order and policy

[Kudrettin Güleryüz]

Thank you.

On Tue, Sep 4, 2018 at 10:34 PM Shalin Shekhar Mangar <
shalinman...@gmail.com> wrote:

> The "sortedNodes" shows the nodes in the system from the most loaded to
> least loaded order. All other things being equal, a node having the least
> free disk space will come first and the last node will be the one with the
> most available disk space.
>
> On Wed, Sep 5, 2018 at 3:01 AM Kudrettin Güleryüz 
> wrote:
>
> > Given exhibit A which shows the current sort order on a 7.3.1 cloud
> > installation, why would the sort order be from less freedisk to more
> > freedisk? Shouldn't it be the reverse since the number of cores is the
> same
> > for all nodes? preferences and policies are listed in exhibit B.
> >
> > Thank you,
> >
> >
> > Exhibit A: (autoscaling/diagnostics)
> > {
> >
> >   "responseHeader":{
> > "status":0,
> > "QTime":281},
> >   "diagnostics":{
> > "sortedNodes":[{
> > "node":"test-46:8983_solr",
> > "cores":138,
> > "freedisk":289.32603454589844,
> > "sysLoadAvg":88.0},
> >   {
> > "node":"test-44:8983_solr",
> > "cores":138,
> > "freedisk":303.9487609863281,
> > "sysLoadAvg":122.0},
> >   {
> > "node":"test-45:8983_solr",
> > "cores":138,
> > "freedisk":323.1516571044922,
> > "sysLoadAvg":166.0},
> >   {
> > "node":"test-43:8983_solr",
> > "cores":138,
> > "freedisk":325.2588005065918,
> > "sysLoadAvg":107.0},
> >   {
> > "node":"test-41:8983_solr",
> > "cores":138,
> > "freedisk":334.3892288208008,
> > "sysLoadAvg":56.99},
> >   {
> > "node":"test-42:8983_solr",
> > "cores":138,
> > "freedisk":360.2049674987793,
> > "sysLoadAvg":98.0}],
> > "violations":[]},
> >   "WARNING":"This response format is experimental.  It is likely to
> > change in the future."}
> >
> >
> > Exhibit B: (admin/autoscaling)
> >
> >  "cluster-preferences":[{
> >   "minimize":"cores",
> >   "precision":1}
> > ,{
> >   "maximize":"freedisk",
> >   "precision":10}
> > ,{
> >   "minimize":"sysLoadAvg",
> >   "precision":3}],
> >   "cluster-policy":[{
> >   "replica":"<2",
> >   "node":"#ANY"}
> > ,{
> >   "replica":"0",
> >   "freedisk":"<10",
> >   "strict":"true"}],
> >
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>

Re: ZooKeeper issues with AWS

[Jack Schlederer]

Ah, yes. We use ZK 3.4.13 for our ZK server nodes, but we never thought to
upgrade the ZK JAR within Solr. We included that in our Solr image, and
it's working like a charm, re-resolving DNS names when new ZKs come up with
different IPs. Thanks for the help guys!

--Jack

On Sat, Sep 1, 2018 at 9:41 AM Shawn Heisey  wrote:

> On 9/1/2018 3:42 AM, Björn Häuser wrote:
> > as far as I can see the required fix for this is finally in 3.4.13:
> >
> > - https://issues.apache.org/jira/browse/ZOOKEEPER-2184 <
> https://issues.apache.org/jira/browse/ZOOKEEPER-2184>
> >
> > Would be great to have this in the next solr update.
>
> Issue created.
>
> https://issues.apache.org/jira/browse/SOLR-12727
>
> Note that you can actually do this upgrade yourself on your Solr
> install.  In server/solr-webapp/webapp/WEB-INF/lib, just delete the
> current zookeeper jar, copy the 3.4.13 jar into the directory, then
> restart Solr.  If you're on Windows, you'll need to stop Solr before you
> can do that.  Windows doesn't allow deleting a file that is open.
>
> I expect that if you do this upgrade yourself, Solr should work without
> problems.  Typically in the past when a new ZK version is included, no
> code changes are required.
>
> Thanks,
> Shawn
>
>

Re: Streaming timeseries() and buckets with no docs

[Jan Høydahl]

I have tested this with latest released ver 7.4.0

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 4. sep. 2018 kl. 16:32 skrev Joel Bernstein :
> 
> Which version are you using?
> 
> I remember addressing this issue, but it may have been in Alfresco's
> version of Solr and never got ported back.
> 
> I do agree that in a time series a null value is not what people want. It
> is a very small change to populate with zeros if it has not already been
> done in the latest versions.
> 
> Joel Bernstein
> http://joelsolr.blogspot.com/
> 
> 
> On Mon, Sep 3, 2018 at 8:58 AM Jan Høydahl  wrote:
> 
>> Hi
>> 
>> We have a timeseries expression with gap="+1DAY" and a sum(imps_l) to
>> aggregate sums of an integer for each bucket.
>> Now, some day buckets do not contain any documents at all, and instead of
>> returning a tuple with value 0, it returns
>> a tuple with no entry at all for the sum, see the bucket for date_dt
>> 2018-06-22 below:
>> 
>> {
>>  "result-set": {
>>"docs": [
>>  {
>>"sum(imps_l)": 0,
>>"date_dt": "2018-06-21",
>>"count(*)": 5
>>  },
>>  {
>>"date_dt": "2018-06-22",
>>"count(*)": 0
>>  },
>>  {
>>"EOF": true,
>>"RESPONSE_TIME": 3
>>  }
>>]
>>  }
>> }
>> 
>> 
>> Now when we want to convert this into a column using col(a,'sum(imps_l)')
>> then that array will get mostly numbers
>> but also some string entries 'sum(imps_l)' which is the key name. I need
>> purely integers in the column.
>> 
>> Should the timeseries() have output values for all functions even if there
>> are no documents in the bucket?
>> Or is there something similar to the select() expression that can take a
>> stream of tuples not originating directly
>> from search() and replace values? Or is there perhaps a function that can
>> loop through the column produced by col()
>> and replace non-numeric values with 0?
>> 
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>> 
>> 

Re: SynonimGraphFilter expands wrong synonims

[Andrea Gazzarini]

You're right, my answer forgot to mention the *tokenizerFactory* 
parameter that you can add in the filter declaration. But, differently 
from what you think the default tokenizer used for parsing the synonyms 
_is not_ the tokenizer of the current analyzer (StandardTokenizer in 
your example) but WhitespaceTokenizer. See here [1] for a complete 
description of the filter capabilities.


So instead of switching the analyzer tokenizer you could also add a 
tokenizerFactory="solr.StandardTokenizerFactory" in the synonym filter 
declaration.


Best,
Andrea

[1] 
https://lucene.apache.org/solr/guide/6_6/filter-descriptions.html#FilterDescriptions-SynonymGraphFilter


On 05/09/2018 15:58, Danilo Tomasoni wrote:

Hi Andrea,

thank you for your answer.

About the second question: The standardTokenizer should be applied 
also to the phrase query, so the ' and - symbols should be removed 
even there, and this should allow a match in the synonim file isn't it?


With an example:


in phrase query:

"Cytosolic 5'-nucleotidase II" -> standardTokenizer -> Cytosolic, 5, 
nucleotidase, II



in synonym parsing:

...,Cytosolic 5'-nucleotidase II,... -> standardTokenizer -> 
Cytosolic, 5, nucleotidase, II



So the two graphs should match.. or I'm wrong?
Thank you
Danilo

ody:On 05/09/2018 13:23, Andrea Gazzarini wrote:

Hi Danilo,
let's see if this can help you (I'm sorry for the poor debugging, I'm 
reading & writing from my mobile): the first issue should have 
something to do with synonym overlapping and since I'm very curious 
about what it is happening, I will be more precise when I will be in 
front of a laptop.


The second: I guess the main problem is the StandardTokenizer, which 
removes the ' and - symbols. That should be the reason why you don't 
have any synonym detection. You should replace it with a 
WhitespaceTokenizer but, be aware that if you do that, the apostrophe 
in the document ( ′ ) is not the same symbol ( ' ) you've used in the 
query and in the synonyms file, so you need to replace it somewhere 
(in the document and/or in the query) otherwise you won't have any 
match.


HTH
Gazza

On 05/09/2018 12:19, Danilo Tomasoni wrote:

Hello to all,

I have an issue related to synonimgraphfilter expanding the wrong 
synonims for a phrase-term at query time.


I have a dictionary with the following lines

P49902,Cytosolic purine 5'-nucleotidase,EC 3.1.3.5,Cytosolic 
5'-nucleotidase II
A8K9N1,Glucosidase\, beta\, acid 3,Cytosolic,Glucosidase\, beta\, 
acid 3,Cytosolic\, isoform CRA_b,cDNA FLJ78196\, highly similar to 
Homo sapiens glucosidase\, beta\, acid 3,cytosolic,GBA3\, 
mRNA,cDNA\, FLJ93688\, Homo sapiens glucosidase\, beta\, acid 
3,cytosolic,GBA3\, mRNA


and two documents

{"body":"8. The method of claim 6 wherein said method inhibits at 
least one 5′-nucleotidase chosen from cytosolic 5′-nucleotidase II 
(cN-II), cytosolic 5′-nucleotidase IA (cN-IA), cytosolic 
5′-nucleotidase IB (cN-IB), cytosolic 5′-nucleotidase IMA (cN-IIIA), 
cytosolic 5′-nucleotidase NIB (cN-IIIB), ecto-5′-nucleotidase (eN, 
CD73), cytosolic 5′(3′)-deoxynucleotidase (cdN) and mitochondrial 
5′(3′)-deoxynucleotidase (mdN)."}
{"body":"Trichomonosis caused by the flagellate protozoan 
Trichomonas vaginalis represents the most prevalent nonviral 
sexually transmitted disease worldwide (WHO-DRHR 2012). In women, 
the symptoms are cyclic and often worsen around the menstruation 
period. In men, trichomonosis is largely asymptomatic and these men 
are considered to be carriers of T. vaginalis (Petrin et al. 1998). 
This infection has been associated with birth outcomes (Klebanoff et 
al. 2001), infertility (Grodstein et al. 1993), cervical and 
prostate cancer (Viikki et al. 2000, Sutcliffe et al. 2012) and 
pelvic inflammatory disease (Cherpes et al. 2006). Importantly, T. 
vaginalis is a co-factor in human immunodeficiency virus 
transmission and acquisition (Sorvillo et al. 2001, Van Der Pol et 
al. 2008). Therefore, it is important to study the host-parasite 
relationship to understand T. vaginalis infection and pathogenesis. 
Colonisation of the mucosa by T. vaginalis is a complex multi-step 
process that involves distinct mechanisms (Alderete et al. 2004). 
The parasite interacts with mucin (Lehker & Sweeney 1999), adheres 
to vaginal epithelial cells (VECs) in a process mediated by adhesion 
proteins (AP120, AP65, AP51, AP33 and AP23) and undergoes dramatic 
morphological changes from a pyriform to an amoeboid form (Engbring 
& Alderete 1998, Kucknoor et al. 2005, Moreno-Brito et al. 2005). 
After adhesion to VECs, the synthesis and gene expression of 
adhesins are increased (Kucknoor et al. 2005). These mechanisms must 
be tightly regulated and iron plays a pivotal role in this 
regulation. Iron is an essential element for all living organisms, 
from the most primitive to the most complex, as a component of 
haeme, iron-sulphur clusters and a variety of proteins. Iron is 
known to contribute to biological functions such as 

Re: Solr 6.4.1: : SolrException:nfs no locks available

[Shawn Heisey]

On 9/5/2018 6:55 AM, Imran Rajjad wrote:

I am using Solr Cloud 6.4.1. After a hard restart the solr nodes are
constantly showing to be in DOWN state and would not go into recovery. I
have also deleted the write.lock files from all the replica folders, but
the problem would not go away. The error displayed at web console is : no
locks available

My replica folders reside in an nfs mount, I am using RHEL 6/CentOS6.8. Has
anyone ever faced this issue?


Lucene-based software (including Solr) does NOT work well on NFS.  NFS 
does not provide all the locking functionality that Lucene tries to use 
by default.


You're probably going to need to change the lock factory, and might even 
need to completely disable locking.  If you do disable locking, you have 
to be VERY careful to never allow more than one core or more than one 
Solr instance to try and open a core directory.  Doing so will likely 
corrupt the index.


I strongly recommend NOT using NFS storage for Solr.  In addition to 
locking problems, it also tends to be extremely slow.Use a local filesystem.


Thanks,
Shawn

Re: SynonimGraphFilter expands wrong synonims

[Danilo Tomasoni]

Hi Andrea,

thank you for your answer.

About the second question: The standardTokenizer should be applied also 
to the phrase query, so the ' and - symbols should be removed even 
there, and this should allow a match in the synonim file isn't it?


With an example:


in phrase query:

"Cytosolic 5'-nucleotidase II" -> standardTokenizer -> Cytosolic, 5, 
nucleotidase, II



in synonym parsing:

...,Cytosolic 5'-nucleotidase II,... -> standardTokenizer -> Cytosolic, 
5, nucleotidase, II



So the two graphs should match.. or I'm wrong?
Thank you
Danilo

ody:On 05/09/2018 13:23, Andrea Gazzarini wrote:

Hi Danilo,
let's see if this can help you (I'm sorry for the poor debugging, I'm 
reading & writing from my mobile): the first issue should have 
something to do with synonym overlapping and since I'm very curious 
about what it is happening, I will be more precise when I will be in 
front of a laptop.


The second: I guess the main problem is the StandardTokenizer, which 
removes the ' and - symbols. That should be the reason why you don't 
have any synonym detection. You should replace it with a 
WhitespaceTokenizer but, be aware that if you do that, the apostrophe 
in the document ( ′ ) is not the same symbol ( ' ) you've used in the 
query and in the synonyms file, so you need to replace it somewhere 
(in the document and/or in the query) otherwise you won't have any match.


HTH
Gazza

On 05/09/2018 12:19, Danilo Tomasoni wrote:

Hello to all,

I have an issue related to synonimgraphfilter expanding the wrong 
synonims for a phrase-term at query time.


I have a dictionary with the following lines

P49902,Cytosolic purine 5'-nucleotidase,EC 3.1.3.5,Cytosolic 
5'-nucleotidase II
A8K9N1,Glucosidase\, beta\, acid 3,Cytosolic,Glucosidase\, beta\, 
acid 3,Cytosolic\, isoform CRA_b,cDNA FLJ78196\, highly similar to 
Homo sapiens glucosidase\, beta\, acid 3,cytosolic,GBA3\, mRNA,cDNA\, 
FLJ93688\, Homo sapiens glucosidase\, beta\, acid 3,cytosolic,GBA3\, 
mRNA


and two documents

{"body":"8. The method of claim 6 wherein said method inhibits at 
least one 5′-nucleotidase chosen from cytosolic 5′-nucleotidase II 
(cN-II), cytosolic 5′-nucleotidase IA (cN-IA), cytosolic 
5′-nucleotidase IB (cN-IB), cytosolic 5′-nucleotidase IMA (cN-IIIA), 
cytosolic 5′-nucleotidase NIB (cN-IIIB), ecto-5′-nucleotidase (eN, 
CD73), cytosolic 5′(3′)-deoxynucleotidase (cdN) and mitochondrial 
5′(3′)-deoxynucleotidase (mdN)."}
{"body":"Trichomonosis caused by the flagellate protozoan Trichomonas 
vaginalis represents the most prevalent nonviral sexually transmitted 
disease worldwide (WHO-DRHR 2012). In women, the symptoms are cyclic 
and often worsen around the menstruation period. In men, 
trichomonosis is largely asymptomatic and these men are considered to 
be carriers of T. vaginalis (Petrin et al. 1998). This infection has 
been associated with birth outcomes (Klebanoff et al. 2001), 
infertility (Grodstein et al. 1993), cervical and prostate cancer 
(Viikki et al. 2000, Sutcliffe et al. 2012) and pelvic inflammatory 
disease (Cherpes et al. 2006). Importantly, T. vaginalis is a 
co-factor in human immunodeficiency virus transmission and 
acquisition (Sorvillo et al. 2001, Van Der Pol et al. 2008). 
Therefore, it is important to study the host-parasite relationship to 
understand T. vaginalis infection and pathogenesis. Colonisation of 
the mucosa by T. vaginalis is a complex multi-step process that 
involves distinct mechanisms (Alderete et al. 2004). The parasite 
interacts with mucin (Lehker & Sweeney 1999), adheres to vaginal 
epithelial cells (VECs) in a process mediated by adhesion proteins 
(AP120, AP65, AP51, AP33 and AP23) and undergoes dramatic 
morphological changes from a pyriform to an amoeboid form (Engbring & 
Alderete 1998, Kucknoor et al. 2005, Moreno-Brito et al. 2005). After 
adhesion to VECs, the synthesis and gene expression of adhesins are 
increased (Kucknoor et al. 2005). These mechanisms must be tightly 
regulated and iron plays a pivotal role in this regulation. Iron is 
an essential element for all living organisms, from the most 
primitive to the most complex, as a component of haeme, iron-sulphur 
clusters and a variety of proteins. Iron is known to contribute to 
biological functions such as DNA and RNA synthesis, oxygen transport 
and metabolic reactions. T. vaginalis has developed multiple iron 
uptake systems such as receptors for hololactoferrin, haemoglobin 
(HB), haemin (HM) and haeme binding as well as adhesins to 
erythrocytes and epithelial cells (Moreno-Brito et al. 2005, Ardalan 
et al. 2009). Iron plays a crucial role in the pathogenesis of 
trichomonosis by increasing cytoadherence and modulating resistance 
to complement lyses, ligation to the extracellular matrix and the 
expression of proteases (Figueroa-Angulo et al. 2012). In agreement 
with this role, the symptoms of trichomonosis worsen after 
menstruation. In addition, iron also influences nucleotide hydrolysis 
in T. 

Re: Replicas do not come up after nodes are restarted in SOLR cloud

[Shawn Heisey]

On 9/5/2018 2:55 AM, Sudip Mukherjee wrote:

I have a 2 node SOLR (7.x) cloud cluster on which I have collection with replicas ( 
replicationFactor = 2, shard = 1 ). I am seeing that the replicas do not come up ( state is 
"down")  when both nodes are restarted. From the "legend" in Graph section, I 
see that the replicas are in
"recovery failed" state.



Caused by: java.net.SocketTimeoutException: Read timed out
 at java.net.SocketInputStream.socketRead0(Native Method)
 at 
java.net.SocketInputStream.socketRead(SocketInputStream.java:116)


Have you changed the socket timeout in Solr's config?

The socket timeout for internode requests defaults to 60 seconds.  If 
something happened that prevented a Solr server from responding within 
60 seconds, then there's something *REALLY* wrong.


My best guess is that your Solr heap is too small, causing Java to spend 
almost all of its time doing garbage collection.  Or that a too-small 
heap has caused one of your servers to experience an OutOfMemoryError, 
which on non-Windows systems will result in the Solr process being killed.


Some questions in case that's not it:

How many collections do you have on this setup?

In the admin UI (Cloud tab), what hostname do your nodes show they are 
registered as?  If it's localhost, that's going to be a problem for a 
2-node cluster.


Thanks,
Shawn

Solr 6.4.1: : SolrException:nfs no locks available

[Imran Rajjad]

Hello,

I am using Solr Cloud 6.4.1. After a hard restart the solr nodes are
constantly showing to be in DOWN state and would not go into recovery. I
have also deleted the write.lock files from all the replica folders, but
the problem would not go away. The error displayed at web console is : no
locks available

My replica folders reside in an nfs mount, I am using RHEL 6/CentOS6.8. Has
anyone ever faced this issue?

regards,
Imran

-- 
I.R

user field(uf) not working with Post filter

[shruti suri]

Hi,

I am using a custom handler with edismax parser. I am using uf parameter in
the handler to restrict some fields from search.  But uf is not working with
post filter(fq). I want to restricted same fields in fq, so that people
could not filter on some fields. Please suggest how can I do that.



explicit
1
10
edismax
name title author key
*:*
*


apix
* -author -key





-
Regards
Shruti
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

user field(uf) not working with Post filter

[shruti suri]

Hi,

I am using a custom handler with edismax parser. I am using uf parameter in
the handler to restrict some fields from search.  But uf is not working with
post filter(fq). I want to restricted same fields in fq, so that people
could not filter on some fields. Please suggest how can I do that.



explicit
1
10
edismax
name title author key
*:*
*


apix
* -author -key





-
Regards
Shruti
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: SynonimGraphFilter expands wrong synonims

[Andrea Gazzarini]

Hi Danilo,
let's see if this can help you (I'm sorry for the poor debugging, I'm 
reading & writing from my mobile): the first issue should have something 
to do with synonym overlapping and since I'm very curious about what it 
is happening, I will be more precise when I will be in front of a laptop.


The second: I guess the main problem is the StandardTokenizer, which 
removes the ' and - symbols. That should be the reason why you don't 
have any synonym detection. You should replace it with a 
WhitespaceTokenizer but, be aware that if you do that, the apostrophe in 
the document ( ′ ) is not the same symbol ( ' ) you've used in the query 
and in the synonyms file, so you need to replace it somewhere (in the 
document and/or in the query) otherwise you won't have any match.


HTH
Gazza

On 05/09/2018 12:19, Danilo Tomasoni wrote:

Hello to all,

I have an issue related to synonimgraphfilter expanding the wrong 
synonims for a phrase-term at query time.


I have a dictionary with the following lines

P49902,Cytosolic purine 5'-nucleotidase,EC 3.1.3.5,Cytosolic 
5'-nucleotidase II
A8K9N1,Glucosidase\, beta\, acid 3,Cytosolic,Glucosidase\, beta\, acid 
3,Cytosolic\, isoform CRA_b,cDNA FLJ78196\, highly similar to Homo 
sapiens glucosidase\, beta\, acid 3,cytosolic,GBA3\, mRNA,cDNA\, 
FLJ93688\, Homo sapiens glucosidase\, beta\, acid 3,cytosolic,GBA3\, mRNA


and two documents

{"body":"8. The method of claim 6 wherein said method inhibits at 
least one 5′-nucleotidase chosen from cytosolic 5′-nucleotidase II 
(cN-II), cytosolic 5′-nucleotidase IA (cN-IA), cytosolic 
5′-nucleotidase IB (cN-IB), cytosolic 5′-nucleotidase IMA (cN-IIIA), 
cytosolic 5′-nucleotidase NIB (cN-IIIB), ecto-5′-nucleotidase (eN, 
CD73), cytosolic 5′(3′)-deoxynucleotidase (cdN) and mitochondrial 
5′(3′)-deoxynucleotidase (mdN)."}
{"body":"Trichomonosis caused by the flagellate protozoan Trichomonas 
vaginalis represents the most prevalent nonviral sexually transmitted 
disease worldwide (WHO-DRHR 2012). In women, the symptoms are cyclic 
and often worsen around the menstruation period. In men, trichomonosis 
is largely asymptomatic and these men are considered to be carriers of 
T. vaginalis (Petrin et al. 1998). This infection has been associated 
with birth outcomes (Klebanoff et al. 2001), infertility (Grodstein et 
al. 1993), cervical and prostate cancer (Viikki et al. 2000, Sutcliffe 
et al. 2012) and pelvic inflammatory disease (Cherpes et al. 2006). 
Importantly, T. vaginalis is a co-factor in human immunodeficiency 
virus transmission and acquisition (Sorvillo et al. 2001, Van Der Pol 
et al. 2008). Therefore, it is important to study the host-parasite 
relationship to understand T. vaginalis infection and pathogenesis. 
Colonisation of the mucosa by T. vaginalis is a complex multi-step 
process that involves distinct mechanisms (Alderete et al. 2004). The 
parasite interacts with mucin (Lehker & Sweeney 1999), adheres to 
vaginal epithelial cells (VECs) in a process mediated by adhesion 
proteins (AP120, AP65, AP51, AP33 and AP23) and undergoes dramatic 
morphological changes from a pyriform to an amoeboid form (Engbring & 
Alderete 1998, Kucknoor et al. 2005, Moreno-Brito et al. 2005). After 
adhesion to VECs, the synthesis and gene expression of adhesins are 
increased (Kucknoor et al. 2005). These mechanisms must be tightly 
regulated and iron plays a pivotal role in this regulation. Iron is an 
essential element for all living organisms, from the most primitive to 
the most complex, as a component of haeme, iron-sulphur clusters and a 
variety of proteins. Iron is known to contribute to biological 
functions such as DNA and RNA synthesis, oxygen transport and 
metabolic reactions. T. vaginalis has developed multiple iron uptake 
systems such as receptors for hololactoferrin, haemoglobin (HB), 
haemin (HM) and haeme binding as well as adhesins to erythrocytes and 
epithelial cells (Moreno-Brito et al. 2005, Ardalan et al. 2009). Iron 
plays a crucial role in the pathogenesis of trichomonosis by 
increasing cytoadherence and modulating resistance to complement 
lyses, ligation to the extracellular matrix and the expression of 
proteases (Figueroa-Angulo et al. 2012). In agreement with this role, 
the symptoms of trichomonosis worsen after menstruation. In addition, 
iron also influences nucleotide hydrolysis in T. vaginalis (Tasca et 
al. 2005, de Jesus et al. 2006). The extracellular concentrations of 
ATP and adenosine can markedly increase under several conditions such 
as inflammation and hypoxia as well as in the presence of pathogens 
(Robson et al. 2006, Sansom 2012). In the extracellular medium, these 
nucleotides can act as immunomodulators by triggering immunological 
effects. Extracellular ATP acts as a proinflammatory immune-mediator 
by triggering multiple immunological effects on cell types such as 
neutrophils, macrophages, dendritic cells and lymphocytes (Bours et 
al. 2006). In this sense, ATP and 

SynonimGraphFilter expands wrong synonims

[Danilo Tomasoni]

Hello to all,

I have an issue related to synonimgraphfilter expanding the wrong 
synonims for a phrase-term at query time.


I have a dictionary with the following lines

P49902,Cytosolic purine 5'-nucleotidase,EC 3.1.3.5,Cytosolic 5'-nucleotidase II
A8K9N1,Glucosidase\, beta\, acid 3,Cytosolic,Glucosidase\, beta\, acid 
3,Cytosolic\, isoform CRA_b,cDNA FLJ78196\, highly similar to Homo sapiens 
glucosidase\, beta\, acid 3,cytosolic,GBA3\, mRNA,cDNA\, FLJ93688\, Homo 
sapiens glucosidase\, beta\, acid 3,cytosolic,GBA3\, mRNA

and two documents

{"body":"8. The method of claim 6 wherein said method inhibits at least one 
5′-nucleotidase chosen from cytosolic 5′-nucleotidase II (cN-II), 
cytosolic 5′-nucleotidase IA (cN-IA), cytosolic 5′-nucleotidase IB 
(cN-IB), cytosolic 5′-nucleotidase IMA (cN-IIIA), cytosolic 
5′-nucleotidase NIB (cN-IIIB), ecto-5′-nucleotidase (eN, CD73), 
cytosolic 5′(3′)-deoxynucleotidase (cdN) and mitochondrial 
5′(3′)-deoxynucleotidase (mdN)."}
{"body":"Trichomonosis caused by the flagellate protozoan Trichomonas vaginalis 
represents the most prevalent nonviral sexually transmitted disease 
worldwide (WHO-DRHR 2012). In women, the symptoms are cyclic and often 
worsen around the menstruation period. In men, trichomonosis is largely 
asymptomatic and these men are considered to be carriers of T. vaginalis 
(Petrin et al. 1998). This infection has been associated with birth 
outcomes (Klebanoff et al. 2001), infertility (Grodstein et al. 1993), 
cervical and prostate cancer (Viikki et al. 2000, Sutcliffe et al. 2012) 
and pelvic inflammatory disease (Cherpes et al. 2006). Importantly, T. 
vaginalis is a co-factor in human immunodeficiency virus transmission 
and acquisition (Sorvillo et al. 2001, Van Der Pol et al. 2008). 
Therefore, it is important to study the host-parasite relationship to 
understand T. vaginalis infection and pathogenesis. Colonisation of the 
mucosa by T. vaginalis is a complex multi-step process that involves 
distinct mechanisms (Alderete et al. 2004). The parasite interacts with 
mucin (Lehker & Sweeney 1999), adheres to vaginal epithelial cells 
(VECs) in a process mediated by adhesion proteins (AP120, AP65, AP51, 
AP33 and AP23) and undergoes dramatic morphological changes from a 
pyriform to an amoeboid form (Engbring & Alderete 1998, Kucknoor et al. 
2005, Moreno-Brito et al. 2005). After adhesion to VECs, the synthesis 
and gene expression of adhesins are increased (Kucknoor et al. 2005). 
These mechanisms must be tightly regulated and iron plays a pivotal role 
in this regulation. Iron is an essential element for all living 
organisms, from the most primitive to the most complex, as a component 
of haeme, iron-sulphur clusters and a variety of proteins. Iron is known 
to contribute to biological functions such as DNA and RNA synthesis, 
oxygen transport and metabolic reactions. T. vaginalis has developed 
multiple iron uptake systems such as receptors for hololactoferrin, 
haemoglobin (HB), haemin (HM) and haeme binding as well as adhesins to 
erythrocytes and epithelial cells (Moreno-Brito et al. 2005, Ardalan et 
al. 2009). Iron plays a crucial role in the pathogenesis of 
trichomonosis by increasing cytoadherence and modulating resistance to 
complement lyses, ligation to the extracellular matrix and the 
expression of proteases (Figueroa-Angulo et al. 2012). In agreement with 
this role, the symptoms of trichomonosis worsen after menstruation. In 
addition, iron also influences nucleotide hydrolysis in T. vaginalis 
(Tasca et al. 2005, de Jesus et al. 2006). The extracellular 
concentrations of ATP and adenosine can markedly increase under several 
conditions such as inflammation and hypoxia as well as in the presence 
of pathogens (Robson et al. 2006, Sansom 2012). In the extracellular 
medium, these nucleotides can act as immunomodulators by triggering 
immunological effects. Extracellular ATP acts as a proinflammatory 
immune-mediator by triggering multiple immunological effects on cell 
types such as neutrophils, macrophages, dendritic cells and lymphocytes 
(Bours et al. 2006). In this sense, ATP and adenosine concentrations in 
the extracellular compartment are controlled by ectoenzymes, including 
those of the nucleoside triphosphate diphosphohydrolase (NTPDase) (EC: 
3.1.4.1) family, which hydrolyze tri and diphosphates and 
ecto-5’-nucleotidase (EC: 3.1.3.5), which hydrolyses monophosphates 
(Zimmermann 2001). Considering that de novo nucleotide synthesis is 
absent in T. vaginalis (Heyworth et al. 1982, 1984), this enzyme cascade 
is important as a source of the precursor adenosine for purine synthesis 
in the parasite (Munagala & Wang 2003). Extracellular nucleotide 
metabolism has been characterised in several parasite species such as 
Toxoplasma gondii, Schistosoma mansoni, Leishmania spp, Trypanosoma 
cruzi, Acanthamoeba, Entamoeba histolytica, Giardia lamblia and fungi, 
Saccharomyces cerevisiae, Cryptococcus neoformans, Candida 

Re: Implementing NeuralNetworkModel RankNet in Solr

[Zheng Lin Edwin Yeo]

By the way, I am doing this on Solr 7.4.0. Sorry, forgot to include that in
my earlier post.

On Wed, 5 Sep 2018 at 17:39, Zheng Lin Edwin Yeo 
wrote:

> Hi,
>
> I am looking at implementing the NeuralNetworkModel for the Solr LTR which
> uses the "pairwise" approach, as it seems to be a better option than the
> LinearModel, which uses the "pointwise" approach..
>
> I am following the setup from the following site:
> https://github.com/airalcorn2/Solr-LTR#RankNet to setup RankNet. However,
> I am not sure of the codes in the example, like which code should we run
> first?
>
> The first code below seems to be already calling the model, but we have
> not train and upload the model yet, and those codes are in the later
> section.
>
> url = "http://localhost:8983/solr/test/query;
> url += "?q={0}=text={{!ltr model=my_efi_model ".format(text)
> url += "efi.text_a='{0}' efi.text_b='{1}' efi.text='{2}'}}".format(text_a, 
> text_b, text)
> url += "=id,score,[features]={1}".format(text, RERANK)
>
>
> Anyone has tried this before and know of the correct sequence to setup the
> RankNet successfully?
>
> Regards,
> Edwin
>
>

Implementing NeuralNetworkModel RankNet in Solr

[Zheng Lin Edwin Yeo]

Hi,

I am looking at implementing the NeuralNetworkModel for the Solr LTR which
uses the "pairwise" approach, as it seems to be a better option than the
LinearModel, which uses the "pointwise" approach..

I am following the setup from the following site:
https://github.com/airalcorn2/Solr-LTR#RankNet to setup RankNet. However, I
am not sure of the codes in the example, like which code should we run
first?

The first code below seems to be already calling the model, but we have not
train and upload the model yet, and those codes are in the later section.

url = "http://localhost:8983/solr/test/query;
url += "?q={0}=text={{!ltr model=my_efi_model ".format(text)
url += "efi.text_a='{0}' efi.text_b='{1}'
efi.text='{2}'}}".format(text_a, text_b, text)
url += "=id,score,[features]={1}".format(text, RERANK)


Anyone has tried this before and know of the correct sequence to setup the
RankNet successfully?

Regards,
Edwin

Replicas do not come up after nodes are restarted in SOLR cloud

[Sudip Mukherjee]

Hi,

I have a 2 node SOLR (7.x) cloud cluster on which I have collection with 
replicas ( replicationFactor = 2, shard = 1 ). I am seeing that the replicas do 
not come up ( state is "down")  when both nodes are restarted. From the 
"legend" in Graph section, I see that the replicas are in
"recovery failed" state.

Below errors are seen :

2018-09-05 14:07:40.157 ERROR (qtp1347137144-10094) [   ] 
org.apache.solr.servlet.HttpSolrCall null:org.apache.solr.common.SolrException: 
Error trying to proxy request for url: http://localhost/solr/metadata/select
at 
org.apache.solr.servlet.HttpSolrCall.remoteQuery(HttpSolrCall.java:646)
at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:500)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)

Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at 
java.net.SocketInputStream.socketRead(SocketInputStream.java:116)

There are other non-replicated collections and they seem to be "active" and 
able to query from SOLR UI.


Is this something that we expect when all nodes are restarted? How can we bring 
the replicas back online from "recovery failed" state?


Thanks,
Sudip Mukherjee



***Legal Disclaimer***
"This communication may contain confidential and privileged material for the
sole use of the intended recipient. Any unauthorized review, use or distribution
by others is strictly prohibited. If you have received the message by mistake,
please advise the sender by reply email and delete the message. Thank you."
**