/solr/admin/metrics fails when there are transient cores

[nandakishorek]

My environment is as follows
 * Solr 7.2.1 in standalone mode.
 * 32GB heap
 * 150 cores with data getting continuously ingested to ~10 cores and all of
the cores queried.
 * transient cache size is set to 30.

The solr.xml is as follows



  32
  true
  ${configSetBaseDir:configsets}

  
${socketTimeout:60}
${connTimeout:6}
  


I get the following error when I request for "/solr/admin/metrics".
{
"responseHeader": {
"status": 500,
"QTime": 31
},
"error": {
"msg": "Already closed",
"trace": "org.apache.lucene.store.AlreadyClosedException: Already
closed\n\tat
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:337)\n\tat
org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:351)\n\tat
org.apache.solr.core.SolrCore.getIndexDir(SolrCore.java:330)\n\tat
org.apache.solr.handler.ReplicationHandler.lambda$initializeMetrics$5(ReplicationHandler.java:849)\n\tat
org.apache.solr.util.stats.MetricUtils.convertGauge(MetricUtils.java:488)\n\tat
org.apache.solr.util.stats.MetricUtils.convertMetric(MetricUtils.java:274)\n\tat
org.apache.solr.util.stats.MetricUtils.lambda$toMaps$4(MetricUtils.java:213)\n\tat
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)\n\tat
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)\n\tat
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)\n\tat
java.util.TreeMap$KeySpliterator.forEachRemaining(TreeMap.java:2746)\n\tat
java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)\n\tat
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)\n\tat
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)\n\tat
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)\n\tat
java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)\n\tat
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)\n\tat
org.apache.solr.util.stats.MetricUtils.toMaps(MetricUtils.java:211)\n\tat
org.apache.solr.handler.admin.MetricsHandler.handleRequestBody(MetricsHandler.java:108)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)\n\tat
org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)\n\tat
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)\n\tat
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:497)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)\n\tat
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1751)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)\n\tat
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat
org.eclipse.jetty.server.Server.handle(Server.java:534)\n\tat
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)\n\tat
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)\n\tat
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)\n\tat
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)\n\tat
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\n\tat
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)\n\tat
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)\n\tat
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)\n\tat

Re: Solr OpenNLP named entity extraction

[Jerome Yang]

Hi Steve,

Put models under " ${solr.solr.home}/lib/ " is not working.
I check the "ZkSolrResourceLoader" seems it will first try to find modes in
config set.
If not find, then it uses class loader to load from resources.

Regards,
Jerome

On Tue, Jul 10, 2018 at 9:58 AM Jerome Yang  wrote:

> Thanks Steve!
>
>
> On Tue, Jul 10, 2018 at 5:20 AM Steve Rowe  wrote:
>
>> Hi Jerome,
>>
>> See the ref guide[1] for a writeup of how to enable uploading files
>> larger than 1MB into ZooKeeper.
>>
>> Local storage should also work - have you tried placing OpenNLP model
>> files in ${solr.solr.home}/lib/ ? - make sure you do the same on each node.
>>
>> [1]
>> https://lucene.apache.org/solr/guide/7_4/setting-up-an-external-zookeeper-ensemble.html#increasing-the-file-size-limit
>>
>> --
>> Steve
>> www.lucidworks.com
>>
>> > On Jul 9, 2018, at 12:50 AM, Jerome Yang  wrote:
>> >
>> > Hi guys,
>> >
>> > In Solrcloud mode, where to put the OpenNLP models?
>> > Upload to zookeeper?
>> > As I test on solr 7.3.1, seems absolute path on local host is not
>> working.
>> > And can not upload into zookeeper if the model size exceed 1M.
>> >
>> > Regards,
>> > Jerome
>> >
>> > On Wed, Apr 18, 2018 at 9:54 AM Steve Rowe  wrote:
>> >
>> >> Hi Alexey,
>> >>
>> >> First, thanks for moving the conversation to the mailing list.
>> Discussion
>> >> of usage problems should take place here rather than in JIRA.
>> >>
>> >> I locally set up Solr 7.3 similarly to you and was able to get things
>> to
>> >> work.
>> >>
>> >> Problems with your setup:
>> >>
>> >> 1. Your update chain is missing the Log and Run update processors at
>> the
>> >> end (I see these are missing from the example in the javadocs for the
>> >> OpenNLP NER update processor; I’ll fix that):
>> >>
>> >> 
>> >> 
>> >>
>> >>   The Log update processor isn’t strictly necessary, but, from <
>> >>
>> https://lucene.apache.org/solr/guide/7_3/update-request-processors.html#custom-update-request-processor-chain
>> >>> :
>> >>
>> >>   Do not forget to add RunUpdateProcessorFactory at the end of any
>> >>   chains you define in solrconfig.xml. Otherwise update requests
>> >>   processed by that chain will not actually affect the indexed
>> data.
>> >>
>> >> 2. Your example document is missing an “id” field.
>> >>
>> >> 3. For whatever reason, the pre-trained model "en-ner-person.bin"
>> doesn’t
>> >> extract anything from text “This is Steve Jobs 2”.  It will extract
>> “Steve
>> >> Jobs” from text “This is Steve Jobs in white” e.g. though.
>> >>
>> >> 4. (Not a problem necessarily) You may want to use a multi-valued
>> “string”
>> >> field for the “dest” field in your update chain, e.g. “people_str”
>> (“*_str”
>> >> in the default configset is so configured).
>> >>
>> >> --
>> >> Steve
>> >> www.lucidworks.com
>> >>
>> >>> On Apr 17, 2018, at 8:23 AM, Alexey Ponomarenko <
>> alex1989s...@gmail.com>
>> >> wrote:
>> >>>
>> >>> Hi once more I am trying to implement named entities extraction using
>> >> this
>> >>> manual
>> >>>
>> >>
>> https://lucene.apache.org/solr/7_3_0//solr-analysis-extras/org/apache/solr/update/processor/OpenNLPExtractNamedEntitiesUpdateProcessorFactory.html
>> >>>
>> >>> I am modified solrconfig.xml like this:
>> >>>
>> >>> 
>> >>>  > >> class="solr.OpenNLPExtractNamedEntitiesUpdateProcessorFactory">
>> >>>opennlp/en-ner-person.bin
>> >>>text_opennlp
>> >>>description_en
>> >>>content
>> >>>  
>> >>> 
>> >>>
>> >>> But when I was trying to add data using:
>> >>>
>> >>> *request:*
>> >>>
>> >>> POST
>> >>>
>> >>
>> http://localhost:8983/solr/numberplate/update?version=2.2=xml=multiple-extract
>> >>>
>> >>> This is Steve Jobs 2
>> >>> This is text 2> >>> name="content">This is text for content 2
>> >>>
>> >>> *response*
>> >>>
>> >>> 
>> >>> 
>> >>>   
>> >>>   0
>> >>>   3
>> >>>   
>> >>> 
>> >>>
>> >>> But I don't see any data inserted to *content* field and in any other
>> >> field.
>> >>>
>> >>> *If you need some additional data I can provide it.*
>> >>>
>> >>> Can you help me? What have I done wrong?
>> >>
>> >>
>> >
>> > --
>> > Pivotal Greenplum | Pivotal Software, Inc. 
>>
>>
>
> --
>  Pivotal Greenplum | Pivotal Software, Inc. 
>
>

-- 
 Pivotal Greenplum | Pivotal Software, Inc. 

Re: Solr OpenNLP named entity extraction

[Jerome Yang]

Thanks Steve!


On Tue, Jul 10, 2018 at 5:20 AM Steve Rowe  wrote:

> Hi Jerome,
>
> See the ref guide[1] for a writeup of how to enable uploading files larger
> than 1MB into ZooKeeper.
>
> Local storage should also work - have you tried placing OpenNLP model
> files in ${solr.solr.home}/lib/ ? - make sure you do the same on each node.
>
> [1]
> https://lucene.apache.org/solr/guide/7_4/setting-up-an-external-zookeeper-ensemble.html#increasing-the-file-size-limit
>
> --
> Steve
> www.lucidworks.com
>
> > On Jul 9, 2018, at 12:50 AM, Jerome Yang  wrote:
> >
> > Hi guys,
> >
> > In Solrcloud mode, where to put the OpenNLP models?
> > Upload to zookeeper?
> > As I test on solr 7.3.1, seems absolute path on local host is not
> working.
> > And can not upload into zookeeper if the model size exceed 1M.
> >
> > Regards,
> > Jerome
> >
> > On Wed, Apr 18, 2018 at 9:54 AM Steve Rowe  wrote:
> >
> >> Hi Alexey,
> >>
> >> First, thanks for moving the conversation to the mailing list.
> Discussion
> >> of usage problems should take place here rather than in JIRA.
> >>
> >> I locally set up Solr 7.3 similarly to you and was able to get things to
> >> work.
> >>
> >> Problems with your setup:
> >>
> >> 1. Your update chain is missing the Log and Run update processors at the
> >> end (I see these are missing from the example in the javadocs for the
> >> OpenNLP NER update processor; I’ll fix that):
> >>
> >> 
> >> 
> >>
> >>   The Log update processor isn’t strictly necessary, but, from <
> >>
> https://lucene.apache.org/solr/guide/7_3/update-request-processors.html#custom-update-request-processor-chain
> >>> :
> >>
> >>   Do not forget to add RunUpdateProcessorFactory at the end of any
> >>   chains you define in solrconfig.xml. Otherwise update requests
> >>   processed by that chain will not actually affect the indexed data.
> >>
> >> 2. Your example document is missing an “id” field.
> >>
> >> 3. For whatever reason, the pre-trained model "en-ner-person.bin"
> doesn’t
> >> extract anything from text “This is Steve Jobs 2”.  It will extract
> “Steve
> >> Jobs” from text “This is Steve Jobs in white” e.g. though.
> >>
> >> 4. (Not a problem necessarily) You may want to use a multi-valued
> “string”
> >> field for the “dest” field in your update chain, e.g. “people_str”
> (“*_str”
> >> in the default configset is so configured).
> >>
> >> --
> >> Steve
> >> www.lucidworks.com
> >>
> >>> On Apr 17, 2018, at 8:23 AM, Alexey Ponomarenko <
> alex1989s...@gmail.com>
> >> wrote:
> >>>
> >>> Hi once more I am trying to implement named entities extraction using
> >> this
> >>> manual
> >>>
> >>
> https://lucene.apache.org/solr/7_3_0//solr-analysis-extras/org/apache/solr/update/processor/OpenNLPExtractNamedEntitiesUpdateProcessorFactory.html
> >>>
> >>> I am modified solrconfig.xml like this:
> >>>
> >>> 
> >>>   >> class="solr.OpenNLPExtractNamedEntitiesUpdateProcessorFactory">
> >>>opennlp/en-ner-person.bin
> >>>text_opennlp
> >>>description_en
> >>>content
> >>>  
> >>> 
> >>>
> >>> But when I was trying to add data using:
> >>>
> >>> *request:*
> >>>
> >>> POST
> >>>
> >>
> http://localhost:8983/solr/numberplate/update?version=2.2=xml=multiple-extract
> >>>
> >>> This is Steve Jobs 2
> >>> This is text 2 >>> name="content">This is text for content 2
> >>>
> >>> *response*
> >>>
> >>> 
> >>> 
> >>>   
> >>>   0
> >>>   3
> >>>   
> >>> 
> >>>
> >>> But I don't see any data inserted to *content* field and in any other
> >> field.
> >>>
> >>> *If you need some additional data I can provide it.*
> >>>
> >>> Can you help me? What have I done wrong?
> >>
> >>
> >
> > --
> > Pivotal Greenplum | Pivotal Software, Inc. 
>
>

-- 
 Pivotal Greenplum | Pivotal Software, Inc. 

Re: AddReplica to shard with lowest node count

[Duncan, Adam]

Thanks for your responses.

I’ve tried to get more familiar with the Autoscaling API. I’ve applied a 
nodeAdded trigger but I’m stuck trying to think of a cluster policy that would 
suit my scenario; something like “All new nodes need must have one replica from 
each available collection”
Is this possible? Or is that the point you were getting at by saying my 
use-case isn’t supported, Shalin? 

Regards,
Adam

On 7/4/18, 9:18 PM, "Shalin Shekhar Mangar"  wrote:

The rule based replica placement was deprecated. The autoscaling APIs are
the way to go. Please see
http://lucene.apache.org/solr/guide/7_3/solrcloud-autoscaling.html

Your use-case is interesting. By default, the trigger for nodeAdded event
will move replicas from the most loaded nodes to the new node. That does
not take care of your use-case. Can you please open a Jira to add this
feature?

On Thu, Jul 5, 2018 at 6:45 AM Gus Heck  wrote:

> Perhaps the rule based replica placement stuff would do the trick?
>
> https://lucene.apache.org/solr/guide/7_3/rule-based-replica-placement.html
>
> I haven't used it myself but I've seen lots of work going into it 
lately...
>
> On Wed, Jul 4, 2018 at 12:35 PM, Duncan, Adam 
> wrote:
>
> > Hi all,
> >
> > Our team use Solrcloud for Solr 5.1 and are investigating an upgrade to
> 7.3
> > Currently we have a working scale-up approach for adding a new server to
> > the cluster beyond the initial collection creation.
> > We’ve automated the install of Solr on new servers and, following that,
> we
> > register the new instance with zookeeper so that the server will be
> > included in the list of live nodes.
> > Finally we use the CoreAdmin API ‘Create’ command to associate the new
> > node with our collection. Solr 5.1's CoreAdmin Create command would
> > conveniently auto-assign the new node to the shard with the least nodes.
> >
> > In Solr 7.3, the CoreAdmin API documentation warns us not to use the
> > Create command with SolrCloud.
> > We tried 7.3’s CoreAdmin API Create command regardless and,
> > unsurprisingly, it did not work.
> > The 7.3 documentation suggests we use the Collections API AddReplica
> > command.The problem with AddReplica is that it expects us to specify the
> > shard name.
> > This is unfortunate as it makes it hard for us to keep shards balanced.
> It
> > puts the onus on us to work out the least populated shard via a call to
> the
> > cluster status endpoint.
> > With that we now face the problem managing this correctly when scaling 
up
> > multiple servers at once.
> >
> > Are we missing something here? Is there really no way for a node to be
> > auto-assigned to a shard in 7.3?
> > And if so, are there any recommendations for an approach to reliably
> doing
> > this ourselves?
> >
> > Thanks!
> > Adam
> >
>
>
>
> --
> http://www.the111shift.com
>


-- 
Regards,
Shalin Shekhar Mangar.

Re: Solr OpenNLP named entity extraction

[Steve Rowe]

Hi Jerome,

See the ref guide[1] for a writeup of how to enable uploading files larger than 
1MB into ZooKeeper.

Local storage should also work - have you tried placing OpenNLP model files in 
${solr.solr.home}/lib/ ? - make sure you do the same on each node.

[1] 
https://lucene.apache.org/solr/guide/7_4/setting-up-an-external-zookeeper-ensemble.html#increasing-the-file-size-limit

--
Steve
www.lucidworks.com

> On Jul 9, 2018, at 12:50 AM, Jerome Yang  wrote:
> 
> Hi guys,
> 
> In Solrcloud mode, where to put the OpenNLP models?
> Upload to zookeeper?
> As I test on solr 7.3.1, seems absolute path on local host is not working.
> And can not upload into zookeeper if the model size exceed 1M.
> 
> Regards,
> Jerome
> 
> On Wed, Apr 18, 2018 at 9:54 AM Steve Rowe  wrote:
> 
>> Hi Alexey,
>> 
>> First, thanks for moving the conversation to the mailing list.  Discussion
>> of usage problems should take place here rather than in JIRA.
>> 
>> I locally set up Solr 7.3 similarly to you and was able to get things to
>> work.
>> 
>> Problems with your setup:
>> 
>> 1. Your update chain is missing the Log and Run update processors at the
>> end (I see these are missing from the example in the javadocs for the
>> OpenNLP NER update processor; I’ll fix that):
>> 
>> 
>> 
>> 
>>   The Log update processor isn’t strictly necessary, but, from <
>> https://lucene.apache.org/solr/guide/7_3/update-request-processors.html#custom-update-request-processor-chain
>>> :
>> 
>>   Do not forget to add RunUpdateProcessorFactory at the end of any
>>   chains you define in solrconfig.xml. Otherwise update requests
>>   processed by that chain will not actually affect the indexed data.
>> 
>> 2. Your example document is missing an “id” field.
>> 
>> 3. For whatever reason, the pre-trained model "en-ner-person.bin" doesn’t
>> extract anything from text “This is Steve Jobs 2”.  It will extract “Steve
>> Jobs” from text “This is Steve Jobs in white” e.g. though.
>> 
>> 4. (Not a problem necessarily) You may want to use a multi-valued “string”
>> field for the “dest” field in your update chain, e.g. “people_str” (“*_str”
>> in the default configset is so configured).
>> 
>> --
>> Steve
>> www.lucidworks.com
>> 
>>> On Apr 17, 2018, at 8:23 AM, Alexey Ponomarenko 
>> wrote:
>>> 
>>> Hi once more I am trying to implement named entities extraction using
>> this
>>> manual
>>> 
>> https://lucene.apache.org/solr/7_3_0//solr-analysis-extras/org/apache/solr/update/processor/OpenNLPExtractNamedEntitiesUpdateProcessorFactory.html
>>> 
>>> I am modified solrconfig.xml like this:
>>> 
>>> 
>>>  > class="solr.OpenNLPExtractNamedEntitiesUpdateProcessorFactory">
>>>opennlp/en-ner-person.bin
>>>text_opennlp
>>>description_en
>>>content
>>>  
>>> 
>>> 
>>> But when I was trying to add data using:
>>> 
>>> *request:*
>>> 
>>> POST
>>> 
>> http://localhost:8983/solr/numberplate/update?version=2.2=xml=multiple-extract
>>> 
>>> This is Steve Jobs 2
>>> This is text 2>> name="content">This is text for content 2
>>> 
>>> *response*
>>> 
>>> 
>>> 
>>>   
>>>   0
>>>   3
>>>   
>>> 
>>> 
>>> But I don't see any data inserted to *content* field and in any other
>> field.
>>> 
>>> *If you need some additional data I can provide it.*
>>> 
>>> Can you help me? What have I done wrong?
>> 
>> 
> 
> -- 
> Pivotal Greenplum | Pivotal Software, Inc. 

Re: Sum and aggregation on nested documents field

[Mikhail Khludnev]

If you have  TTC_i as a function query on child level, and apply
score=total on the parent query.

On Mon, Jul 9, 2018 at 10:11 AM jeebix  wrote:

> Hello everybody,
>
> First, an example of a parent document with 2 nested documents :
>
> {
> "id":"10891.10893.142693",
> "parent_i":10891,
> "asso_i":10893,
> "personne_i":142693,
> "etat_technique_s":"avec_documents",
> "zone_scolaire_s":"B",
> "territoire_s":"France Métropolitaine",
> "region_s":"PROVENCE-ALPES-COTE D'AZUR",
> "departement_s":"13 BOUCHES-DU-RHONE",
> "postal_country_s":"FR",
> "asso_country_s":"FRANCE",
> "object_type_s":"contact",
> "date_derni_re_commande_dt":"2017-11-21T00:00:00Z",
> "_version_":1605492468358316033,
> "_childDocuments_":[
> {
>   "kit_sans_suite":["false"],
>   "TTC_i":152,
>   "type_cde_s":"KIT",
>   "object_type":["order"],
>   "statut_s":"V",
>   "campagne_s":"G",
>   "date_dt":"2017-09-28T00:00:00Z",
>   "id":"G15608",
>   "enseigne_s":"SAV",
>   "gamme":["CATALOGUES > Catalogues Saveurs",
> "KITS > Kits Saveurs"]},
> {
>   "object_type":["order"],
>   "TTC_i":2793,
>   "kit_sans_suite":["false"],
>   "type_cde_s":"CDE",
>   "statut_s":"V",
>   "campagne_s":"G",
>   "date_dt":"2017-11-21T00:00:00Z",
>   "id":"G43625",
>   "enseigne_s":"SAV",
>  }
> }
>
> And i would like to get a sum of TTC_i field, by parent document... I tried
> with stats, pivot, without any results...
>
> Thanks for your help.
>
> Best
> JB
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>


-- 
Sincerely yours
Mikhail Khludnev

Solr Cloud Intermittent Backup Failure

[jwrenn]

Hello Everyone,

I have a Solr Cloud cluster running 6.4.2 supporting a large Sitecore
install. As part of our CI/CD pipeline we deploy many times a day to a
blue/green set-up in AWS. In order to keep our Solr collections in sync
across these deployments we create a temporary snapshot at the beginning of
the deployment. Each pod refers to their indexes using collection aliases
e.g._green vs _blue. During the deployment, we can
switch where these aliases are pointed so that essentially we "pause" an
index by pointing the alias at a snapshot version of the collection. We can
then "resume" indexing by pointing the collection alias back to the live
index. This exposes a bug ( SOLR-11616
  ) that causes our
deployments to break and get stuck until someone manually goes to recreate
the affected collections. According to that ticket it was fixed and patched
in Solr 7.2 and on. Unfortunately we cannot upgrade as Solr 7.x is not
compatible with our application (Sitecore CMS).

It seems to occur randomly and I haven't been able to get a solid repro case
yet. I did notice that forcing a leader election in the cloud can heal the
problem (by restarting the current leader host). I wasn't able to figure out
how to change the leader in a satisfactory way via the Solr API so I would
just have to cycle all of my Solr Cloud nodes in order to automate a
solution, which isn't ideal.

Has anyone else wrestled with that bug or have a suggestion for how to
minimize the impact? 

Thanks,
Joe

*Here are some of the error messages:*

org.apache.solr.common.SolrException: Exception while restoring the backup
index
at org.apache.solr.handler.RestoreCore.doRestore(RestoreCore.java:130)
at
org.apache.solr.handler.admin.RestoreCoreOp.execute(RestoreCoreOp.java:65)
at
org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377)
at
org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:379)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:165)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:166)
at
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:445)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:296)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.nio.file.NoSuchFileException:

Register now for ApacheCon and save $250

[Rich Bowen]

Greetings, Apache software enthusiasts!

(You’re getting this because you’re on one or more dev@ or users@ lists 
for some Apache Software Foundation project.)


ApacheCon North America, in Montreal, is now just 80 days away, and 
early bird prices end in just two weeks - on July 21. Prices will be 
going up from $550 to $800 so register NOW to save $250, at 
http://apachecon.com/acna18


And don’t forget to reserve your hotel room. We have negotiated a 
special rate and the room block closes August 24. 
http://www.apachecon.com/acna18/venue.html


Our schedule includes over 100 talks and we’ll be featuring talks from 
dozens of ASF projects.,  We have inspiring keynotes from some of the 
brilliant members of our community and the wider tech space, including:


 * Myrle Krantz, PMC chair for Apache Fineract, and leader in the open 
source financing space
 * Cliff Schmidt, founder of Literacy Bridge (now Amplio) and creator 
of the Talking Book project

 * Bridget Kromhout, principal cloud developer advocate at Microsoft
 * Euan McLeod, Comcast engineer, and pioneer in streaming video

We’ll also be featuring tracks for Geospatial science, Tomcat, 
Cloudstack, and Big Data, as well as numerous other fields where Apache 
software is leading the way. See the full schedule at 
http://apachecon.com/acna18/schedule.html


As usual we’ll be running our Apache BarCamp, the traditional ApacheCon 
Hackathon, and the Wednesday evening Lighting Talks, too, so you’ll want 
to be there.


Register today at http://apachecon.com/acna18 and we’ll see you in Montreal!

--
Rich Bowen
VP, Conferences, The Apache Software Foundation
h...@apachecon.com
@ApacheCon

Sum and aggregation on nested documents field

[jeebix]

Hello everybody,

First, an example of a parent document with 2 nested documents :

{
"id":"10891.10893.142693",
"parent_i":10891,
"asso_i":10893,
"personne_i":142693,
"etat_technique_s":"avec_documents",
"zone_scolaire_s":"B",
"territoire_s":"France Métropolitaine",
"region_s":"PROVENCE-ALPES-COTE D'AZUR",
"departement_s":"13 BOUCHES-DU-RHONE",
"postal_country_s":"FR",
"asso_country_s":"FRANCE",
"object_type_s":"contact",
"date_derni_re_commande_dt":"2017-11-21T00:00:00Z",
"_version_":1605492468358316033,
"_childDocuments_":[
{
  "kit_sans_suite":["false"],
  "TTC_i":152,
  "type_cde_s":"KIT",
  "object_type":["order"],
  "statut_s":"V",
  "campagne_s":"G",
  "date_dt":"2017-09-28T00:00:00Z",
  "id":"G15608",
  "enseigne_s":"SAV",
  "gamme":["CATALOGUES > Catalogues Saveurs",
"KITS > Kits Saveurs"]},
{
  "object_type":["order"],
  "TTC_i":2793,
  "kit_sans_suite":["false"],
  "type_cde_s":"CDE",
  "statut_s":"V",
  "campagne_s":"G",
  "date_dt":"2017-11-21T00:00:00Z",
  "id":"G43625",
  "enseigne_s":"SAV",
 }
}

And i would like to get a sum of TTC_i field, by parent document... I tried
with stats, pivot, without any results...

Thanks for your help.

Best
JB



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

RE: cmd to enable debug logs

[Prateek Jain J]

Thanks Rahul.


Regards,
Prateek Jain
Team: Totoro
Mobile: +353 894 391716

-Original Message-
From: Rahul Chhiber  
Sent: 09 July 2018 12:10 PM
To: solr-user@lucene.apache.org
Subject: RE: cmd to enable debug logs

Use -v option in the bin/solr start command.

Regards,
Rahul Chhiber


-Original Message-
From: Prateek Jain J [mailto:prateek.j.j...@ericsson.com] 
Sent: Monday, July 09, 2018 4:26 PM
To: solr-user@lucene.apache.org
Subject: cmd to enable debug logs


Hi All,

What's the command (from CLI) to enable debug logs for a core in solr? To be 
precise, I am using solr 4.8.1. I looked into admin guide and it talks about 
how to do it from UI but nothing from CLI perspective.  Any help pointers will 
be of help.

Note: I can't update solrconfig.xml.


Regards,
Prateek Jain

Re: Delta import not working with Oracle in Solr

[Alexandre Rafalovitch]

I think you are moving so fast it is hard to understand where you need help.

Can you setup one clean smallest issue (maybe as test) and try our original
suggestions.

Otherwise, nobody has enough attention energy to figure out what is
happening.

And even then, this list is voluntary help, we are just trying to give you
pointers the best we can. It is quite possible you have outgrown DIH and
need to move up to a propper stand alone ETL tool.

Regards,
Alex

On Sun, Jul 8, 2018, 11:49 PM shruti suri,  wrote:

> Still not working, same issue documents are not getting pushed to index.
>
>
>
> -
> Regards
> Shruti
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>

RE: cmd to enable debug logs

[Rahul Chhiber]

Use -v option in the bin/solr start command.

Regards,
Rahul Chhiber


-Original Message-
From: Prateek Jain J [mailto:prateek.j.j...@ericsson.com] 
Sent: Monday, July 09, 2018 4:26 PM
To: solr-user@lucene.apache.org
Subject: cmd to enable debug logs


Hi All,

What's the command (from CLI) to enable debug logs for a core in solr? To be 
precise, I am using solr 4.8.1. I looked into admin guide and it talks about 
how to do it from UI but nothing from CLI perspective.  Any help pointers will 
be of help.

Note: I can't update solrconfig.xml.


Regards,
Prateek Jain

cmd to enable debug logs

[Prateek Jain J]

Hi All,

What's the command (from CLI) to enable debug logs for a core in solr? To be 
precise, I am using solr 4.8.1. I looked into admin guide and it talks about 
how to do it from UI but nothing from CLI perspective.  Any help pointers will 
be of help.

Note: I can't update solrconfig.xml.


Regards,
Prateek Jain

Re: CDCR traffic

[Greenhorn Techie]

Amrit,

Further to the below conversation:

As I understand, Solr supports SSL encryption between nodes within a Solr
cluster and as well communications to and from clients. In the case of
CDCR, assuming both the source and target clusters are SSL enabled, can we
say that the source clusters’ shard leaders act as clients to the target
cluster and hence the data is encrypted while its transmitted between the
clusters?

Thanks


On 25 June 2018 at 15:56:07, Amrit Sarkar (sarkaramr...@gmail.com) wrote:

Hi Rajeswari,

No it is not. Source forwards the update to the Target in classic manner.

Amrit Sarkar
Search Engineer
Lucidworks, Inc.
415-589-9269
www.lucidworks.com
Twitter http://twitter.com/lucidworks
LinkedIn: https://www.linkedin.com/in/sarkaramrit2
Medium: https://medium.com/@sarkaramrit2

On Fri, Jun 22, 2018 at 11:38 PM, Natarajan, Rajeswari <
rajeswari.natara...@sap.com> wrote:

> Hi,
>
> Would like to know , if the CDCR traffic is encrypted.
>
> Thanks
> Ra
>

Re: Solr Kerberos Authentication

[Greenhorn Techie]

Hi,

Any thoughts on this please?

Thanks


On 5 July 2018 at 15:06:26, Greenhorn Techie (greenhorntec...@gmail.com)
wrote:

Hi,

In the solr documentation, it is mentioned that blockUnknown property for
Authentication plugin has the default value of false, which means any
authenticated users will be allowed to use Solr. However, wondering whether
this parameter only makes sense for Basic Authentication only or does it
impact Kerberos authentication as well?

I couldn’t find any Kerberos plugin example in the documentation where the
blockUnknown parameter has been set or defined. Hence my question.

Thanks