Re: Solr fails to start with G1 GC

[Ishan Chattopadhyaya]

Try using haveged for your entropy problem.

On Thu, 16 Jul, 2020, 10:20 pm Walter Underwood, 
wrote:

> Instead of editing bin/solr, you should be able to set GC_TUNE in
> solr.in.sh, as I showed in my post below.
>
> wunder
> Walter Underwood
> wun...@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
>
> > On Jul 16, 2020, at 7:52 AM, krishan goyal 
> wrote:
> >
> > The issue was figured out by starting solr with the -f parameter which
> > starts solr in foreground and provides the errors if any
> >
> > Got an error - "Conflicting collector combinations in option list; please
> > refer to the release notes for the combinations allowed"
> >
> > Turns out bin/solr file starts with CMS by default and had to disable
> that
> > to resolve the conflict.
> >
> >
> > On Wed, Jul 15, 2020 at 10:20 PM Walter Underwood  >
> > wrote:
> >
> >> I don’t see a heap size specified, so it is probably trying to run with
> >> a 512 Megabyte heap. That might just not work with the 32M region
> >> size.
> >>
> >> Here are the options we have been using for 3+ years on about 150 hosts.
> >>
> >> SOLR_HEAP=8g
> >> # Use G1 GC  -- wunder 2017-01-23
> >> # Settings from https://wiki.apache.org/solr/ShawnHeisey
> >> GC_TUNE=" \
> >> -XX:+UseG1GC \
> >> -XX:+ParallelRefProcEnabled \
> >> -XX:G1HeapRegionSize=8m \
> >> -XX:MaxGCPauseMillis=200 \
> >> -XX:+UseLargePages \
> >> -XX:+AggressiveOpts \
> >> "
> >>
> >> wunder
> >> Walter Underwood
> >> wun...@wunderwood.org
> >> http://observer.wunderwood.org/  (my blog)
> >>
> >>> On Jul 15, 2020, at 4:24 AM, krishan goyal 
> >> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I am using Solr 7.7
> >>>
> >>> I am trying to start my solr server with G1 GC instead of the default
> CMS
> >>> but the solr service doesn't get up.
> >>>
> >>> The command I use to start solr is
> >>>
> >>> bin/solr start -p 25280 -a "-Dsolr.solr.home=
> >>> -Denable.slave=true -Denable.master=false -XX:+UseG1GC
> >>> -XX:MaxGCPauseMillis=500 -XX:+UnlockExperimentalVMOptions
> >>> -XX:G1MaxNewSizePercent=30 -XX:G1NewSizePercent=5
> >> -XX:G1HeapRegionSize=32M
> >>> -XX:InitiatingHeapOccupancyPercent=70"
> >>>
> >>> I have tried various permutations of the start command by dropping /
> >> adding
> >>> other parameters but it doesn't work. However starts up just fine with
> >>> just "-Dsolr.solr.home= -Denable.slave=true
> >>> -Denable.master=false" and starts up with the default CMS collector
> >>>
> >>> I don't get any useful error logs too. It waits for default 180 secs
> and
> >>> then prints
> >>>
> >>> Warning: Available entropy is low. As a result, use of the UUIDField,
> >> SSL,
> >>> or any other features that require
> >>> RNG might not work properly. To check for the amount of available
> >> entropy,
> >>> use 'cat /proc/sys/kernel/random/entropy_avail'.
> >>>
> >>> Waiting up to 180 seconds to see Solr running on port 25280 [|]  Still
> >> not
> >>> seeing Solr listening on 25280 after 180 seconds!
> >>> 2020-07-15 07:07:52.042 INFO  (coreCloseExecutor-60-thread-6) [
> >>> x:coreName] o.a.s.c.SolrCore [coreName]  CLOSING SolrCore
> >>> org.apache.solr.core.SolrCore@7cc638d8
> >>> 2020-07-15 07:07:52.099 INFO  (coreCloseExecutor-60-thread-6) [
> >>> x:coreName] o.a.s.m.SolrMetricManager Closing metric reporters for
> >>> registry=solr.core.coreName, tag=7cc638d8
> >>> 2020-07-15 07:07:52.100 INFO  (coreCloseExecutor-60-thread-6) [
> >>> x:coreName] o.a.s.m.r.SolrJmxReporter Closing reporter
> >>> [org.apache.solr.metrics.reporters.SolrJmxReporter@5216981f: rootName
> =
> >>> null, domain = solr.core.coreName, service url = null, agent id = null]
> >> for
> >>> registry solr.core.coreName /
> >> com.codahale.metrics.MetricRegistry@32988ddf
> >>> 2020-07-15 07:07:52.173 INFO  (ShutdownMonitor) [   ]
> >>> o.a.s.m.SolrMetricManager Closing metric reporters for
> >> registry=solr.node,
> >>> tag=null
> >>> 2020-07-15 07:07:52.173 INFO  (ShutdownMonitor) [   ]
> >>> o.a.s.m.r.SolrJmxReporter Closing reporter
> >>> [org.apache.solr.metrics.reporters.SolrJmxReporter@28952dea: rootName
> =
> >>> null, domain = solr.node, service url = null, agent id = null] for
> >> registry
> >>> solr.node / com.codahale.metrics.MetricRegistry@655f4a3f
> >>> 2020-07-15 07:07:52.175 INFO  (ShutdownMonitor) [   ]
> >>> o.a.s.m.SolrMetricManager Closing metric reporters for
> registry=solr.jvm,
> >>> tag=null
> >>> 2020-07-15 07:07:52.175 INFO  (ShutdownMonitor) [   ]
> >>> o.a.s.m.r.SolrJmxReporter Closing reporter
> >>> [org.apache.solr.metrics.reporters.SolrJmxReporter@69c6161d: rootName
> =
> >>> null, domain = solr.jvm, service url = null, agent id = null] for
> >> registry
> >>> solr.jvm / com.codahale.metrics.MetricRegistry@1252ce77
> >>> 2020-07-15 07:07:52.176 INFO  (ShutdownMonitor) [   ]
> >>> o.a.s.m.SolrMetricManager Closing metric reporters for
> >> registry=solr.jetty,
> >>> tag=null
> >>> 2020-07-15 07:07:52.176 INFO  (ShutdownMonitor) [   ]
> >>> o.a.s.m.r.SolrJmxReporter Closing reporter
> >>> 

Re: UpdateProcessorChains -cdcr processor along with ignore commit processor

[Shawn Heisey]

On 7/15/2020 11:39 PM, Natarajan, Rajeswari wrote:

Resending this again as I still could not make this work. So would like to know 
if this is even possible to have
both solr.CdcrUpdateProcessorFactory and 
solr.IgnoreCommitOptimizeUpdateProcessorFactory  in solrconfig.xml and get both 
functionalities work.


You need to create one update chain that uses both processors.  Only one 
update chain can be applied at a time.  So create one chain with all the 
processors you meed and use that.


Your config has two chains.  Only one of them can be active on each update.

Thanks,
Shawn

Re: AtomicUpdate on SolrCloud is not working

[Shawn Heisey]

On 7/17/2020 1:32 AM, yo tomi wrote:

When I did AtomicUpdate on SolrCloud by the following setting, it does
not work properly.


As Jörn Franke already mentioned, you haven't said exactly what "does 
not work properly" actually means in your situation.  Without that 
information, it will be very difficult to provide any real help.


Atomic update functionality is currently implemented in 
DistributedUpdateProcessorFactory.



---

  
  
  
  
  

---
When changed as follows and made it work, it became as expected.
---

  
  
  
  

---


The effective result difference between these configurations is that 
atomic updates will happen first with the first config, and in the 
second, atomic updates will happen second to last -- just before 
RunUpdateProcessorFactory.


Also, with the first config, most of the update processors are going to 
be executed on the machine with the shard leader (after the update is 
distributed) and if there is more than one NRT replica, they will be 
executed multiple times.  With the second config, most of the processors 
will be executed on the machine that actually receives the update 
request.  For the purposes of that discussion, remember that when a PULL 
replica is elected leader, it is effectively an NRT replica.


Does that information help you determine why it doesn't do what you expect?


The later setting and the way of using post-processor could make the
same result, I though,
but using post-processor, bug of SOLR-8030 makes me not feel like using it.
By the latter setting even, is there any possibility of SOLR-8030 to
become?


See this part of the reference guide for a bunch of gory details about 
DistributedUpdateProcessorFactory:


https://cwiki.apache.org/confluence/display/SOLR/UpdateRequestProcessor#UpdateRequestProcessor-DistributedUpdates

In SOLR-8030, the general consensus among committers is that you should 
configure almost all update processors as "pre" processors -- placed 
before DistributedUpdatePorcessorFactory in the config.  When done this 
way, updates are usually faster and less likely to yield inconsistent 
results.


There may be situations where having them as "post" processors is 
correct, but that won't happen very often.  The second config above does 
implicitly use "pre" for most of the processors.


Thanks,
Shawn