Shawn,
What version of Solr? 8.7.0
How is it installed and started? I download binary file and change
configuration in solr.in.sh, then start it by ./bin/solr start &
What OS? centos 7
Java version? openJDK 8
I don't see any log in solr.log, but there is OutOfMemory error in
solr-8983-console.log file.
I changed the heap to 10G now, it's ok now, but I am wondering why solr
uses 6.4G memory when there is no data at all. there are just 10 empty
collections.
also the way to start solr is matter?
>>>> the below is from my gc log, can anything be optimized?
{Heap before GC invocations=55 (full 0):
garbage-first heap total 10485760K, used 9360694K [0x0000000540000000,
0x0000000540405000, 0x00000007c0000000)
region size 4096K, 704 young (2883584K), 2 survivors (8192K)
Metaspace used 62884K, capacity 66378K, committed 66844K, reserved
1107968K
class space used 7238K, capacity 8128K, committed 8220K, reserved
1048576K
2021-01-24T09:17:17.467+0800: 39708.837: [GC pause (G1 Evacuation Pause)
(young) (initial-mark)
Desired survivor size 184549376 bytes, new threshold 15 (max 15)
- age 1: 860160 bytes, 860160 total
- age 2: 25376 bytes, 885536 total
- age 3: 293920 bytes, 1179456 total
- age 4: 1498520 bytes, 2677976 total
, 0.0078650 secs]
[Parallel Time: 5.6 ms, GC Workers: 4]
[GC Worker Start (ms): Min: 39708837.9, Avg: 39708837.9, Max:
39708838.0, Diff: 0.0]
[Ext Root Scanning (ms): Min: 1.5, Avg: 1.6, Max: 1.9, Diff: 0.4,
Sum: 6.6]
[Update RS (ms): Min: 0.7, Avg: 0.8, Max: 0.9, Diff: 0.2, Sum: 3.3]
[Processed Buffers: Min: 5, Avg: 10.8, Max: 25, Diff: 20, Sum: 43]
[Scan RS (ms): Min: 0.1, Avg: 0.2, Max: 0.2, Diff: 0.1, Sum: 0.8]
[Code Root Scanning (ms): Min: 0.0, Avg: 0.5, Max: 1.9, Diff: 1.9,
Sum: 2.0]
[Object Copy (ms): Min: 0.7, Avg: 2.2, Max: 2.7, Diff: 2.0, Sum: 8.6]
[Termination (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
[Termination Attempts: Min: 2, Avg: 3.2, Max: 5, Diff: 3, Sum: 13]
[GC Worker Other (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum:
0.2]
[GC Worker Total (ms): Min: 5.3, Avg: 5.4, Max: 5.4, Diff: 0.0, Sum:
21.4]
[GC Worker End (ms): Min: 39708843.3, Avg: 39708843.3, Max:
39708843.3, Diff: 0.0]
[Code Root Fixup: 0.0 ms]
[Code Root Purge: 0.0 ms]
[Clear CT: 0.4 ms]
[Other: 1.9 ms]
[Choose CSet: 0.0 ms]
[Ref Proc: 0.8 ms]
[Ref Enq: 0.0 ms]
[Redirty Cards: 0.0 ms]
[Humongous Register: 0.0 ms]
[Humongous Reclaim: 0.0 ms]
[Free CSet: 0.6 ms]
[Eden: 2808.0M(2808.0M)->0.0B(2824.0M) Survivors: 8192.0K->8192.0K Heap:
9141.3M(10240.0M)->6333.1M(10240.0M)]
Heap after GC invocations=56 (full 0):
garbage-first heap total 10485760K, used 6485044K [0x0000000540000000,
0x0000000540405000, 0x00000007c0000000)
region size 4096K, 2 young (8192K), 2 survivors (8192K)
Metaspace used 62884K, capacity 66378K, committed 66844K, reserved
1107968K
class space used 7238K, capacity 8128K, committed 8220K, reserved
1048576K
}
[Times: user=0.03 sys=0.00, real=0.01 secs]
2021-01-24T09:17:17.475+0800: 39708.845: [GC
concurrent-root-region-scan-start]
2021-01-24T09:17:17.475+0800: 39708.846: Total time for which application
threads were stopped: 0.0088115 seconds, Stopping threads took: 0.0000670
seconds
2021-01-24T09:17:17.481+0800: 39708.851: [GC
concurrent-root-region-scan-end, 0.0057470 secs]
2021-01-24T09:17:17.481+0800: 39708.851: [GC concurrent-mark-start]
2021-01-24T09:17:32.665+0800: 39724.035: [GC concurrent-mark-end,
15.1839300 secs]
2021-01-24T09:17:32.665+0800: 39724.036: [GC remark
2021-01-24T09:17:32.665+0800: 39724.036: [Finalize Marking, 0.0002351 secs]
2021-01-24T09:17:32.666+0800: 39724.036: [GC ref-proc, 0.0018365 secs]
2021-01-24T09:17:32.668+0800: 39724.038: [Unloading, 0.0125520 secs],
0.0339855 secs]
[Times: user=0.12 sys=0.00, real=0.04 secs]
2021-01-24T09:17:32.700+0800: 39724.070: Total time for which application
threads were stopped: 0.0347120 seconds, Stopping threads took: 0.0000616
seconds
2021-01-24T09:17:32.700+0800: 39724.071: [GC cleanup 6355M->6355M(10240M),
0.0164190 secs]
[Times: user=0.06 sys=0.00, real=0.01 secs]
Thanks
Derrick
On Sat, Jan 23, 2021 at 7:54 PM Shawn Heisey <apa...@elyograg.org> wrote:
> On 1/23/2021 6:29 AM, Luke Oak wrote:
> > I use default settings to start solr , I set heap to 6G, I created 10
> collections with 1node and 1 replica, however, there is not much data at
> all, just 100 documents.
> >
> > My server is 32 G memory and 4 core cpu, ssd drive 300g
> >
> > It was ok when i created 5 collections. It got oom killed when 10
> collections are created. Please, no data in new collections.
>
> What version of Solr? How is it installed and started? What OS? What
> Java version?
>
> Do you have the actual OutOfMemoryError text? If I remember correctly
> from my own reading, there are eight possible causes for OOME, and not
> all of them are related to memory. The actual exception, which will be
> recorded in the main Solr logfile if it is even recorded (sometimes it's
> not), will contain the reason for the error.
>
> A 6GB heap is definitely enough for a handful of empty cores. So my
> best guess is that another resource, possibly thread count or open
> files, is running out.
>
> > Also I found that solr doesn’t do garbage collection when the 6G is used
> ( from dashboard, jvm usage is reached 6 g)
>
> Sorry to be pedantic, but Solr doesn't EVER do Garbage Collection. Java
> does. And it is completely normal for the entire Java heap to be
> consumed on occasion, no matter what's happening. Solr does not expose
> any way to force a GC.
>
> Thanks,
> Shawn
>
