Hi experts: Need some help and suggestion about an issue I am facing
Solr info: - Solr 8.7 - Solr cloud with tlog replica; replica size is 3 for my Solr collection Issue: - before issuing collection reload; I observed a new tlog file are created after every commit; and those tlog files are deleted after a while (may be after index are merged?) - then I issued a collection reload using collection API on my collection at 20:15 - after leader replica is reloaded; no new tlog file are created; instead latest tlog file is growing, and no tlog file is deleted after reload. Below under "files under tlog directory" section is a snapshot of the tlog files under tlog directory of the leader replica. Again, I issued collection reload at 20:15, and after that tlog.0000000000000000883 is growing - I looked into log file and find error log entries below under "log entries" section, and the log entry repeats continuously for every auto commit after reload. I hope this log entry can provide some information for the issue. Please help and suggestion what I may do incorrectly. Or this is a known issue, is there a way I can fix or work-around it? Thank you so much! --Michael Hu --- beginning for files under tlog directory --- -rw-r--r-- 1 solr solr 47527321 Mar 4 20:14 tlog.0000000000000000877 -rw-r--r-- 1 solr solr 42614907 Mar 4 20:14 tlog.0000000000000000878 -rw-r--r-- 1 solr solr 37524663 Mar 4 20:14 tlog.0000000000000000879 -rw-r--r-- 1 solr solr 44067997 Mar 4 20:14 tlog.0000000000000000880 -rw-r--r-- 1 solr solr 33209784 Mar 4 20:15 tlog.0000000000000000881 -rw-r--r-- 1 solr solr 55435186 Mar 4 20:15 tlog.0000000000000000882 -rw-r--r-- 1 solr solr 2179991713 Mar 4 20:29 tlog.0000000000000000883 --- end for files under tlog directory --- --- beginning for log entries --- 2021-03-04 20:15:38.251 ERROR (commitScheduler-4327-thread-1) [c:mycollection s:myshard r:core_node10 x:mycolletion_myshard_replica_t7] o.a.s.u.CommitTracker auto commit error...: org.apache.solr.common.SolrException: java.nio.channels.ClosedChannelException at org.apache.solr.update.TransactionLog.writeCommit(TransactionLog.java:503) at org.apache.solr.update.UpdateLog.postCommit(UpdateLog.java:835) at org.apache.solr.update.UpdateLog.preCommit(UpdateLog.java:819) at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:673) at org.apache.solr.update.CommitTracker.run(CommitTracker.java:273) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: java.nio.channels.ClosedChannelException at java.base/sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:150) at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:266) at java.base/java.nio.channels.Channels.writeFullyImpl(Channels.java:74) at java.base/java.nio.channels.Channels.writeFully(Channels.java:97) at java.base/java.nio.channels.Channels$1.write(Channels.java:172) at org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:216) at org.apache.solr.common.util.FastOutputStream.flushBuffer(FastOutputStream.java:209) at org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:193) at org.apache.solr.update.TransactionLog.writeCommit(TransactionLog.java:498) ... 10 more --- end for log entries ---