looks like youre repeater is grabbing a file that the master merged into a different file, why not lower how often you go from master->repeater, and/or dont commit so often so you can make the index faster
On Fri, Nov 13, 2020 at 12:13 PM Parshant Kumar <parshant.ku...@indiamart.com.invalid> wrote: > All,please help on this > > On Tue, Nov 3, 2020, 6:01 PM Parshant Kumar <parshant.ku...@indiamart.com> > wrote: > > > Hi team, > > > > We are having solr architecture as *master->repeater-> 3 slave servers.* > > > > We are doing incremental indexing on the master server(every 20 min) . > > Replication of index is done from master to repeater server(every 10 > mins) > > and from repeater to 3 slave servers (every 3 hours). > > *We are facing the frequent replication failure between master to > repeater > > server as well as between repeater to slave servers.* > > On checking logs found that every time one of the below exceptions > > occurred whenever the replication has failed . > > > > 1)WARN : Error in fetching file: _4rnu_t.liv (downloaded 0 of 11505507 > > bytes) > > java.io.EOFException: Unexpected end of ZLIB input stream > > at > > java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:240) > > at > > java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158) > > at > > > org.apache.solr.common.util.FastInputStream.readWrappedStream(FastInputStream.java:79) > > at > > > org.apache.solr.common.util.FastInputStream.refill(FastInputStream.java:88) > > at > > > org.apache.solr.common.util.FastInputStream.read(FastInputStream.java:139) > > at > > > org.apache.solr.common.util.FastInputStream.readFully(FastInputStream.java:166) > > at > > > org.apache.solr.common.util.FastInputStream.readFully(FastInputStream.java:160) > > at > > > org.apache.solr.handler.IndexFetcher$FileFetcher.fetchPackets(IndexFetcher.java:1443) > > at > > > org.apache.solr.handler.IndexFetcher$FileFetcher.fetch(IndexFetcher.java:1409) > > > > > > 2) > > WARN : Error getting file length for [segments_568] > > java.nio.file.NoSuchFileException: > > > /data/solr/search/application/core-conf/im-search/data/index.20200711012319226/segments_568 > > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) > > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) > > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) > > at > > > sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) > > at > > > sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) > > at > > > sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) > > at java.nio.file.Files.readAttributes(Files.java:1737) > > at java.nio.file.Files.size(Files.java:2332) > > at org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:243) > > at > > > org.apache.solr.handler.admin.LukeRequestHandler.getFileLength(LukeRequestHandler.java:615) > > at > > > org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(LukeRequestHandler.java:588) > > at > > > org.apache.solr.handler.admin.CoreAdminOperation.getCoreStatus(CoreAdminOperation.java:335) > > > > 3) > > WARN : Error in fetching file: _4nji.nvd (downloaded 507510784 of > > 555377795 bytes) > > org.apache.http.MalformedChunkCodingException: CRLF expected at end of > > chunk > > at > > org.apache.http.impl.io > .ChunkedInputStream.getChunkSize(ChunkedInputStream.java:255) > > at > > org.apache.http.impl.io > .ChunkedInputStream.nextChunk(ChunkedInputStream.java:227) > > at > > org.apache.http.impl.io > .ChunkedInputStream.read(ChunkedInputStream.java:186) > > at > > > org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:137) > > at > > java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:238) > > at > > java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158) > > at > > > org.apache.solr.common.util.FastInputStream.readWrappedStream(FastInputStream.java:79) > > at > > > org.apache.solr.common.util.FastInputStream.read(FastInputStream.java:128) > > at > > > org.apache.solr.common.util.FastInputStream.readFully(FastInputStream.java:166) > > at > > > org.apache.solr.handler.IndexFetcher$FileFetcher.fetchPackets(IndexFetcher.java:1458) > > at > > > org.apache.solr.handler.IndexFetcher$FileFetcher.fetch(IndexFetcher.java:1409) > > at > > > org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetcher.java:1390) > > at > > > org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.java:872) > > at > > > org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:438) > > at > > > org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:254) > > > > *Replication configuration of master,repeater,slave's is given below:* > > > > <requestHandler name="/replication" class="solr.ReplicationHandler"> > > <lst name="master"> > > <str name="enable">${enable.master:false}</str> > > <str name="replicateAfter">commit</str> > > <str name="replicateAfter">startup</str> > > <str name="commitReserveDuration">00:00:10</str> > > </lst> > > > > > > *Commit Configuration master,repeater,slave's is given below :* > > > > > <autoCommit><maxDocs>100000</maxDocs><openSearcher>false</openSearcher></autoCommit> > > > > > > Please help in finding the root cause of replication failure.Let me know > for any queries. > > > > Thanks > > > > Parshant kumar > > > > > > > > > > > > > > > > > > > > > > < > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> > Virus-free. > > www.avast.com > > < > https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail > > > > <#m_5505033333099918512_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> > > > > -- > >