Ivan Daschinskiy created IGNITE-8429: ----------------------------------------
Summary: Unexpected error during incorrect WAL segment decompression, causes node termination. Key: IGNITE-8429 URL: https://issues.apache.org/jira/browse/IGNITE-8429 Project: Ignite Issue Type: Bug Components: persistence Affects Versions: 2.5 Reporter: Ivan Daschinskiy Fix For: 2.5 File decompressor failure due to incorrect (zero-length) archived segment. 2018-04-30 00:00:02.811 [ERROR][wal-file-decompressor%DPL_GRID%DplGridNodeName][org.apache.ignite.Ignite] Critical system error detected. Will be handled accordingly to configured handler [hnd=class o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: Thread wal-file-decompressor%DPL_GRID%DplGridNodeName is terminated unexpectedly]] java.lang.IllegalStateException: Thread wal-file-decompressor%DPL_GRID%DplGridNodeName is terminated unexpectedly at org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileDecompressor.run(FileWriteAheadLogManager.java:2104) 2018-04-30 00:00:02.812 [ERROR][wal-file-decompressor%DPL_GRID%DplGridNodeName][org.apache.ignite.Ignite] JVM will be halted immediately due to the failure: [failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=java.lang.IllegalStateException: Thread wal-file-decompressor%DPL_GRID%DplGridNodeName is terminated unexpectedly]] touch 0000000000000754.wal zip 0000000000000754.wal.zip 0000000000000754.wal ls -l -rw-rw-r-- 1 dmitriy dmitriy 0 май 1 16:40 0000000000000754.wal -rw-rw-r-- 1 dmitriy dmitriy 190 май 1 16:46 0000000000000754.wal.zip Archive: /tmp/temp/0000000000000754.wal.zip Length Method Size Cmpr Date Time CRC-32 Name -------- ------ ------- ---- ---------- ----- -------- ---- 0 Stored 0 0% 2018-05-01 16:40 00000000 0000000000000754.wal -------- ------- --- ------- 0 0 0% 1 file We should softly handle this situation: print message in log and continue the compression with next segment. We also should handle "skipped" segments and don't delete them in deleteObsoleteRawSegments(). -- This message was sent by Atlassian JIRA (v7.6.3#76005)