The Second exception states there is a File of an sstable Missing. Is it possible you didnt only delete commit logs or nfs Mount is stale or Not mounted?
Am 01.11.2016 19:52 schrieb "John Sanda" <john.sa...@gmail.com>: > Using nfs for a distribited System like Cassandra is like putting a >> Ferrari on a Truck and going for a Race with the Truck. It is simply >> nonsense. > > > As I mentioned in my original post, I am aware that using NFS is > considered bad and even documented as an anti-pattern. Your analogy, > interesting as it may be, is not helpful. It simply restating what has > already been said. I don't even know that NFS is to blame for the > CommitLogReplayException that I cited. > > On Tue, Nov 1, 2016 at 2:43 PM, Benjamin Roth <benjamin.r...@jaumo.com> > wrote: > >> Using nfs for a distribited System like Cassandra is like putting a >> Ferrari on a Truck and going for a Race with the Truck. It is simply >> nonsense. >> >> Am 01.11.2016 19:39 schrieb "Vladimir Yudovin" <vla...@winguzone.com>: >> >>> Hi, >>> >>> it's not only performance issue. In case of network problem writer tread >>> can be blocked, also in case of failure loss of data can occur. >>> >>> Best regards, Vladimir Yudovin, >>> >>> *Winguzone <https://winguzone.com?from=list> - Hosted Cloud >>> CassandraLaunch your cluster in minutes.* >>> >>> >>> ---- On Tue, 01 Nov 2016 14:10:10 -0400*John Sanda >>> <john.sa...@gmail.com <john.sa...@gmail.com>>* wrote ---- >>> >>> I know that using NFS is discouraged, particularly for the commit log. >>> Can anyone shed some light into what kinds of problems I might encounter >>> aside from performance? The reason for my inquiry is because I have some >>> deployments with Cassandra 2.2.1 that use NFS and are experiencing some >>> problems like reoccurring corrupted commit log segments on start up: >>> >>> ERROR 19:38:42 Exiting due to error while processing commit log during >>> initialization. >>> >>> org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException: >>> Mutation checksum failure at 33296351 in CommitLog-5-1474325237114.log >>> at >>> org.apache.cassandra.db.commitlog.CommitLogReplayer.handleReplayError(CommitLogReplayer.java:622) >>> [apache-cassandra-2.2.1.jar] >>> at >>> org.apache.cassandra.db.commitlog.CommitLogReplayer.replaySyncSection(CommitLogReplayer.java:492) >>> [apache-cassandra-2.2.1.jar] >>> at >>> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:388) >>> [apache-cassandra-2.2.1.jar] >>> at >>> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:147) >>> [apache-cassandra-2.2.1.jar] >>> at >>> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:189) >>> [apache-cassandra-2.2.1.jar] >>> at >>> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:169) >>> [apache-cassandra-2.2.1.jar] >>> at >>> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:266) >>> [apache-cassandra-2.2.1.jar] >>> at >>> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:488) >>> [apache-cassandra-2.2.1.jar] >>> at >>> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:595) >>> [apache-cassandra-2.2.1.jar] >>> >>> >>> In one deployment after removing all of corrupted commit log segments I >>> got a different error: >>> >>> Exception (java.lang.RuntimeException) encountered during startup: >>> java.nio.file.NoSuchFileException: >>> /cassandra_data/data/hawkular_metrics/metrics_idx-1ea881506ee311e6890b131c1bc89929/la-86-big-Statistics.db >>> java.lang.RuntimeException: java.nio.file.NoSuchFileException: >>> /cassandra_data/data/hawkular_metrics/metrics_idx-1ea881506ee311e6890b131c1bc89929/la-86-big-Statistics.db >>> at >>> org.apache.cassandra.io.util.ChannelProxy.openChannel(ChannelProxy.java:55) >>> at >>> org.apache.cassandra.io.util.ChannelProxy.<init>(ChannelProxy.java:66) >>> at >>> org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:78) >>> at >>> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:91) >>> at >>> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:101) >>> at >>> org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:672) >>> at >>> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:216) >>> at >>> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:488) >>> at >>> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:595) >>> Caused by: java.nio.file.NoSuchFileException: >>> /cassandra_data/data/hawkular_metrics/metrics_idx-1ea881506ee311e6890b131c1bc89929/la-86-big-Statistics.db >>> at >>> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) >>> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) >>> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) >>> at >>> sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177) >>> at java.nio.channels.FileChannel.open(FileChannel.java:287) >>> at java.nio.channels.FileChannel.open(FileChannel.java:335) >>> at >>> org.apache.cassandra.io.util.ChannelProxy.openChannel(ChannelProxy.java:51) >>> ... 8 more >>> >>> >>> The latter error looks like it involves compaction and might be >>> unrelated. I don't know if it matters, but I have commit log compression >>> enabled in these environments. >>> >>> -- >>> >>> - John >>> >>> >>> > > > -- > > - John >
