[ https://issues.apache.org/jira/browse/CASSANDRA-15724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083927#comment-17083927 ]
Boopalan commented on CASSANDRA-15724: --------------------------------------- Thanks [~jmeredithco]. I have prepare a new ubuntu setup. # Install Java 1.8.0_242 # Install Cassandra 3.11.6 # Create ext4 filesystem on our SSD disk and mount in /mnt/ directory # Edit the cassandra.yaml file and replace all data, hints, saved_caches and commit to /mnt/ directory # Edit the cassandra-en.sh file and replace lineVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=*127*.0.0.1" # Edit jvm.options file -Dcassandra.config=/etc/cassandra/cassandra.yaml # Restart Cassandra # nodetool status Now am running into different issue. Please find the below snippet. {noformat} root@cassandra-master:/mnt/data/system_auth# systemctl status cassandra ● cassandra.service - LSB: distributed storage system for structured data Loaded: loaded (/etc/init.d/cassandra; generated) Active: active (exited) since Wed 2020-04-15 14:07:06 IST; 7min ago Docs: man:systemd-sysv-generator(8) Process: 29015 ExecStop=/etc/init.d/cassandra stop (code=exited, status=0/SUCCESS) Process: 29033 ExecStart=/etc/init.d/cassandra start (code=exited, status=0/SUCCESS) Apr 15 14:07:06 cassandra-master systemd[1]: Starting LSB: distributed storage system for structured data... Apr 15 14:07:06 cassandra-master systemd[1]: Started LSB: distributed storage system for structured data. {noformat} {noformat} root@cassandra-master:/etc/cassandra# nodetool status nodetool: Failed to connect to '127.0.0.1:7199' - ConnectException: 'Connection refused (Connection refused)'. {noformat} *cat /var/log/cassandra/system.log* {noformat} ERROR [MemtableFlushWriter:1] 2020-04-15 14:07:14,791 LogTransaction.java:277 - Transaction log [md_txn_flush_4ed121f0-7ef4-11ea-8dc8-cf6c58093313.log in /mnt/data/system/local-7ad54392bcdd35a684174e047860b377] indicates txn was not completed, trying to abort it now ERROR [MemtableFlushWriter:1] 2020-04-15 14:07:14,795 LogTransaction.java:280 - Failed to abort transaction log [md_txn_flush_4ed121f0-7ef4-11ea-8dc8-cf6c58093313.log in /mnt/data/system/local-7ad54392bcdd35a684174e047860b377] java.lang.RuntimeException: java.nio.file.AccessDeniedException: /mnt/data/system/local-7ad54392bcdd35a684174e047860b377/md_txn_flush_4ed121f0-7ef4-11ea-8dc8-cf6c58093313.log at org.apache.cassandra.io.util.FileUtils.write(FileUtils.java:605) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.io.util.FileUtils.appendAndSync(FileUtils.java:586) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogReplica.append(LogReplica.java:104) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogReplicaSet.lambda$null$5(LogReplicaSet.java:224) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:113) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:103) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogReplicaSet.append(LogReplicaSet.java:224) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogFile.addRecord(LogFile.java:350) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogFile.abort(LogFile.java:269) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:113) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:103) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:98) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogTransaction$TransactionTidier.run(LogTransaction.java:278) [apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogTransaction$TransactionTidier.tidy(LogTransaction.java:260) [apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.concurrent.Ref$GlobalState.release(Ref.java:326) [apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.concurrent.Ref$State.ensureReleased(Ref.java:204) [apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.concurrent.Ref.ensureReleased(Ref.java:124) [apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogTransaction.complete(LogTransaction.java:397) [apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogTransaction.doAbort(LogTransaction.java:414) [apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:141) [apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LifecycleTransaction.doAbort(LifecycleTransaction.java:244) [apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:141) [apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1173) [apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1119) [apache-cassandra-3.11.6.jar:3.11.6] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_242] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_242] at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84) [apache-cassandra-3.11.6.jar:3.11.6] at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_242] Caused by: java.nio.file.AccessDeniedException: /mnt/data/system/local-7ad54392bcdd35a684174e047860b377/md_txn_flush_4ed121f0-7ef4-11ea-8dc8-cf6c58093313.log at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[na:1.8.0_242] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[na:1.8.0_242] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[na:1.8.0_242] at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) ~[na:1.8.0_242] at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434) ~[na:1.8.0_242] at java.nio.file.Files.newOutputStream(Files.java:216) ~[na:1.8.0_242] at java.nio.file.Files.write(Files.java:3351) ~[na:1.8.0_242] at org.apache.cassandra.io.util.FileUtils.write(FileUtils.java:598) ~[apache-cassandra-3.11.6.jar:3.11.6] ... 27 common frames omitted ERROR [MemtableFlushWriter:1] 2020-04-15 14:07:14,798 LogTransaction.java:225 - Unable to delete /mnt/data/system/local-7ad54392bcdd35a684174e047860b377/md_txn_flush_4ed121f0-7ef4-11ea-8dc8-cf6c58093313.log as it does not exist, see debug log file for stack trace ERROR [MemtablePostFlush:1] 2020-04-15 14:07:14,802 CassandraDaemon.java:235 - Exception in thread Thread[MemtablePostFlush:1,5,main] java.lang.RuntimeException: java.nio.file.AccessDeniedException: /mnt/data/system/local-7ad54392bcdd35a684174e047860b377/md_txn_flush_4ed121f0-7ef4-11ea-8dc8-cf6c58093313.log at org.apache.cassandra.io.util.FileUtils.write(FileUtils.java:605) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.io.util.FileUtils.appendAndSync(FileUtils.java:586) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogReplica.append(LogReplica.java:104) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogReplicaSet.lambda$null$5(LogReplicaSet.java:224) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:113) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:103) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogReplicaSet.append(LogReplicaSet.java:224) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogFile.addRecord(LogFile.java:350) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogFile.add(LogFile.java:297) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogTransaction.trackNew(LogTransaction.java:139) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LifecycleTransaction.trackNew(LifecycleTransaction.java:528) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.io.sstable.format.big.BigTableWriter.<init>(BigTableWriter.java:81) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:92) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:102) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:119) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:588) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:1027) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:533) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.Memtable$FlushRunnable.createFlushWriter(Memtable.java:504) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.Memtable$FlushRunnable.<init>(Memtable.java:443) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.Memtable$FlushRunnable.<init>(Memtable.java:420) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.Memtable.createFlushRunnables(Memtable.java:307) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.Memtable.flushRunnables(Memtable.java:298) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1154) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1119) ~[apache-cassandra-3.11.6.jar:3.11.6] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_242] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_242] at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84) [apache-cassandra-3.11.6.jar:3.11.6] at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_242] Suppressed: java.lang.RuntimeException: java.nio.file.AccessDeniedException: /mnt/data/system/local-7ad54392bcdd35a684174e047860b377/md_txn_flush_4ed121f0-7ef4-11ea-8dc8-cf6c58093313.log at org.apache.cassandra.io.util.FileUtils.write(FileUtils.java:605) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.io.util.FileUtils.appendAndSync(FileUtils.java:586) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogReplica.append(LogReplica.java:104) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogReplicaSet.lambda$null$5(LogReplicaSet.java:224) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:113) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:103) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogReplicaSet.append(LogReplicaSet.java:224) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogFile.addRecord(LogFile.java:350) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogFile.abort(LogFile.java:269) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:113) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:103) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:98) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogTransaction.doAbort(LogTransaction.java:414) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:141) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LifecycleTransaction.doAbort(LifecycleTransaction.java:244) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:141) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1173) ~[apache-cassandra-3.11.6.jar:3.11.6] ... 5 common frames omitted Caused by: java.nio.file.AccessDeniedException: /mnt/data/system/local-7ad54392bcdd35a684174e047860b377/md_txn_flush_4ed121f0-7ef4-11ea-8dc8-cf6c58093313.log at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434) at java.nio.file.Files.newOutputStream(Files.java:216) at java.nio.file.Files.write(Files.java:3351) at org.apache.cassandra.io.util.FileUtils.write(FileUtils.java:598) ... 21 common frames omitted Caused by: java.nio.file.AccessDeniedException: /mnt/data/system/local-7ad54392bcdd35a684174e047860b377/md_txn_flush_4ed121f0-7ef4-11ea-8dc8-cf6c58093313.log at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[na:1.8.0_242] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[na:1.8.0_242] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[na:1.8.0_242] at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) ~[na:1.8.0_242] at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434) ~[na:1.8.0_242] at java.nio.file.Files.newOutputStream(Files.java:216) ~[na:1.8.0_242] at java.nio.file.Files.write(Files.java:3351) ~[na:1.8.0_242] at org.apache.cassandra.io.util.FileUtils.write(FileUtils.java:598) ~[apache-cassandra-3.11.6.jar:3.11.6] ... 28 common frames omitted ERROR [main] 2020-04-15 14:07:14,803 CassandraDaemon.java:774 - Exception encountered during startup java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.nio.file.AccessDeniedException: /mnt/data/system/local-7ad54392bcdd35a684174e047860b377/md_txn_flush_4ed121f0-7ef4-11ea-8dc8-cf6c58093313.log at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:443) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.SystemKeyspace.forceBlockingFlush(SystemKeyspace.java:829) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.SystemKeyspace.removeTruncationRecord(SystemKeyspace.java:668) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:568) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.ColumnFamilyStore.invalidate(ColumnFamilyStore.java:544) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.schema.LegacySchemaMigrator.lambda$unloadLegacySchemaTables$1(LegacySchemaMigrator.java:137) ~[apache-cassandra-3.11.6.jar:3.11.6] at java.lang.Iterable.forEach(Iterable.java:75) ~[na:1.8.0_242] at org.apache.cassandra.schema.LegacySchemaMigrator.unloadLegacySchemaTables(LegacySchemaMigrator.java:137) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.schema.LegacySchemaMigrator.migrate(LegacySchemaMigrator.java:83) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:263) [apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:630) [apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:757) [apache-cassandra-3.11.6.jar:3.11.6] Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.nio.file.AccessDeniedException: /mnt/data/system/local-7ad54392bcdd35a684174e047860b377/md_txn_flush_4ed121f0-7ef4-11ea-8dc8-cf6c58093313.log at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[na:1.8.0_242] at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[na:1.8.0_242] at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:439) ~[apache-cassandra-3.11.6.jar:3.11.6] ... 11 common frames omitted Caused by: java.lang.RuntimeException: java.nio.file.AccessDeniedException: /mnt/data/system/local-7ad54392bcdd35a684174e047860b377/md_txn_flush_4ed121f0-7ef4-11ea-8dc8-cf6c58093313.log at org.apache.cassandra.io.util.FileUtils.write(FileUtils.java:605) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.io.util.FileUtils.appendAndSync(FileUtils.java:586) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogReplica.append(LogReplica.java:104) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogReplicaSet.lambda$null$5(LogReplicaSet.java:224) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:113) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:103) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogReplicaSet.append(LogReplicaSet.java:224) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogFile.addRecord(LogFile.java:350) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogFile.add(LogFile.java:297) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogTransaction.trackNew(LogTransaction.java:139) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LifecycleTransaction.trackNew(LifecycleTransaction.java:528) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.io.sstable.format.big.BigTableWriter.<init>(BigTableWriter.java:81) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.io.sstable.format.big.BigFormat$WriterFactory.open(BigFormat.java:92) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.io.sstable.format.SSTableWriter.create(SSTableWriter.java:102) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.create(SimpleSSTableMultiWriter.java:119) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.compaction.AbstractCompactionStrategy.createSSTableMultiWriter(AbstractCompactionStrategy.java:588) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.compaction.CompactionStrategyManager.createSSTableMultiWriter(CompactionStrategyManager.java:1027) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.ColumnFamilyStore.createSSTableMultiWriter(ColumnFamilyStore.java:533) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.Memtable$FlushRunnable.createFlushWriter(Memtable.java:504) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.Memtable$FlushRunnable.<init>(Memtable.java:443) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.Memtable$FlushRunnable.<init>(Memtable.java:420) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.Memtable.createFlushRunnables(Memtable.java:307) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.Memtable.flushRunnables(Memtable.java:298) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1154) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1119) ~[apache-cassandra-3.11.6.jar:3.11.6] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_242] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_242] at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84) ~[apache-cassandra-3.11.6.jar:3.11.6] at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_242] Suppressed: java.lang.RuntimeException: java.nio.file.AccessDeniedException: /mnt/data/system/local-7ad54392bcdd35a684174e047860b377/md_txn_flush_4ed121f0-7ef4-11ea-8dc8-cf6c58093313.log at org.apache.cassandra.io.util.FileUtils.write(FileUtils.java:605) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.io.util.FileUtils.appendAndSync(FileUtils.java:586) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogReplica.append(LogReplica.java:104) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogReplicaSet.lambda$null$5(LogReplicaSet.java:224) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:113) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:103) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogReplicaSet.append(LogReplicaSet.java:224) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogFile.addRecord(LogFile.java:350) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogFile.abort(LogFile.java:269) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:113) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:103) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.Throwables.perform(Throwables.java:98) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LogTransaction.doAbort(LogTransaction.java:414) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:141) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.lifecycle.LifecycleTransaction.doAbort(LifecycleTransaction.java:244) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.utils.concurrent.Transactional$AbstractTransactional.abort(Transactional.java:141) ~[apache-cassandra-3.11.6.jar:3.11.6] at org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1173) ~[apache-cassandra-3.11.6.jar:3.11.6] ... 5 common frames omitted Caused by: java.nio.file.AccessDeniedException: /mnt/data/system/local-7ad54392bcdd35a684174e047860b377/md_txn_flush_4ed121f0-7ef4-11ea-8dc8-cf6c58093313.log at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434) at java.nio.file.Files.newOutputStream(Files.java:216) at java.nio.file.Files.write(Files.java:3351) at org.apache.cassandra.io.util.FileUtils.write(FileUtils.java:598) ... 21 common frames omitted Caused by: java.nio.file.AccessDeniedException: /mnt/data/system/local-7ad54392bcdd35a684174e047860b377/md_txn_flush_4ed121f0-7ef4-11ea-8dc8-cf6c58093313.log at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[na:1.8.0_242] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[na:1.8.0_242] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[na:1.8.0_242] at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) ~[na:1.8.0_242] at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434) ~[na:1.8.0_242] at java.nio.file.Files.newOutputStream(Files.java:216) ~[na:1.8.0_242] at java.nio.file.Files.write(Files.java:3351) ~[na:1.8.0_242] at org.apache.cassandra.io.util.FileUtils.write(FileUtils.java:598) ~[apache-cassandra-3.11.6.jar:3.11.6] ... 28 common frames omitted {noformat} Please help me to resolve this issue > Modify cassandra.yaml and replace the default data location, still data > stored in default location > -------------------------------------------------------------------------------------------------- > > Key: CASSANDRA-15724 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15724 > Project: Cassandra > Issue Type: Bug > Components: Local/SSTable > Reporter: Boopalan > Priority: Normal > > I have created a separated directory (/mnt/) for cassandra data, commit log, > hint and saved_cache. But data are not created on created directory, stored > in default location /var/lib/cassandra/data. > > *Configuration File:* > [root@node-master cassandra]# cat /etc/cassandra/default.conf/cassandra.yaml > | grep /mnt/ > hints_directory: */mnt/*hints > - */mnt/*data > commitlog_directory: */mnt/*commitlog > cdc_raw_directory: */mnt/*cdc_raw > saved_caches_directory: */mnt/*saved_caches > > Please help me to resolve this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org