[
https://issues.apache.org/jira/browse/CASSANDRA-17933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17612812#comment-17612812
]
Stefan Miklosovic commented on CASSANDRA-17933:
-----------------------------------------------
[~maedhroz]
Yes, I can do such test. This is a valid point, actually. There is not a lot of
information available online about the relationship between these two files. It
seems like metadata.cq4t is "static". It does not change its size when new cq4
files are created (seems like it). The actual size and allocated size might be
different though. (ls -la vs ls -ls) I will investigate if we are safe to
delete it on a deeper level and I ll add that test you mentioned.
By the way, we do not know what has caused the creation of that empty file we
try to delete. That bothers me too.
> Zero length file in Audit log folder, prevents a node from starting
> -------------------------------------------------------------------
>
> Key: CASSANDRA-17933
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17933
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Startup and Shutdown
> Reporter: Andrew Hogg
> Assignee: Stefan Miklosovic
> Priority: Normal
> Fix For: 4.0.x, 4.1.x, 4.x
>
>
> We have encountered a 4.0.3 cluster where the audit log folder had a zero
> byte length file within it after the node had stopped. It is not clear how
> Cassandra got to the point of this file existing. On restarting the node, the
> node will not start and throws the following stack trace.
> {code:java}
> ERROR [main] 2022-09-26 14:01:27,892 CassandraDaemon.java:911 - Exception
> encountered during startup
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:468)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:765)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:889)
> Caused by: org.apache.cassandra.exceptions.ConfigurationException: Unable to
> create instance of IAuditLogger.
> at
> org.apache.cassandra.utils.FBUtilities.newAuditLogger(FBUtilities.java:686)
> at
> org.apache.cassandra.audit.AuditLogManager.getAuditLogger(AuditLogManager.java:95)
> at
> org.apache.cassandra.audit.AuditLogManager.<init>(AuditLogManager.java:74)
> at
> org.apache.cassandra.audit.AuditLogManager.<clinit>(AuditLogManager.java:60)
> ... 3 common frames omitted
> Caused by: java.lang.reflect.InvocationTargetException: null
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.cassandra.utils.FBUtilities.newAuditLogger(FBUtilities.java:682)
> ... 6 common frames omitted
> Caused by: java.nio.channels.OverlappingFileLockException: null
> at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)
> at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)
> at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1068)
> at java.nio.channels.FileChannel.lock(FileChannel.java:1053)
> at
> net.openhft.chronicle.bytes.MappedFile.resizeRafIfTooSmall(MappedFile.java:369)
> at
> net.openhft.chronicle.bytes.MappedFile.acquireByteStore(MappedFile.java:307)
> at
> net.openhft.chronicle.bytes.MappedFile.acquireByteStore(MappedFile.java:269)
> at
> net.openhft.chronicle.bytes.MappedBytes.acquireNextByteStore0(MappedBytes.java:434)
> at
> net.openhft.chronicle.bytes.MappedBytes.readVolatileInt(MappedBytes.java:792)
> at
> net.openhft.chronicle.queue.impl.single.SingleChronicleQueue$StoreSupplier.headerRecovery(SingleChronicleQueue.java:1027)
> at
> net.openhft.chronicle.queue.impl.single.SingleChronicleQueue$StoreSupplier.acquire(SingleChronicleQueue.java:981)
> at
> net.openhft.chronicle.queue.impl.WireStorePool.acquire(WireStorePool.java:53)
> at
> net.openhft.chronicle.queue.impl.single.SingleChronicleQueue.cleanupStoreFilesWithNoData(SingleChronicleQueue.java:821)
> at
> net.openhft.chronicle.queue.impl.single.StoreAppender.<init>(StoreAppender.java:75)
> at
> net.openhft.chronicle.queue.impl.single.SingleChronicleQueue.newAppender(SingleChronicleQueue.java:422)
> at
> net.openhft.chronicle.core.threads.CleaningThreadLocal.initialValue(CleaningThreadLocal.java:54)
> at java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:180)
> at java.lang.ThreadLocal.get(ThreadLocal.java:170)
> at
> net.openhft.chronicle.core.threads.CleaningThreadLocal.get(CleaningThreadLocal.java:59)
> at
> net.openhft.chronicle.queue.impl.single.SingleChronicleQueue.acquireAppender(SingleChronicleQueue.java:441)
> at org.apache.cassandra.utils.binlog.BinLog.<init>(BinLog.java:133)
> at org.apache.cassandra.utils.binlog.BinLog.<init>(BinLog.java:65)
> at
> org.apache.cassandra.utils.binlog.BinLog$Builder.build(BinLog.java:453)
> at
> org.apache.cassandra.audit.BinAuditLogger.<init>(BinAuditLogger.java:55)
> ... 11 common frames omitted {code}
> To reproduce, we place a zero length file and attempted to start the node,
> and saw the same stack trace.
> {code:java}
> ll ../logs/audit/
> total 4
> rw-rw-r-. 1 automaton automaton 0 Sep 28 13:00 20220928-12.cq4
> rw-rw-r-. 1 automaton automaton 131072 Sep 28 13:00 metadata.cq4t {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
