[
https://issues.apache.org/jira/browse/CASSANDRA-18737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17765282#comment-17765282
]
Jacek Lewandowski edited comment on CASSANDRA-18737 at 9/14/23 5:14 PM:
------------------------------------------------------------------------
Ahhh.... I think I've got it!
When we build an {{SSTableWriter}} using new builders, the empty files for
index and data components get create in the constructor, as we create partition
writer. It happens even before we start a transaction, so at that moment, we
have two sstable files and no transaction log. Whatever scans that directory at
that moment will that sstable as completed. I think this is what we are
encountering in that test, here is an example - stopped at the constructor of
sstable 9:
{noformat}
jlewandowski@deauville:~/dev/cassandra/wip[CASSANDRA-18737][j11]$ ls
/tmp/15704397017644002949/SSTableLoaderTest/Standard2-65d925a0532111eea827adf63f6cb5
be/ -l
total 264
-rw-rw-r-- 1 jlewandowski jlewandowski 575 wrz 14 19:11
nc-8-big-CompressionInfo.db
-rw-rw-r-- 1 jlewandowski jlewandowski 231990 wrz 14 19:11 nc-8-big-Data.db
-rw-rw-r-- 1 jlewandowski jlewandowski 9 wrz 14 19:11 nc-8-big-Digest.crc32
-rw-rw-r-- 1 jlewandowski jlewandowski 16 wrz 14 19:11 nc-8-big-Filter.db
-rw-rw-r-- 1 jlewandowski jlewandowski 5845 wrz 14 19:11 nc-8-big-Index.db
-rw-rw-r-- 1 jlewandowski jlewandowski 6327 wrz 14 19:11
nc-8-big-Statistics.db
-rw-rw-r-- 1 jlewandowski jlewandowski 111 wrz 14 19:11 nc-8-big-Summary.db
-rw-rw-r-- 1 jlewandowski jlewandowski 92 wrz 14 19:11 nc-8-big-TOC.txt
-rw-rw-r-- 1 jlewandowski jlewandowski 0 wrz 14 19:11 nc-9-big-Data.db
-rw-rw-r-- 1 jlewandowski jlewandowski 0 wrz 14 19:11 nc-9-big-Index.db
{noformat}
was (Author: jlewandowski):
Ahhh.... I think I've got it!
> Test failure: org.apache.cassandra.io.sstable.SSTableLoaderTest
> (testLoadingIncompleteSSTable-.jdk17)
> -----------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-18737
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18737
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
> Reporter: Brandon Williams
> Assignee: Jacek Lewandowski
> Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {noformat}
> java.lang.RuntimeException: Failed to list files in
> /tmp/1409486429862512729/SSTableLoaderTest/Standard2-57877ac036d311eea01f83fcb8f6fee5
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:77)
> at
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:626)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:103)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:202)
> at
> org.apache.cassandra.io.sstable.SSTableLoaderTest.testLoadingIncompleteSSTable(SSTableLoaderTest.java:213)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException:
> Corrupted:
> /tmp/1409486429862512729/SSTableLoaderTest/Standard2-57877ac036d311eea01f83fcb8f6fee5/nc-17-big
> at
> org.apache.cassandra.io.sstable.format.SSTableReaderLoadingBuilder.build(SSTableReaderLoadingBuilder.java:111)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:397)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.openForBatch(SSTableReader.java:373)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.lambda$openSSTables$0(SSTableLoader.java:152)
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.lambda$innerList$2(LogAwareFileLister.java:99)
> at
> java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:178)
> at
> java.base/java.util.TreeMap$EntrySpliterator.forEachRemaining(TreeMap.java:3287)
> at
> java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
> at
> java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
> at
> java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
> at
> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.innerList(LogAwareFileLister.java:101)
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:73)
> Caused by: java.lang.NullPointerException
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:903)
> at
> org.apache.cassandra.io.sstable.format.big.BigSSTableReaderLoadingBuilder.buildSummaryAndBloomFilter(BigSSTableReaderLoadingBuilder.java:193)
> at
> org.apache.cassandra.io.sstable.format.big.BigSSTableReaderLoadingBuilder.openComponents(BigSSTableReaderLoadingBuilder.java:116)
> at
> org.apache.cassandra.io.sstable.format.big.BigSSTableReaderLoadingBuilder.openComponents(BigSSTableReaderLoadingBuilder.java:58)
> at
> org.apache.cassandra.io.sstable.format.SSTableReaderLoadingBuilder.build(SSTableReaderLoadingBuilder.java:92)
> {noformat}
> Seen here:
> https://app.circleci.com/pipelines/github/driftx/cassandra/1174/workflows/263f1e22-e4d0-48b8-b3e2-496edb30a068/jobs/41924/tests
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
