[
https://issues.apache.org/jira/browse/CASSANDRA-716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Dusbabek updated CASSANDRA-716:
------------------------------------
Attachment: 0002-ensure-all-files-for-an-sstable-are-streamed-to-the-.patch
0001-rename-DD.getDAtaFileLocationForTable-to-something-m.patch
ensures that all files for a single sstable end up in the same directory. The
same round-robin approach is used in the case of multiple sstables/keyspaces.
> bootstrapping does not work properly using multiple DataFileDirectory
> ---------------------------------------------------------------------
>
> Key: CASSANDRA-716
> URL: https://issues.apache.org/jira/browse/CASSANDRA-716
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5
> Environment: storage-conf.xml:
> <CommitLogDirectory>/home/store1/commitlog</CommitLogDirectory>
> <DataFileDirectories>
> <DataFileDirectory>/home/store0/data</DataFileDirectory>
> <DataFileDirectory>/home/store1/data</DataFileDirectory>
> </DataFileDirectories>
> <CalloutLocation>/home/store1/cassandra/callouts</CalloutLocation>
> <StagingFileDirectory>/home/store1/cassandra/staging</StagingFileDirectory>
> Reporter: david.pan
> Assignee: Gary Dusbabek
> Attachments:
> 0001-rename-DD.getDAtaFileLocationForTable-to-something-m.patch,
> 0002-ensure-all-files-for-an-sstable-are-streamed-to-the-.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I was adding a new machine A which has 2 DataFileDirectories into the ring.
> The A will throw exception while bootstrapping.
> DEBUG [MESSAGING-SERVICE-POOL:4] 2010-01-19 11:43:32,837
> ContentStreamState.java (line 88) Removing stream context
> /home/store0/data/pic/raw_data-tmp-1-Data.db:209833142
> WARN [MESSAGING-SERVICE-POOL:4] 2010-01-19 11:43:32,837 TcpConnection.java
> (line 484) Problem reading from socket connected to :
> java.nio.channels.SocketChannel[connected local
> =/10.81.37.65:7000 remote=/10.81.42.26:10418]
> WARN [MESSAGING-SERVICE-POOL:4] 2010-01-19 11:43:32,837 TcpConnection.java
> (line 485) Exception was generated at : 01/19/2010 11:43:32 on thread
> MESSAGING-SERVICE-POOL:4
> java.io.IOException: rename failed of
> /home/store0/data/pic/raw_data-1-Filter.db
> java.io.IOError: java.io.IOException: rename failed of
> /home/store0/data/pic/raw_data-1-Filter.db
> at
> org.apache.cassandra.io.SSTableWriter.rename(SSTableWriter.java:154)
> at
> org.apache.cassandra.io.SSTableWriter.renameAndOpen(SSTableWriter.java:162)
> at
> org.apache.cassandra.io.Streaming$StreamCompletionHandler.onStreamCompletion(Streaming.java:284)
> at
> org.apache.cassandra.net.io.ContentStreamState.handleStreamCompletion(ContentStreamState.java:108)
> at
> org.apache.cassandra.net.io.ContentStreamState.read(ContentStreamState.java:90)
> at org.apache.cassandra.net.io.TcpReader.read(TcpReader.java:96)
> at
> org.apache.cassandra.net.TcpConnection$ReadWorkItem.run(TcpConnection.java:445)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: rename failed of
> /home/store0/data/pic/raw_data-1-Filter.db
> at
> org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:306)
> at
> org.apache.cassandra.io.SSTableWriter.rename(SSTableWriter.java:150)
> ... 9 more
> I traced the exception and maybe found the reason.
> StreamInitiateVerbHandler::doVerb() will create 3 temporary files(index,
> filter, data) for each ssTable. The name for each file is generated by
> getNewFileNameFromOldContextAndNames(). This method will generate a file name
> and a path for each ssTable, but the path is generated with
> DatabaseDescriptor.getDataFileLocationForTable() which will return different
> path for ech call when we configure multi-DataFileDirectory.
> eg: the ssTable raw_data-1 may have 3 temporary files :
> /home/store0/data/pic/raw_data-tmp-1-Index.db
> /home/store1/data/pic/raw_data-tmp-1-Filter.db
> /home/store0/data/pic/raw_data-tmp-1-Data.db
> After receiving all data, StreamCompletionHandler::onStreamCompletion() will
> rename all temporary files and this method think all ssTable files will have
> the same path as data.db file.
> if (streamContext.getTargetFile().contains("-Data.db"))
> {
> ......
> try
> {
> SSTableReader sstable =
> SSTableWriter.renameAndOpen(streamContext.getTargetFile());
> ......
> }
> ......
> }
> Then the renameAndOpen() will throw that exception.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
