[
https://issues.apache.org/jira/browse/CASSANDRA-10990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15181464#comment-15181464
]
Paulo Motta commented on CASSANDRA-10990:
-----------------------------------------
Updated branch with reworked {{RewindableDataInputStreamPlus}}, now treating
the spill file as circular buffer with max size. Now required space for spill
buffer file of legacy {{StreamDeserializer}} is {{max(sstableSize, MAX_INT)}},
capping the max required space for streaming older version sstables on ~2GB. I
also updated documentation and tests. WDYT [~yukim]?
Besides that, I added a new
[dtest|https://github.com/pauloricardomg/cassandra-dtest/commit/b59ea329a5b9302372c57956f7015111fd47a4ae]
for testing repair of older version sstables on 3.0+, and it seems to work.
Now we have the following dtests:
* [bootstrap
dtests|https://github.com/pauloricardomg/cassandra-dtest/blob/10990/upgrade_8099_test.py#L340]
* [sstable loader
dtests|https://github.com/pauloricardomg/cassandra-dtest/blob/10990/sstable_generation_loading_test.py#L181]
* [repair
dtests|https://github.com/pauloricardomg/cassandra-dtest/commit/b59ea329a5b9302372c57956f7015111fd47a4ae]
For simplicity, I opted for writing the old streamed sstables in the new format
without {{EncodingStats}} (since there is no {{SerializationHeader}}
available), what may make these sstables less optimized in terms of storage
space from what I understood. Do you think we should construct these stats when
receiving the sstable or not bother [~slebresne] ? Also, do you recall of any
other edge case we should watch here?
Thanks!
> Support streaming of older version sstables in 3.0
> --------------------------------------------------
>
> Key: CASSANDRA-10990
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10990
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
> Reporter: Jeremy Hanna
> Assignee: Paulo Motta
>
> In 2.0 we introduced support for streaming older versioned sstables
> (CASSANDRA-5772). In 3.0, because of the rewrite of the storage layer, this
> became no longer supported. So currently, while 3.0 can read sstables in the
> 2.1/2.2 format, it cannot stream the older versioned sstables. We should do
> some work to make this still possible to be consistent with what
> CASSANDRA-5772 provided.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
