[
https://issues.apache.org/jira/browse/CASSANDRA-11875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15306004#comment-15306004
]
Paulo Motta commented on CASSANDRA-11875:
-----------------------------------------
Thanks for the update! See follow-up comments below:
* There is still some common code between {{StandaloneConverter}} and
{{StandaloneUpgrader}} main methods, most of it can be extracted to common
methods use by both.
* The supported version check will probably be used in other places, so we
should probably move it to {{BigVersion}}. There are already {{isCompatible}}
and {{isCompatibleForStream}} methods, so we can maybe add a
{{isCompatibeForWriting}}. It would be nice if you could add a unit test that
checks that trying to converting to an unsupported version should fail.
bq. Also added a new Test suit SSTableConversionTest, but it has some problem
when performing conversion.
It seems this is due to re-loading the schema on
{{Schema.instance.loadFromDisk(false)}} on {{StandaloneConverter}}. Since we
want to focus testing the conversion itself, we will probably have more
flexibility testing the internal class {{SSTableConverter}}, so we don't have
to add special options for testing on {{StandaloneConverter}} and can also play
around with SSTableReaders directly.
Instead of basing our tests on {{SSTableRewriterTest}} as initially discussed,
it's probably more convenient to base it on {{CQLTester}} since we will be
doing data conversions and testing this at a higher level with CQL is the only
way to ensure converted data is being interpreted correctly. The tests should
have more or less the following structure:
* Insert Data with CQL
* Flush to disk
* Read and verify data in current version with CQL
* Keep reference to original SSTableReaders and cleanup ColumnFamilyStore
(clearUnsafe)
* Perform conversion on original SSTableReaders
* Verify metadata was converted correctly on converted SSTableReaders
* Add converted SSTableReaders to ColumnFamilyStore (addSSTable)
* Read and verify data in converted version with CQL
For the mb to ma conversion, since there is no data conversion involved (only
metadata), you can use {{SimpleQueryTest.testTableWithoutClustering}} as a
example to write the first test case. You may use utility {{CQLTester}} utility
methods such as {{flush()}} and {{getCurrentColumnFamilyStore()}} to flush and
access {{ColumnFamilyStore}}.
> Create sstableconvert tool with support to ma format
> ----------------------------------------------------
>
> Key: CASSANDRA-11875
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11875
> Project: Cassandra
> Issue Type: Sub-task
> Components: Tools
> Reporter: Paulo Motta
> Assignee: Kaide Mu
> Priority: Minor
> Attachments: trunk-11875-WIP-V1.patch
>
>
> Currently {{Upgrader}} receives an sstable in any readable format, and writes
> into {{BigFormat.getLatestVersion()}}. We should generalize it by making it
> receive a {{target}} version and probably also rename it to
> {{SSTableConverter}}.
> Based on this we can create an {{StandaloneDowngrader}} tool which will
> perform downgrade of specified sstables to a target version. To start with,
> we should support only downgrading to {{ma}} format (from current format
> {{mb}}), downgrade to any other version should be forbidden. Since we already
> support serializing to "ma" we will not need to do any data conversion.
> We should also create a test suite that creates an sstable with data in the
> current format, perform the downgrade, and verify data in the new format is
> correct. This will be the base tests suite for more advanced conversions in
> the future.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
