json2sstable should support streaming
-------------------------------------
Key: CASSANDRA-1898
URL: https://issues.apache.org/jira/browse/CASSANDRA-1898
Project: Cassandra
Issue Type: Improvement
Components: Tools
Affects Versions: 0.7.0 rc 2, 0.7.0
Reporter: Nick Bailey
Fix For: 0.7.1
json2sstable loads the entire json file into memory. This is so it can sort the
file before creating an sstable. If the file was created using sstable2json and
the partitioner isn't changing, this isn't necessary. For very large files
this means json2sstable requires a huge amount of memory.
There should be an option to stream the file. A simple check for out of order
keys will prevent writing bad sstables.
This should be possible with the SAX style parser available in our current json
library.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
