[
https://issues.apache.org/jira/browse/CASSANDRA-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12982401#action_12982401
]
Stu Hood commented on CASSANDRA-1472:
-------------------------------------
> What do you think?
I think that implementing a compressible block based file format is a
non-trivial task, and that before we commit to re-implementing Avro's (in a
bounded timeframe especially), we should review our requirements. This decision
needs to be made for technical reasons and not grounded in NIH.
After reviewing the Avro spec again, and having written a very similar file
format for #674, there is nothing I would change about the format:
* Has a header to store file format version, compression information and any
other arbitrary data (including Avro's schema)
* Is blocked based, with framing around the blocks for fast skipping, and with
synchronization points for recovering a corrupt file
* Implements object reuse: iterating over a file requires a single record
object, which is re-filled with data in the file
In the interest of full disclosure, Avro is lacking one serialization feature I
would like to see (AVRO-679), but there is a fair chance it will be implemented
in a future version, and until then we can trivially implement it above Avro.
http://avro.apache.org/docs/1.4.1/spec.html#Object+Container+Files
> Add bitmap secondary indexes
> ----------------------------
>
> Key: CASSANDRA-1472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1472
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Stu Hood
> Assignee: Stu Hood
> Fix For: 0.7.1
>
> Attachments: 0.7-1472-v5.tgz, 0.7-1472-v6.tgz,
> 0019-Rename-bugfixes-and-fileclose.txt, 1472-v3.tgz, 1472-v4.tgz,
> 1472-v5.tgz, anatomy.png, v4-bench-c32.txt
>
>
> Bitmap indexes are a very efficient structure for dealing with immutable
> data. We can take advantage of the fact that SSTables are immutable by
> attaching them directly to SSTables as a new component (supported by
> CASSANDRA-1471).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
