[
https://issues.apache.org/jira/browse/AVRO-806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13446228#comment-13446228
]
Doug Cutting commented on AVRO-806:
-----------------------------------
I'd like to bring the Trevni (https://github.com/cutting/trevni) code to
Apache. How do folks think we ought to do this?
Trevni's serialization, columns of scalar values, is different from Avro. It
doesn't require a schema, but implements a mapping between Avro schemas and
columns. I see three options:
# incorporate it into Avro's lang/* tree. Currently Trevni has only a Java
implementation, so its code could be merged into lang/java as new modules. The
trevni-core module is independent of Avro, but the trevni-avro module depends
on other Avro modules.
# have an independent code tree for Trevni but still managed by the Avro PMC.
If we expect that folks from Avro will also be the folks who work on Trevni
then having the Avro project produce Trevni as a separately released product
might be reasonable.
# submit an incubator proposal for Trevni, aiming for an independent TLP. I
worry that Trevni's too small to sustain an independent community and that
bundling it with Avro might be best.
Which do folks think is best? I'm leaning towards (1). Any objections to
that? If not, I'll prepare a patch.
> add a column-major codec for data files
> ---------------------------------------
>
> Key: AVRO-806
> URL: https://issues.apache.org/jira/browse/AVRO-806
> Project: Avro
> Issue Type: New Feature
> Components: java, spec
> Reporter: Doug Cutting
> Assignee: Doug Cutting
> Attachments: AVRO-806.patch, AVRO-806-v2.patch, avro-file-columnar.pdf
>
>
> Define a codec that, when a data file's schema is a record schema, writes
> blocks within the file in column-major order. This would permit better
> compression and also permit efficient skipping of fields that are not of
> interest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
