[
https://issues.apache.org/jira/browse/FLUME-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485586#comment-13485586
]
Mubarak Seyed commented on FLUME-1669:
--------------------------------------
Trevi in Avro 1.7.2 supports two writers
1. Writes data to a Trevni column file -
http://avro.apache.org/docs/current/api/java/org/apache/trevni/ColumnFileWriter.html
2. Write Avro records to a Trevni column file -
http://avro.apache.org/docs/current/api/java/org/apache/trevni/avro/AvroColumnWriter.html
I am thinking about #2 (Avro records)
> Add support for columnar event serializer in HDFS
> -------------------------------------------------
>
> Key: FLUME-1669
> URL: https://issues.apache.org/jira/browse/FLUME-1669
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Reporter: Mubarak Seyed
> Assignee: Mubarak Seyed
> Labels: noob
> Fix For: v1.4.0
>
>
> Motivation:
> Columnar storage is preferred for better performance and compression for
> low-latency analytical workloads. Avro 1.7.2 supports column-major file
> format [1]
> and we can implement {{AbstractTrevniColumnarEventSerializer}} (as like
> {{AbstractAvroEventSerializer}}). {{HDFSSink}} can have serializer type to
> store events in Trevni column-major file format.
> [1] http://avro.apache.org/docs/current/trevni/spec.html
> https://github.com/cutting/trevni
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
