[
https://issues.apache.org/jira/browse/FLUME-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mubarak Seyed updated FLUME-1669:
---------------------------------
Description:
Motivation:
Columnar storage is preferred for better performance and compression for
low-latency analytical workloads. Avro 1.7.2 supports column-major file format
[1]
and we can implement {{AbstractTrevniColumnarEventSerializer}} (as like
{{AbstractAvroEventSerializer}}). {{HDFSSink}} can have serializer type to
store events in Trevni column-major file format.
[1] http://avro.apache.org/docs/current/trevni/spec.html
https://github.com/cutting/trevni
was:
Motivation:
Columnar storage is preferred for better performance and compression for
low-latency analytical workloads. Avro 1.7.2 supports column-major file format
[1]
and we can implement {{AbstractTrevniColumnarEventSerializer}} (as like
{{AbstractAvroEventSerializer}}). {{HDFSSink}} can have serializer type to
store events in Trevni column-major file format.
[1] https://github.com/cutting/
http://avro.apache.org/docs/current/trevni/spec.html
> Add support for columnar event serializer in HDFS
> -------------------------------------------------
>
> Key: FLUME-1669
> URL: https://issues.apache.org/jira/browse/FLUME-1669
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Reporter: Mubarak Seyed
> Assignee: Mubarak Seyed
> Labels: noob
> Fix For: v1.4.0
>
>
> Motivation:
> Columnar storage is preferred for better performance and compression for
> low-latency analytical workloads. Avro 1.7.2 supports column-major file
> format [1]
> and we can implement {{AbstractTrevniColumnarEventSerializer}} (as like
> {{AbstractAvroEventSerializer}}). {{HDFSSink}} can have serializer type to
> store events in Trevni column-major file format.
> [1] http://avro.apache.org/docs/current/trevni/spec.html
> https://github.com/cutting/trevni
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
