[
https://issues.apache.org/jira/browse/FLUME-776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472150#comment-13472150
]
Mike Percy commented on FLUME-776:
----------------------------------
I think this issue is resolved.
File serialization:
http://flume.apache.org/releases/content/1.2.0/apidocs/org/apache/flume/serialization/EventSerializer.html
Avro file serialization:
http://flume.apache.org/releases/content/1.2.0/apidocs/org/apache/flume/serialization/AbstractAvroEventSerializer.html
HBase serialization:
http://flume.apache.org/releases/content/1.2.0/apidocs/org/apache/flume/sink/hbase/HbaseEventSerializer.html
> Create generic APIs for input / output formats and serialization
> ----------------------------------------------------------------
>
> Key: FLUME-776
> URL: https://issues.apache.org/jira/browse/FLUME-776
> Project: Flume
> Issue Type: New Feature
> Affects Versions: NG alpha 1
> Reporter: E. Sammer
> Priority: Blocker
> Fix For: v1.3.0
>
>
> Flume should have a generic set of APIs to handle input and output formats as
> well as event serialization.
> These APIs should offer the same level of abstraction as Hadoop's
> InputFormat, OutputFormat, RecordReader, RecordWriter, and serializer
> interfaces / classes. The only rationale for not using Hadoop's specific
> implementation of these APIs is because we want to avoid that dependency and
> everything that comes with it. Examples of API usage would be:
> * HDFS sink, text file output, events serialized as JSON
> * HDFS sink, text file output, events serialized as text, Snappy compressed
> * HDFS sink, Avro file output, events serialized as Avro records, GZIP
> compressed.
> * HBase sink, event fields[1] serialized as Thrift
> [1] The case of HBase is odd in that the event needs to be broken into
> individual fields (i.e. extracted to a complex type). This means some kind of
> custom mapping / extraction code or configuration needs to supplied by the
> user; we're not overly concerned with that for this issue.
> The implementations of the formats (text file, Avro), serializations (JSON,
> Avro, Thrift), and compression codecs (Snappy, GZIP) listed above are just
> examples. We'll open separate JIRAs for implementations. The scope of this
> JIRA is the framework / infrastructure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
