[
https://issues.apache.org/jira/browse/FLUME-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254480#comment-13254480
]
[email protected] commented on FLUME-1117:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4708/#review6928
-----------------------------------------------------------
I have one general question about this. I am not entirely sure that the
serializer should handle the stream directly. I think it would be better if we
simply return the serialized event as a byte array from the EventSerializer,
which the class that deals with the output streams(the sinks etc) can deal with
directly. This would in general remove the requirement of the serializer
component to have a bunch of functions which are not associated with
serializing at all, but with writing/flushing etc of the output stream. The
serializer should simply return a serialized event which the sink can write to
the stream. The serializer should not really be concerned about flushing the
stream etc.
The event serializer in my opinion should be simple. It knows the schema and
simply returns the byte array which can be written.
- Hari
On 2012-04-12 13:06:04, Mike Percy wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4708/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-12 13:06:04)
bq.
bq.
bq. Review request for Flume.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Patch with support for Avro Container File format.
bq.
bq.
bq. This addresses bug FLUME-1117.
bq. https://issues.apache.org/jira/browse/FLUME-1117
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. flume-ng-core/pom.xml 37fb112
bq.
flume-ng-core/src/main/java/org/apache/flume/formatter/output/EventFormatter.java
c8b953d
bq.
flume-ng-core/src/main/java/org/apache/flume/formatter/output/TextDelimitedOutputFormatter.java
a12afd3
bq.
flume-ng-core/src/main/java/org/apache/flume/serialization/AbstractAvroEventSerializer.java
PRE-CREATION
bq.
flume-ng-core/src/main/java/org/apache/flume/serialization/BodyTextEventSerializer.java
PRE-CREATION
bq.
flume-ng-core/src/main/java/org/apache/flume/serialization/EventSerializer.java
PRE-CREATION
bq.
flume-ng-core/src/main/java/org/apache/flume/serialization/EventSerializerFactory.java
PRE-CREATION
bq.
flume-ng-core/src/main/java/org/apache/flume/serialization/EventSerializerType.java
PRE-CREATION
bq.
flume-ng-core/src/main/java/org/apache/flume/serialization/FlumeEventAvroEventSerializer.java
PRE-CREATION
bq. flume-ng-core/src/main/java/org/apache/flume/sink/RollingFileSink.java
a2f4f66
bq.
flume-ng-core/src/test/java/org/apache/flume/serialization/SyslogAvroEventSerializer.java
PRE-CREATION
bq.
flume-ng-core/src/test/java/org/apache/flume/serialization/TestBodyTextEventSerializer.java
PRE-CREATION
bq.
flume-ng-core/src/test/java/org/apache/flume/serialization/TestFlumeEventAvroEventSerializer.java
PRE-CREATION
bq.
flume-ng-core/src/test/java/org/apache/flume/serialization/TestSyslogAvroEventSerializer.java
PRE-CREATION
bq. flume-ng-core/src/test/resources/syslog_event.avsc PRE-CREATION
bq.
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java
fdad75b
bq.
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSCompressedDataStream.java
c8e1df9
bq.
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSDataStream.java
39a4456
bq.
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java
9f28d82
bq.
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSSequenceFile.java
49a62df
bq.
flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSWriter.java
2e5470e
bq.
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/MockHDFSWriter.java
73d3284
bq.
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestBucketWriter.java
fb61092
bq.
flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java
5a6ad47
bq. pom.xml 3a9bc42
bq.
bq. Diff: https://reviews.apache.org/r/4708/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Unit tests pass. New unit tests added for new functionality.
bq.
bq.
bq. Thanks,
bq.
bq. Mike
bq.
bq.
> Support output to files in Avro container format
> ------------------------------------------------
>
> Key: FLUME-1117
> URL: https://issues.apache.org/jira/browse/FLUME-1117
> Project: Flume
> Issue Type: Improvement
> Affects Versions: v1.2.0
> Reporter: Mike Percy
> Assignee: Mike Percy
> Fix For: v1.2.0
>
>
> Support output of events to a file in Avro container format.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
