[
https://issues.apache.org/jira/browse/AVRO-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13799376#comment-13799376
]
Doug Cutting commented on AVRO-1387:
------------------------------------
> is it reasonable to write one block per record and write a sync marker at the
> end of a record?
I believe that each block written causes a flush of the output stream, which is
a system call. This could probably be avoided if it's found to slow things
considerably, though we might want to make that optional, since folks might
depend on the current flush-per-block behavior. A system call every few
hundred bytes though might be tolerable.
> Avro container file format update to write checksums for individual record
> --------------------------------------------------------------------------
>
> Key: AVRO-1387
> URL: https://issues.apache.org/jira/browse/AVRO-1387
> Project: Avro
> Issue Type: Bug
> Reporter: Hari Shreedharan
>
> We are considering changes in Flume's file channel to use Avro, one of the
> requirements is that each event (which maps to one avro record) be
> checksummed so we know if the data is corrupt.
> We'd probably have to add a new version for this, since this will change the
> data format on disk. I can start working on a Java version if there are no
> objections
--
This message was sent by Atlassian JIRA
(v6.1#6144)
