[jira] [Commented] (AVRO-1387) DataFileWriter must allow files not to be flushed at the end of every block.
[ https://issues.apache.org/jira/browse/AVRO-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13805752#comment-13805752 ] ASF subversion and git services commented on AVRO-1387: --- Commit 1535888 from [~cutting] in branch 'avro/trunk' [ https://svn.apache.org/r1535888 ] AVRO-1387. Java: Add DataFileWriter option to inhibit flush per block. Contributed by Hari Shreedharan. DataFileWriter must allow files not to be flushed at the end of every block. Key: AVRO-1387 URL: https://issues.apache.org/jira/browse/AVRO-1387 Project: Avro Issue Type: Bug Reporter: Hari Shreedharan Assignee: Hari Shreedharan Fix For: 1.7.6 Attachments: AVRO-1387.patch We are considering changes in Flume's file channel to use Avro, one of the requirements is that each event (which maps to one avro record) be checksummed so we know if the data is corrupt. We'd probably have to add a new version for this, since this will change the data format on disk. I can start working on a Java version if there are no objections -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (AVRO-1387) DataFileWriter must allow files not to be flushed at the end of every block.
[ https://issues.apache.org/jira/browse/AVRO-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13805929#comment-13805929 ] Hudson commented on AVRO-1387: -- SUCCESS: Integrated in AvroJava #399 (See [https://builds.apache.org/job/AvroJava/399/]) AVRO-1387. Java: Add DataFileWriter option to inhibit flush per block. Contributed by Hari Shreedharan. (cutting: rev 1535888) * /avro/trunk/CHANGES.txt * /avro/trunk/lang/java/avro/src/main/java/org/apache/avro/file/DataFileStream.java * /avro/trunk/lang/java/avro/src/main/java/org/apache/avro/file/DataFileWriter.java * /avro/trunk/lang/java/avro/src/test/java/org/apache/avro/TestDataFile.java DataFileWriter must allow files not to be flushed at the end of every block. Key: AVRO-1387 URL: https://issues.apache.org/jira/browse/AVRO-1387 Project: Avro Issue Type: Bug Reporter: Hari Shreedharan Assignee: Hari Shreedharan Fix For: 1.7.6 Attachments: AVRO-1387.patch We are considering changes in Flume's file channel to use Avro, one of the requirements is that each event (which maps to one avro record) be checksummed so we know if the data is corrupt. We'd probably have to add a new version for this, since this will change the data format on disk. I can start working on a Java version if there are no objections -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (AVRO-1387) DataFileWriter must allow files not to be flushed at the end of every block.
[ https://issues.apache.org/jira/browse/AVRO-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13804490#comment-13804490 ] Hari Shreedharan commented on AVRO-1387: [~cutting] - The attached patch is ready for review. Please take a look when you get some time. Thanks! DataFileWriter must allow files not to be flushed at the end of every block. Key: AVRO-1387 URL: https://issues.apache.org/jira/browse/AVRO-1387 Project: Avro Issue Type: Bug Reporter: Hari Shreedharan Assignee: Hari Shreedharan Attachments: AVRO-1387.patch We are considering changes in Flume's file channel to use Avro, one of the requirements is that each event (which maps to one avro record) be checksummed so we know if the data is corrupt. We'd probably have to add a new version for this, since this will change the data format on disk. I can start working on a Java version if there are no objections -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (AVRO-1387) DataFileWriter must allow files not to be flushed at the end of every block.
[ https://issues.apache.org/jira/browse/AVRO-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13800182#comment-13800182 ] Hari Shreedharan commented on AVRO-1387: I updated the jira title to reflect the current discussion. DataFileWriter must allow files not to be flushed at the end of every block. Key: AVRO-1387 URL: https://issues.apache.org/jira/browse/AVRO-1387 Project: Avro Issue Type: Bug Reporter: Hari Shreedharan We are considering changes in Flume's file channel to use Avro, one of the requirements is that each event (which maps to one avro record) be checksummed so we know if the data is corrupt. We'd probably have to add a new version for this, since this will change the data format on disk. I can start working on a Java version if there are no objections -- This message was sent by Atlassian JIRA (v6.1#6144)
