[
https://issues.apache.org/jira/browse/AVRO-828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Douglas Britsch updated AVRO-828:
---------------------------------
Description: When opening an avro file to be written, the 16-byte, sync
marker for file not generated till first record written or the file properly
closed. Our usecase has the logs being rotated if they have no open filehandles
which means if we open a file and experience an unclean shutdown we get files
that throw exceptions on processing. When records are written the sync marker
is written on each write which prevents this from being a problem on non-empty
files. Can we write the sync marker when the header is written instead of on
first record write? (was: When opening an avro file to be written, the
16-byte, sync marker for file not generated till first record written or the
file properly closed. Our usecase has the logs being rotated if they have no
open filehandles which means if we open a file and experience an unclean
shutdown we get files that throw exceptions on processing. When records are
written the sync marker is written on each write which prevents this from being
a problem on non-empty files. Can we write the sync marker when the header is
written instead of on first write?)
> The 16-byte, sync marker for file not generated till first record written.
> --------------------------------------------------------------------------
>
> Key: AVRO-828
> URL: https://issues.apache.org/jira/browse/AVRO-828
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.1
> Environment: Java 1.6
> Reporter: Douglas Britsch
> Labels: empty, file, header, marker, sync
>
> When opening an avro file to be written, the 16-byte, sync marker for file
> not generated till first record written or the file properly closed. Our
> usecase has the logs being rotated if they have no open filehandles which
> means if we open a file and experience an unclean shutdown we get files that
> throw exceptions on processing. When records are written the sync marker is
> written on each write which prevents this from being a problem on non-empty
> files. Can we write the sync marker when the header is written instead of on
> first record write?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
