[
https://issues.apache.org/jira/browse/AVRO-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561161#comment-16561161
]
Thiruvalluvan M. G. commented on AVRO-2178:
-------------------------------------------
It is an interesting use case.
I don't think implementation Avro data file in any language in this project
will handle this use case well.
Looking at C++ implementation specifically, we have an underlying byte stream
which can be "backed up" a bit, but it is not guaranteed to work. I don't think
it is going to be easy to support this use case.
If you can take the latency impact, your reader process will have to wait for
the writer to complete writing the whole file. The easiest way to achieve this
is to let the writer to write into a temporary file and when done let the file
be renamed to the actual destination name. This approach has a side benefit
that partially written files are never visible to potential consumers.
> avro C++ api support of tail reading of a growing avro file
> -----------------------------------------------------------
>
> Key: AVRO-2178
> URL: https://issues.apache.org/jira/browse/AVRO-2178
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Affects Versions: 1.8.2
> Reporter: peien
> Priority: Major
>
> Two processes, one is writing to an avro data file, another wishes to read
> the latest written data.
> The problem with current C++ API is that when it reaches the EOF, an
> exception will be thrown, and from the user perspective, I have no way to
> retry or 'tail read' it again from the last good position.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
