[
https://issues.apache.org/jira/browse/AVRO-3768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christophe Le Saec updated AVRO-3768:
-------------------------------------
Attachment: Main.java
GenericDataIteratorInputStream.java
IteratorInputStream.java
> Provide InputStream implementation that wraps an Iterator of Record
> -------------------------------------------------------------------
>
> Key: AVRO-3768
> URL: https://issues.apache.org/jira/browse/AVRO-3768
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
> Reporter: Victor
> Priority: Major
> Attachments: GenericDataIteratorInputStream.java,
> IteratorInputStream.java, Main.java
>
>
> Hello,
> I have some code that generates avro record (using GenericData.Record
> precisely) record per record (I used an Iterator<GenericData.Record> in
> practice, but we could imagine anything similar including an actual
> java.util.Stream or even an avro-provided interface), and I would like to
> serialized it to some external system (an http request in my particular
> case). So basically the data is generated as it is pulled.
> Right now, the only option I found is to use a combination of
> java.io.PipedInputStream and PipedOutputStream, wrapping the later inside a
> DataFileWriter and then feeding the records to the writer (in a separate
> thread) so that the PipedInputStream can be read by anything else.
> As you can see this is a bit cumbersome and a more straightforward approach
> would be welcomed. I tried to implement this myself but I admit I got lost in
> all the moving pieces and I couldn't find a simple way of doing that without
> implementing a lot of low-level stuff. So maybe I'm missing something or we
> could add something to avro :)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
