Kevin A Shaw created AVRO-3472:
----------------------------------
Summary: Writing to an object store (without intermediate file)
Key: AVRO-3472
URL: https://issues.apache.org/jira/browse/AVRO-3472
Project: Apache Avro
Issue Type: New Feature
Components: java
Environment: OpenJDK version 11 on Ubuntu
Reporter: Kevin A Shaw
We need to write a large number files for a production system to an Object
Store (AWS S3 in this case) which cannot be represented by a java.io.File
object. We are currently using a DataFileWriter() to create a file in a /tmp
directory, putting it to S3 and then deleting the file. This is becoming
cumbersome and just seems inefficient. Is there a way to instead create the
file using an in-memory object, instead of writing to disk?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
