[
https://issues.apache.org/jira/browse/AVRO-3684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17643758#comment-17643758
]
Christophe Le Saec commented on AVRO-3684:
------------------------------------------
I test with using SyncableFileOutputStream(File, boolean append) constructor,
and separate file to copy from target file
Here, the code, and it works fine.
{code:java}
private void testAppendStream(CodecFactory codec) throws IOException {
File file = makeFile(codec);
File filein = new File(file.getParentFile(),
file.getName().substring(0, file.getName().length() - 5) + "Copy.avro");
DatumWriter<Object> datumWriter = new SpecificDatumWriter<>();
// write COUNT objects to datafile
try (DataFileWriter<Object> writer = new DataFileWriter<>(datumWriter);
SyncableFileOutputStream fileOutputStream = new
SyncableFileOutputStream(file, true)) {
writer.create(SCHEMA, fileOutputStream);
for (Object datum : new RandomData(SCHEMA, COUNT, SEED)) {
writer.append(datum);
writer.flush();
}
writer.sync();
}
Files.copy(file.toPath(), filein.toPath());
// append to existing file
try (DataFileWriter<Object> writer = new DataFileWriter<>(datumWriter);
SyncableFileOutputStream fileOutputStream = new
SyncableFileOutputStream(file, true);
SeekableFileInput in = new SeekableFileInput(filein)) {
// append to existing fileStream
writer.appendTo(in, fileOutputStream);
for (Object datum : new RandomData(SCHEMA, COUNT, SEED + 1)) {
writer.append(datum);
writer.flush();
}
}
// verify objects in file
long recordCounter = 0;
DatumReader<Object> datumReader = new SpecificDatumReader<>();
try (FileReader<? extends Object> reader = DataFileReader.openReader(file,
datumReader)) {
while (reader.hasNext()) {
reader.next();
recordCounter++;
}
}
assertEquals(COUNT * 2, recordCounter);
}
{code}
I didn't made fine analyze to see why this code work better, but hope it will
help to solve your issue.
> appending to existing file with outputstream not working
> --------------------------------------------------------
>
> Key: AVRO-3684
> URL: https://issues.apache.org/jira/browse/AVRO-3684
> Project: Apache Avro
> Issue Type: Bug
> Reporter: Mikko Kortelainen
> Priority: Major
>
> appending to existing file with outputstream not working
>
> i created a test case at [https://github.com/apache/avro/pull/2000/files] for
> the reference. If this is an incorrect way then please redirect to
> documentation or if no such thing exist, please link to existing working
> example or test case.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
