mukesh katariya created AVRO-2166:
-------------------------------------
Summary: Wrong Sync header appended to the file.
Key: AVRO-2166
URL: https://issues.apache.org/jira/browse/AVRO-2166
Project: Avro
Issue Type: Bug
Components: java
Affects Versions: 1.8.2
Reporter: mukesh katariya
Attachments: corrected-manualy.PNG, original-snappy-file.PNG
We are using the following code, pseudo code for representation.
{code:java}
File file1 = new File("/tmp/tmpdata/message-" + System.currentTimeMillis());
FileOutputStream out = new FileOutputStream(file1, false);
DatumWriter datumWriter = new GenericDatumWriter<>();
writer = new DataFileWriter(datumWriter);
writer.setCodec(CodecFactory.snappyCodec());
writer.create(avroSchema, out);
//Records are from sink connector of kafka.
//Iterator over the records and bundling them together.
for( set of records ... )
writer.append(value);
writer.close();
{code}
The issue which we are facing is the sync code is not matching.
Original file with sync code at the end is wrong. avro tool failed with error
invalid sync.
Since the header.sync and sync bytes after the block didnot match.
!original-snappy-file.PNG! manually -replaced the 16 byte with sync bytes from
the header section. the file i was able to read using avro tools.
!corrected-manualy.PNG!
Thanks and Best Regards
Mukesh
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
