Hi Users,
I am trying to debug a rather unusual bug where Avro tool fails to read a
simple file. I compiled Avro on Windows as explained in
README.maintaining_win32.txt. I am using this library with a program I have
attached to the mail.
When I execute the program it creates the output.avro file. Depending on
the inner loop, if its 5 times - the generated Avro file reading fails as
under
$ java -jar "D:\jobs_and_tasks\avro-tools.jar" tojson
"D:\jobs_and_tasks\simple_copy\output.avro"
Exception in thread "main" org.apache.avro.AvroRuntimeException:
java.io.IOException: Invalid sync!
at
org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:210)
at
org.apache.avro.tool.DataFileReadTool.run(DataFileReadTool.java:63)
at org.apache.avro.tool.Main.run(Main.java:84)
at org.apache.avro.tool.Main.main(Main.java:73)
If the loop is anything but 5, it works. for e.g: loop of 6
$ java -jar "D:\jobs_and_tasks\avro-tools.jar" tojson
"D:\jobs_and_tasks\simple_copy\output.avro"
{"field1":"a"}
{"field1":"a"}
{"field1":"a"}
{"field1":"a"}
{"field1":"a"}
{"field1":"a"}
Has anyone seen this error/behavior before ?
Best,
Ujjwal
#include "avro.h"
#include "avro_private.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
avro_schema_t avroSchema;
avro_file_writer_t writer;
avro_value_iface_t *iface_ptr;
avro_value_t record, field;
const char *schemaString =
"{"
" \"type\":\"record\","
" \"name\":\"recordlayout_output\","
" \"fields\":["
" {\"name\":\"field1\",\"type\" :\"string\"}"
" ]"
"}";
int main(int argc, char *argv[])
{
avro_schema_from_json_length(schemaString, strlen(schemaString), &avroSchema);
iface_ptr = avro_generic_class_from_schema(avroSchema);
avro_generic_value_new(iface_ptr, &record);
FILE* fileHandlePtr = fopen("output.avro","wb");
avro_file_writer_create_with_codec_fp(fileHandlePtr, "", 0, avroSchema, &writer, "null", (16 * 1024));
for(int i =0;i<5;i++)
{
avro_value_get_by_index(&record, 0, &field, NULL);
avro_value_set_string_len(&field, "a", 2);
avro_file_writer_append_value(writer, &record);
}
avro_file_writer_close(writer);
avro_value_decref(&record);
avro_value_iface_decref(iface_ptr);
avro_schema_decref(avroSchema);
return 0;
}