Hi,
I am writing a avro file, with 3 millions records, with each record having
around 40 values each.
The issue i am facing is that the writing is very slow, taking around 80
seconds.
I am using the avro value interface for this. Below are some of the API's
that i am using.
avro_value_get_by_name(&_row, name.c_str(), &_column, NULL);
avro_type_t type = avro_value_get_type(&_column);
if(type == AVRO_UNION)
{
avro_value_set_branch(&_column, 1, &_branch);
avro_value_set_int(&_branch, value);
}
else
avro_value_set_int(&_branch, value);
once all the values for a record are set, a append the record to the writer.
avro_file_writer_append_value(_writer, &_row);
avro_value_decref(&_row);
avro_generic_value_new(_writer_iface, &_row);
Am i doing something wrong in this?
Is there any way to increase the speed at which i am writing the data?
Thanks
Amit
