C implementation does not write datum values that are larger than the memory
write buffer (currently 16K)
---------------------------------------------------------------------------------------------------------
Key: AVRO-724
URL: https://issues.apache.org/jira/browse/AVRO-724
Project: Avro
Issue Type: Bug
Components: c
Affects Versions: 1.4.1
Reporter: Jeremy Hinegardner
The current C implementation does not allow for datum values greater than 16K.
The {{avro_file_writer_append}} flushes blocks to disk over time, but does not
deal with the single case of a single datum being larger than
{{avro_file_writer_t.datum_buffer}}. This is noted in the source code:
{code:title=datafile.c:294-313}
int avro_file_writer_append(avro_file_writer_t w, avro_datum_t datum)
{
int rval;
if (!w || !datum) {
return EINVAL;
}
rval = avro_write_data(w->datum_writer, w->writers_schema, datum);
if (rval) {
check(rval, file_write_block(w));
rval =
avro_write_data(w->datum_writer, w->writers_schema, datum);
if (rval) {
/* TODO: if the datum encoder larger than our buffer,
just write a single large datum */
return rval;
}
}
w->block_count++;
return 0;
}
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
