After looking at it further, I think only the buffers themselves need to
be aligned, not the metadata and/or schema. Would there be any problem
with changing the alignment to 64 bytes then?
Thanks,
Emilio
On 08/08/2017 08:08 AM, Emilio Lahr-Vivaz wrote:
I'm looking into buffer alignment in the java writer classes.
Currently some files written with the java streaming writer can't be
read due to the javascript TypedArray's restriction that the start
offset of the array must be a multiple of the data size of the array
type (i.e. Int32Vectors must start on a multiple of 4, Float64Vectors
must start on a multiple of 8, etc). From a cursory look at the java
writer, I believe that the schema that is written first is not aligned
at all, and then each record batch pads out its size to a multiple of
8. So:
1. should the schema block pad itself so that the first record batch
is aligned, and is there any problem with doing so?
2. is there any problem with changing the alignment to 64 bytes, as
recommended (but not required) by the spec?
Thanks,
Emilio
