buffer alignment (format/java/js)

[Emilio Lahr-Vivaz]

I'm looking into buffer alignment in the java writer classes. Currently 
some files written with the java streaming writer can't be read due to 
the javascript TypedArray's restriction that the start offset of the 
array must be a multiple of the data size of the array type (i.e. 
Int32Vectors must start on a multiple of 4, Float64Vectors must start on 
a multiple of 8, etc). From a cursory look at the java writer, I believe 
that the schema that is written first is not aligned at all, and then 
each record batch pads out its size to a multiple of 8. So:

1. should the schema block pad itself so that the first record batch is 
aligned, and is there any problem with doing so?
2. is there any problem with changing the alignment to 64 bytes, as 
recommended (but not required) by the spec?


Advertising







Thanks,

Emilio