hi Emilio,

>From your description, it isn't clear why 8-byte alignment is causing
a problem (as compare with 64-byte alignment). My understanding is
that JavaScript's TypedArray classes range in size from 1 to 8 bytes.

The starting offset for all buffers should be 8-byte aligned, if not
that is a bug. Could you clarify?

- Wes

On Tue, Aug 8, 2017 at 8:52 AM, Emilio Lahr-Vivaz <elahrvi...@ccri.com> wrote:
> After looking at it further, I think only the buffers themselves need to be
> aligned, not the metadata and/or schema. Would there be any problem with
> changing the alignment to 64 bytes then?
> Thanks,
> Emilio
> On 08/08/2017 08:08 AM, Emilio Lahr-Vivaz wrote:
>> I'm looking into buffer alignment in the java writer classes. Currently
>> some files written with the java streaming writer can't be read due to the
>> javascript TypedArray's restriction that the start offset of the array must
>> be a multiple of the data size of the array type (i.e. Int32Vectors must
>> start on a multiple of 4, Float64Vectors must start on a multiple of 8,
>> etc). From a cursory look at the java writer, I believe that the schema that
>> is written first is not aligned at all, and then each record batch pads out
>> its size to a multiple of 8. So:
>> 1. should the schema block pad itself so that the first record batch is
>> aligned, and is there any problem with doing so?
>> 2. is there any problem with changing the alignment to 64 bytes, as
>> recommended (but not required) by the spec?
>> Thanks,
>> Emilio

Reply via email to