Paul Taylor commented on ARROW-6370:


bq. From my understanding of Arrow it should be a platform-independent format, 
meaning that if I am sending an arrow table from Python to JS it should turn 
out the same, right?

Yes, and that's what's happening here. But you're sending 8-byte integers to a 
platform which has historically only supported 4-byte integers, which is why 
you see each 8-byte integer as a pair of 4-byte integers.

I recommend reading [this post|https://v8.dev/features/bigint] on BigInts in 
the v8 blog.

BigInts (and their related typed arrays) are relatively new additions to JS, 
and aren't supported in all engines yet.

We have done our best to support geting and setting BigInt values when running 
in VM that supports them, but for now we still have to support platforms 
without BigInt. That's why the values Array for Int64Vector is a stride-2 
Int32Array instead of a BigInt64Array.

> [JS] Table.from adds 0 on int columns
> -------------------------------------
>                 Key: ARROW-6370
>                 URL: https://issues.apache.org/jira/browse/ARROW-6370
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: JavaScript
>    Affects Versions: 0.14.1
>            Reporter: Sascha Hofmann
>            Priority: Major
> I am generating an arrow table in pyarrow and send it via gRPC like this:
> {code:java}
> sink = pa.BufferOutputStream()        
> writer = pa.RecordBatchStreamWriter(sink, batch.schema)        
> writer.write_batch(batch)        
> writer.close()        
> yield ds.Response(
>     status=200,
>     loading=False,
>     response=[sink.getvalue().to_pybytes()]   
> )
> {code}
> On the javascript end, I parse it like that:
> {code:java}
>  Table.from(response.getResponseList()[0])
> {code}
> That works but when I look at the actual table, int columns have a 0 for 
> every other row. String columns seem to be parsed just fine. 
> The Python byte array created from to_pybytes() has the same length as 
> received in javascript. I am also able to recreate the original table for the 
> byte array in Python. 

This message was sent by Atlassian Jira

Reply via email to