singhpratech commented on issue #88:
URL: https://github.com/apache/arrow-js/issues/88#issuecomment-5619886467
I can put time into this. The scope I have in mind, so it can be corrected
here rather than in
review:
**Import**: `parseSchema(buffer, ptr)` and `parseArray(buffer, ptr, type, {
copy })` returning
`Field` and `Data` from an `ArrayBuffer` plus a pointer, which is the shape
`arrow-js-ffi` already
has. @kylebarron, if you would rather bring that code in yourself, or keep
it as the separate
package, say so; I would not copy it without you.
**Export**, which no package provides today: `writeSchema(type, buffer,
alloc)` and
`writeArray(data, buffer, alloc)` filling `ArrowSchema` and `ArrowArray`
structs in a
caller-supplied `ArrayBuffer` through a caller-supplied allocator, so the
same code serves
WebAssembly memory (the allocator is the module's `malloc`) and a Node
addon's memory. A buffer
that already lives in the target `ArrayBuffer` is pointed at, not copied;
any other is copied
through the allocator. `Data.offset` becomes the struct's `offset`, with the
values and offsets
buffers rewound to row 0, the mapping from my comment above.
The one design point I do not want to decide alone is the `release`
callback. JS cannot place a
function pointer in WebAssembly memory by itself, so the caller would pass
one (a table entry the
module exports, or one the addon provides) and `writeArray` would return the
allocations it made
so that callback can free them.
First PR: null, bool, the ints and floats, utf8 and large utf8, binary,
large binary, fixed-size
binary, date, time, timestamp, duration, decimal, list, fixed-size list,
struct, dictionary.
Union, map, run-end encoded and extension types after. Tests: JS round trips
for every type with
nulls and slices, and a check against pyarrow through the Node addon where
the sliced-validity
mapping was found.
Two questions: core (`apache-arrow`, an `ffi` entry point, no new
dependency) or a separate
package in this repository; and the release-callback shape above. If the
answer is that
`arrow-js-ffi` should stay where it is, that is fine too; I mostly want to
avoid building it twice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
