paddymul opened a new issue, #40561:
URL: https://github.com/apache/arrow/issues/40561
### Describe the usage question you have. Please include as many useful
details as possible.
I do not know how to use ipywidgets traitlets.Bytes or even base64 encoding
to generate an arrow table that will reify in JS.
lets start with Base64
```python
import pyarrow as pa
import base64
df = pd.DataFrame({'a':[10,50], 'b': ['paddy', 'margaret']})
table = pa.Table.from_pandas(df)
batch = table.to_batches()[0]
pabuffer = batch.serialize()
pabuffer
b64_table = base64.b64encode(pabuffer.to_pybytes())
print(base_64_table)
```
given the following JS code, that string doesn't reify properly
```typescript
export function base64ToBytes(base64:string) {
const binString = atob(base64);
//@ts-ignore
return Uint8Array.from(binString, (m) => m.codePointAt(0));
}
const b64bytes = base64ToBytes(base64table)
const t2 = tableFromIPC(b64bytes);
```
I get an error of `TypeError: this.schema is undefined`
I can get an arrow table to reify when I do base64 in JS and use that string.
I have made a sample repo to play with, benchmark, and document dataframe
serialization. https://github.com/paddymul/df_cereal
I'm eager to collaborate and help with documentation for using arrow-js. I
think it solves a lot of serialization problems for performance and typing vs
json. It's a little hard to approach as is, because the docs are written for
someone who understands arrow, vs someone who knows less about arrow, but wants
to use it for serialization.
### Component(s)
JavaScript, Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
