0xderek commented on issue #42406:
URL: https://github.com/apache/doris/issues/42406#issuecomment-2467112888
Please refer to the following steps to reproduce.
1. create test table
```sql
CREATE TABLE `demo` (
`id` int NULL,
`value` variant NULL
)
DUPLICATE KEY(`id`)
DISTRIBUTED BY RANDOM BUCKETS 16
```
2. generate data in arrow stream format
```python
import numpy as np
import pyarrow as pa
ids = pa.array(np.arange(100))
values = pa.array([f'{{\"a\": {i}}}' if i % 2 == 0 else f'{{\"a\":
\"{i}\"}}' for i in range(100)])
schema = pa.schema([
pa.field('id', pa.int32()),
pa.field('value', values.type)
])
with pa.OSFile('payload.dat', 'wb') as sink:
with pa.ipc.new_stream(sink, schema=schema) as writer:
batch = pa.record_batch([ids, values], schema=schema)
writer.write(batch)
print('Done')
```
Here I'm populating json strings to `value` field which is `VARIANT` in
Doris table. The type for this field in arrow structure does not matter in this
case.
3. initiate stream load
```bash
curl --location-trusted -u user:pass \
-H "Expect:100-continue" \
-H "format:arrow" \
-T payload.dat \
-XPUT http://host:port/api/db_name/demo/_stream_load
```
The stream load operation will fail with error message "Failed to convert
from arrow to block: [E3] read_column_from_arrow with type variant".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
