jiangzhx commented on issue #4105:
URL: https://github.com/apache/arrow-rs/issues/4105#issuecomment-1517307815
> Btw I was able to fix this error by removing all empty arrays from my json
source data.
>
> But it would have been nice if it was possible to pass some option to
ignore un-inferable fields like this so I don't get errors on 'SELECT * ...'
In fact, you do not need to remove empty arrays. This situation only occurs
when a file has just one line, like `{"items": []}`.
If the file contains more than one line, such as:
```
{"items": []}
{"items": [1,2]}
```
then arrow_json::reader::infer_json_schema will use the second line to infer
the data type of the items. Therefore, you should not encounter errors when
using "SELECT *".
SELECT * should return
```
+--------+
| items |
+--------+
| [] |
| [1, 2] |
+--------+
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
