smallzhongfeng opened a new issue, #862:
URL: https://github.com/apache/arrow-ballista/issues/862
**Describe the bug**
After the partition table is created, it cannot be read normally
**To Reproduce**
```
echo "1,2" > tmp/year=2022/data.csv
echo "3,4" > tmp/year=2021/data.csv
```
run in ballista-cli
```
❯ CREATE EXTERNAL TABLE t1 (a INT, b INT) STORED AS CSV LOCATION 'tmp';
0 rows in set. Query took 0.002 seconds.
❯ select * from t1;
+---+---+
| a | b |
+---+---+
| 3 | 4 |
| 1 | 2 |
+---+---+
2 rows in set. Query took 0.012 seconds.
❯ CREATE EXTERNAL TABLE t2 (a INT, b INT) STORED AS CSV PARTITIONED BY
(year) LOCATION 'tmp';
0 rows in set. Query took 0.001 seconds.
❯ select * from t2;
+---+---+------+
| a | b | year |
+---+---+------+
| 1 | 2 | 2022 |
| 3 | 4 | 2021 |
+---+---+------+
2 rows in set. Query took 0.011 seconds.
❯ select * from t2 where year = '2022';
[2023-08-08T02:44:21Z ERROR
ballista_core::execution_plans::distributed_query] Job qABP9b9 failed: Job
failed due to stage 1 failed: Task failed due to runtime execution error:
DataFusionError(Execution("ArrowError(InvalidArgumentError(\"Column 'year' is
declared as non-nullable but contains null values\"))"))
DataFusionError(ArrowError(ExternalError(Execution("Job qABP9b9 failed: Job
failed due to stage 1 failed: Task failed due to runtime execution error:
DataFusionError(Execution(\"ArrowError(InvalidArgumentError(\\\"Column 'year'
is declared as non-nullable but contains null values\\\"))\"))\n"))))
```
I deployed it in standalone mode.
**Expected behavior**
A clear and concise description of what you expected to happen.
**Additional context**
Add any other context about the problem here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
