rebasedming opened a new issue, #8763:
URL: https://github.com/apache/arrow-datafusion/issues/8763
### Describe the bug
When a table is created with a non-nullable `Field`, executing a
`DMLStatement` logical plan to write to that table always fails with the error
`Column [name] is declared as non-nullable but contains null values`. This
happens no matter what values I try to insert.
### To Reproduce
The following code will reproduce the error. It creates a table called `t1`
with a non-nullable int32 `Field` called `a`. When we try to add an integer
into this table, it errors.
```rust
use async_std::task;
use datafusion::arrow::datatypes::{DataType, Field, Schema};
use datafusion::arrow::record_batch::RecordBatch;
use datafusion::common::{DFSchema, ScalarValue};
use datafusion::datasource::MemTable;
use datafusion::logical_expr::{DmlStatement, Expr, LogicalPlan, Values};
use datafusion::logical_expr::WriteOp::InsertInto;
use datafusion::prelude::SessionContext;
use std::sync::Arc;
fn main() {
let table_name = "t1";
let field = Field::new("a", DataType::Int32, false);
let ctx = SessionContext::new();
let schema = Schema::new(vec![field]);
let df_schema = DFSchema::try_from(schema.clone()).unwrap();
match MemTable::try_new(schema.clone().into(),
vec![Vec::<RecordBatch>::new()]).ok() {
Some(mem_table) => {
let _ = ctx.register_table(table_name.clone(),
Arc::new(mem_table));
}
None => panic!("An unexpected error occured creating the table"),
};
let values = vec![vec![Expr::Literal(ScalarValue::Int32(Some(1)))]];
let values_plan = LogicalPlan::Values(Values {
schema: df_schema.clone().into(),
values: values.clone(),
});
let dml_plan = LogicalPlan::Dml(DmlStatement {
table_name: table_name.into(),
table_schema: df_schema.clone().into(),
op: InsertInto,
input: values_plan.into(),
});
// This step always errors out
// Error:
// ArrowError(InvalidArgumentError("Column 'a' is declared as
non-nullable but
// contains null values"))'
let dataframe =
task::block_on(ctx.execute_logical_plan(dml_plan.clone()))
.expect("Could not execute plan");
let _ = task::block_on(dataframe.collect()).expect("Could not collect
dataframe");
println!("Success!");
}
```
### Expected behavior
We would expect this insert to succeed, as the values we are inserting are
not null.
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
