jbonofre opened a new issue, #1454: URL: https://github.com/apache/arrow-java/issues/1454
### Describe the bug, including details regarding any error messages, version, and platform. The nightly RC workflow builds the JNI libraries from Arrow C++ `main`. Since Oct 15, every nightly run that reached the arrow-dataset tests has failed the Binaries job on: ``` TestAceroSubstraitConsumer.testRunQueryLocalFiles:94 expected: <Schema<ID: Int(32, true), NAME: Utf8>> but was: <Schema<ID: Int(32, true) not null, NAME: Utf8>> ``` Latest: https://github.com/apache/arrow-java/actions/runs/42539841523/job/128913506709. Push and pull request runs still pass because they build against the latest Arrow C++ release (26.0.1). The Substrait plan in `dataset/src/test/resources/substrait/local_files_users.json` declares `ID` as `NULLABILITY_REQUIRED` in the read relation's base schema. The Parquet file the test reads has an optional `id` column (`user.avsc` declares it as `["int", "null"]`), and the test expects a nullable `ID`. apache/arrow#51691 (GH-51231), merged into Arrow C++ `main` on Oct 9, makes Acero's project node keep the input nullability of direct field references. The plan's project relation now returns `ID` as `not null`, as the plan declares. 26.0.1 doesn't include that change, so once Arrow C++ 27.0.0 is released, push and pull request runs will fail the same way. The plan should declare `ID` as nullable, which matches the file. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
