vegarsti opened a new pull request, #8851: URL: https://github.com/apache/arrow-rs/pull/8851
# Which issue does this PR close? Closes #8840. # Rationale for this change From the issue: > When merging two schemas using Schema::try_merge where one side doesn't have a column but the other does, then it keep the nullability setting of the preexisting column. However, this semantically doesn't make sense, the merged version of that would be the field with nullable being true, since that is the implicit property of the schema that doesn't have the field. > Consequently what this means is that it's impossible to merge schemas and therefore record batches where one side has a field that is nullable false, and the other doesn't have it at all. # What changes are included in this PR? Adds a `preserve_nullability` argument to `SchemaBuilder::try_merge` which, when set to `true` will preserve the current behavior of using the field's nullability. When set to `false`, the nullability of the field will always be set to `true`. # Are these changes tested? Yes, existing tests (some modified), and new test. # Are there any user-facing changes? Yes. When merging schemas, any non-nullable fields not present in all schemas will now always be nullable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
