adhardy commented on issue #41667: URL: https://github.com/apache/arrow/issues/41667#issuecomment-2515049214
From a user perspective, my expectation is that if I set nullable=False in my schema I am expecting some sort of failure, I guess in a similar way to if I was to pass a mis-matched type. At the moment if I pass in a None/null to a non-nullable field, my data becomes corrupted and I get no warning about it. I would rather the None gets written, at least my data is valid. This is maybe less of a problem I guess in statically typed languages as something would have probably already failed before I got to that point but this is really easy to do in Python. At present there is really no use in setting nullable=False from the Python API - it does not validate that there are no nulls, and it will just corrupt my data if I do, I may as well always leave nullable=True. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
