tustvold commented on issue #5716: URL: https://github.com/apache/arrow-rs/issues/5716#issuecomment-2094220904
> However, in practice, when write a parquet file, the default attribute of a column is OPTIONAL Is this the case if you set `nullable` to `false`? If so I would probably raise a bug on pyarrow as that is incorrect. > This change is only to relax parquet_derive's restriction against parquet input, without introducing risk since checks are done after parsing. If user's input does have NULL values, the parser will panic, like what it is doing now. So long as we don't regress performance for existing workloads I suppose this is an acceptable workaround. I will try to take a look next week at your PR, although I will need to allocate enough time to get up to speed on that crate (there isn't really anyone maintaining it actively anymore). FWIW reading parquet via the arrow interface will be faster, especially for string columns, but appreciate if you'd rather stick to a row-oriented model -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
