phpsxg commented on issue #35441: URL: https://github.com/apache/arrow/issues/35441#issuecomment-1535896743
> I'm not expert on pyarrow, but seems that pyarrow has its internal schema. So, when casting to pyarrow, if not explicit set a schema, `[None] * 3` might be reduce as "null" type. And write null type to parquet might failed. > > With explicit set a schema, it would be set to "nullable string", which is a valid schema here. I found pa. Table.from_pandas and ds.write_dataset do not add the schema parameter and will not report an error. ``` table = pa.Table.from_pandas(df) ds.write_dataset(table, file_path, format='parquet', existing_data_behavior='delete_matching') ``` Is there no need to add this schema parameter? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
