I'm currently using parquet as the intermediate format when uploading a pandas DataFrame to Google BigQuery. We encounter a problem when trying to append a parquet file to a table with required fields (issue: https://github.com/googleapis/google-cloud-python/issues/8093).
Is there a way to mark fields as required / non-nullable in parquet files? If there is, is there a way to set that option with pyarrow.parquet.write_table? https://arrow.apache.org/docs/python/generated/pyarrow.parquet.write_table.html#pyarrow.parquet.write_table * • **Tim Swast* * • *Software Friendliness Engineer * • *Google Cloud Developer Relations * • *Seattle, WA, USA
