prashantwason opened a new issue, #20355:
URL: https://github.com/apache/hudi/issues/20355
### Bug Description
**What happened:**
BigQuery sync fails for any table whose schema contains a field declared
with the Avro primitive type `"null"` (as opposed to a nullable union such as
`["null","string"]`), e.g.:
```json
{"name": "clientPromotion", "type": "null", "default": null}
```
`BigQuerySchemaResolver.getFields` walks the table's `HoodieSchema` and
`getField` has no case for `NULL`, so the field hits the `default` branch and
sync throws:
```
java.lang.RuntimeException: Unexpected field type: NULL
at
org.apache.hudi.gcp.bigquery.BigQuerySchemaResolver.getField(BigQuerySchemaResolver.java:169)
```
This is the BigQuery counterpart of the Hive sync failure fixed in #20354.
The parquet writer does not write `"null"` type fields ("Avro nulls are not
encoded, unless they are null unions"), so these fields have no column in the
data files.
**What you expected:**
BigQuery sync succeeds and skips `"null"` type fields (top-level and
nested), the same way Hive sync does after #20354, so the synced schema matches
the columns in the data files.
**Steps to reproduce:**
1. Write a Hudi table whose schema has a top-level or nested field of type
`"null"`.
2. Run BigQuery sync on it (`BigQuerySyncTool`).
3. Sync fails with `Unexpected field type: NULL`.
### Environment
**Hudi version:** current master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
