jonathansudhakar1 commented on issue #41017: URL: https://github.com/apache/beam/issues/41017#issuecomment-6830337044
@ahmedabu98, a polite follow-up, since you worked on the Avro/Beam logical-type conversions in `AvroUtils` (#36605). Could you take a look at the two-step plan in the description when you have a moment? 1. Let `AvroUtils`' strict conversion turn an Avro `date` value into a `LocalDate` when the target field is `SqlTypes.DATE`. Today `convertDateStrict` only accepts `DATETIME`. This is additive: schemas inferred from Avro still get `DATETIME`. 2. Have `ParquetTable` build rows with the table's own (projected) Beam schema instead of the one ParquetIO infers from Avro. None of the files involved have changed on master since I filed this. Related: #21003 asks for the inference side (Avro `date` inferred as Beam `DATE`). I've left that alone, since it would change inferred schemas for every Avro-based IO. If you'd rather fix this somewhere else, for example in ParquetIO, I'm happy to do it that way. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
