mosche commented on pull request #16947: URL: https://github.com/apache/beam/pull/16947#issuecomment-1056795786
@reuvenlax I had a closer look why JavaFieldSchema does not work. There's quite some issues, and this is by far not an exhaustive list: - Only [public, non-static, non-transient fields](https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/utils/ReflectUtils.java#L129-L151) are considered. Generated models in the AWS SDK are private, however. Using some kind of Field filter in JavaFieldSchema, that can then be overwritten, this could be addressed. But this obviously also means to generate different FieldValueGetter using ByteBuddy. - Next, more involved I suppose, such AWS SDK models contain final fields only, but there's no known creator. The annotation based approach isn't feasible for such "external" classes. - An annoyance, but has to be handled as well: These model classes may contain maps with non primitive types. This is not supported. Also, in some cases there might be recursive type hierarchies that can't be supported and have to be dealt with. Also, I've seen all the optimisations done in JavaFieldSchema and JavaBeanSchema. Honestly, at this point I don't bother much about that yet. I'm more interested to understand what the right approach forward is giving all the AWS IOs currently use hand crafted low level coders. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
