Hi Brandon, I have added what i called "recursive" support to find child fields on any level to the Cast and ReplaceField transforms as well, the PR is here https://github.com/apache/kafka/pull/9493
I plan to try and write up a KIP for that one as well maybe in the next day or two. It is not exactly the same as specifically targeting one nested field based on its full path, but instead will look for the field name at any level of the structure and cast/replace all of them if they match. The same could be added in theory to almost all of the transforms i guess but maybe that is for another discussion 😊 But for now with this TimestampConverter the only thing which I have proposed are the two mentioned before - multiple fields and support for pattern like matching of string timestamps. Joshua Den lör 7 nov. 2020 18:42Brandon Brown <bran...@bbrownsound.com> skrev: > I love this idea! I have a current KIP for a hash transform and am working > adding multi field/nested support to that one. I can think of a few times > we’ve needed functionality like that. I’d add that adding support for > transforming nested fields would be a great feature for this. > > -Brandon > > Brandon Brown > > On Nov 7, 2020, at 11:11 AM, Joshua Grisham <grishamj...@gmail.com> > wrote: > > > > Hello everyone! > > (Thanks to Tom Bentley for directing me in this direction!) > > > > I have made a series of changes to some of the standard Connect > transforms > > to meet some of the challenges at my company to consume messages using > > Connect, and have been running them for a few weeks now as custom SMTs. > > > > I realized that several of these changes might actually be really good > > features to be included in the standard transforms so I will open some > KIPs > > to go along with PRs which I already submitted (apologize for going a bit > > backwards in the process as this was my first time working with the Kafka > > project). > > > > The first one I have created a KIP for is KIP-682 on the > TimestampConverter > > transform. > > > > Basically the 2 limitations that I am proposing to remove are the ability > > to only convert one field at a time, but instead convert multiple fields > > using a common config, and that if any producer sends a slightly > different > > date/timestamp string format on any message then the transformation > > fails... so to instead add the ability to support variations in the > string > > input format. > > > > More info here: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-682%3A+Connect+TimestampConverter+support+for+multiple+fields+and+multiple+input+formats > > > > I would appreciate any kind of discussion to make improvements and it > would > > be great to see changes like this being pushed up. > > > > Thanks for now! > > > > Best regards, > > Joshua Grisham >
