Hi all, I'm working on an integration project trying to write some library code that will allow us at the Wikimedia Foundation to use Flink with our 'Event Platform <https://wikitech.wikimedia.org/wiki/Event_Platform>'. Specifically, I'm trying to write a reusable step near the end of a pipeline that will ensure our JSON events satisfy some criteria before producing them to Kafka. Details here <https://phabricator.wikimedia.org/T310218>.
I'm experimenting with writing my own custom format <https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sourcessinks/#encoding--decoding-formats> to do this. But all I really need to do is override JsonRowDataSerializationSchema's serialize method <https://github.com/apache/flink/blob/master/flink-formats/flink-json/src/main/java/org/apache/flink/formats/json/JsonRowDataSerializationSchema.java#L90-L101> and augment and validate the ObjectNode before it is serialized to byte[]. I'm running into an issue where the ObjectNode that is used by Flink here is the shaded one: org.apache.flink.shaded.jackson2.com.fasterxml.jackson. databind.node.ObjectNode, whereas the WMF code <https://gerrit.wikimedia.org/r/plugins/gitiles/wikimedia-event-utilities/+/refs/heads/master/eventutilities/src/main/java/org/wikimedia/eventutilities/core/event/JsonEventGenerator.java#85> I want to use to augment the ObjectNode is using a regular non shaded one. I can't pass the shaded ObjectNode instance to a function that takes a non shaded one, and I can't cast the shaded ObjectNode to non shaded either. My Q is: is there a way to extend Flink APIs that use shaded dependencies? I suppose I could copy/paste the whole of the "json" format code that I need into my project and just make it my own, but this feels quite obnoxious. Thank you! -Andrew Otto Wikimedia Foundation
