swapna267 opened a new pull request, #246: URL: https://github.com/apache/flink-connector-kafka/pull/246
Currently OpenLineage event emitter relies on availability of custom Kafka facet in the classpath. https://github.com/OpenLineage/OpenLineage/blob/main/integration/flink/flink2/src/main/java/io/openlineage/flink/util/KafkaDatasetFacetUtil.java#L25 This fails when Kafka connector is being loaded as part of user jar and openlineage library loaded as part of Flink distribution. Openlineage emitters gets loaded in ApplicationClassLoader , whereas the Kafka connector libs gets loaded into FlinkUserClassLoader. In order to fix this, we make Kafka connector emit facet that is an instance of `org.apache.flink.streaming.api.lineage.DatasetConfigFacet`. Openlineage libs can then rely on the existence of facet by name and pull configs by casting to DatasetConfigFacet. This will avoid depending on Kafka connector classes to be in same class loader as lineage emitter libs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
