Sudarshan Vasudevan created GOBBLIN-1345:
--------------------------------------------
Summary: Implement a Kafka schema change detecting converter
Key: GOBBLIN-1345
URL: https://issues.apache.org/jira/browse/GOBBLIN-1345
Project: Apache Gobblin
Issue Type: Bug
Components: gobblin-kafka
Affects Versions: 0.16.0
Reporter: Sudarshan Vasudevan
Assignee: Shirshanka Das
Fix For: 0.16.0
For long-running Gobblin pipelines such as Gobblin-Kafka with streaming mode
enabled, we need the ability for the pipeline to detect upstream schema changes
to enable proper handling by downstream constructs. For instance, in the case
of sinks such as HDFS and blob stores, such schema changes should result in the
writer closing any open files.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
