Michael Andre Pearce created NIFI-3967:
------------------------------------------
Summary: Support JDBC Schema Evolution with Avro input data.
Key: NIFI-3967
URL: https://issues.apache.org/jira/browse/NIFI-3967
Project: Apache NiFi
Issue Type: New Feature
Reporter: Michael Andre Pearce
Currently NiFi does not have built-in support for several of those tasks:
* Identifying a schema evolution (separate from a non-conforming flowfile)
* Identifying safe schema changes
* Automatically applying schema changes to a SQL database
However, the new schema registry and record handling components introduced in
NiFi 1.2.0 might be useful to build some of these features.
The use case for this is:
In our system we have kafka with avro data, we want to persist this to our
traditional SQL data warehouse staging tables, as just an Extract and Load
process similar to Hadoop.
To do this though we need the staging to evolve automatically safely (aka don’t
remove columns), based on the inbound avro data evolving.
This use case is similar in nature to confluent’s kafka connect jdbc connector,
solves.
It would be great if Apache NiFi could solve this use case also, so that
separate integration tools are not needed.
Thanks in advance,
Mike
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
