Rahul Soni created NIFI-8167:
--------------------------------
Summary: Decouple NiFi & Schema Registry dependencies
Key: NIFI-8167
URL: https://issues.apache.org/jira/browse/NIFI-8167
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Reporter: Rahul Soni
Today, NiFi depends on Schema Registry for Avro schemas for the data it is
processing. By default, the caching duration for the Schema Registry controller
service is 1 hour, which can be of course increased or decreased. But it has
its flaws.
* Make the cache duration too high, and you would need to wait for a long time
before an updated schema is reflected in the NiFi flow.
* Make the cache duration too low, and the NiFi flow may spend higher time in
fetching the schemas from SR and also may fail more often with *_schema not
available_* errors, in case the Schema Registry instance is not available for a
short period.
I would suggest having Schema Registry behave like Ranger do. All the schemas
from the Schema Registry can be cached at the NiFi nodes like Ranger's _policy
cache_ and any change to the schema can be propagated to the NiFi nodes using
that Schema Registry instance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
