Koji Kawamura created NIFI-4994:
-----------------------------------
Summary: Reload instance class loader at every component restart
Key: NIFI-4994
URL: https://issues.apache.org/jira/browse/NIFI-4994
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Reporter: Koji Kawamura
Assignee: Koji Kawamura
h2. Current Class loader reload mechanism
NiFi framework provides RequiresInstanceClassLoading to annotate components
(Processor, Controller Service or Reporting Task) to use per instance class
loader. Instance class loaders are reloaded when associated components are
restarted, if any component property flagged as 'dynamicallyModifiesClasspath'
is updated. Such component property is used to let user specify additional
class pass resource URLs. E.g. PutHDFS 'Additional Classpath Resources'.
h2. Limitation of current mechanism
Current mechanism requires component properties to trigger reloading to
represent class path resource URLs. That works well in scenarios for users to
add/del/mod additional class path resources.
However there is different needs for reloading instance class loader. That is
re-initializing static state in dependencies.
For example, ReportLineageToAtlas reporting task uses Atlas client library, and
the library has static code blocks to initialize its dependencies, such as
Kafka client with parameters passed by NiFi reporting task based on user
inputs. Once the initialization is done, there is no way to re-initialize those
objects, meaning even if NiFi user changes reporting task configuration, those
will not be able to reflected.
https://github.com/apache/atlas/blob/master/notification/src/main/java/org/apache/atlas/hook/AtlasHook.java#L63
The only possible operations to reset such static state are, creating another
instance, configure the instance again, remove the old one and start new one.
Or restart NiFi process. This limitation affects UX negatively.
h2. Improvement design
Possible approaches are:
# Enhance RequiresInstanceClassLoading annotation so that custom components can
specify whether it needs to be reloaded on every component restart without
condition.
# Add new PropertyDescriptor instance field to denote that instance class
loader needs to be reloaded if a property is changed. Similar to
'dynamicallyModifiesClasspath' but for configurations other than URI. Then,
include those property value to
AbstractConfiguredComponent.additionalResourcesFingerprint
No.1 is more naive approach, but require less framework code change. No.2 looks
redundant, also could be error-prone, e.g. by forgetting to mark some
properties.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
