ASF GitHub Bot commented on NIFI-4994:

Github user ijokarumawak closed the pull request at:


> Reload instance class loader at every component restart
> -------------------------------------------------------
>                 Key: NIFI-4994
>                 URL: https://issues.apache.org/jira/browse/NIFI-4994
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Core Framework
>            Reporter: Koji Kawamura
>            Assignee: Koji Kawamura
>            Priority: Major
> h2. Current Class loader reload mechanism
> NiFi framework provides RequiresInstanceClassLoading to annotate components 
> (Processor, Controller Service or Reporting Task) to use per instance class 
> loader. Instance class loaders are reloaded when associated components are 
> restarted, if any component property flagged as 
> 'dynamicallyModifiesClasspath' is updated. Such component property is used to 
> let user specify additional class pass resource URLs. E.g. PutHDFS 
> 'Additional Classpath Resources'.
> h2. Limitation of current mechanism
> Current mechanism requires component properties to trigger reloading to 
> represent class path resource URLs. That works well in scenarios for users to 
> add/del/mod additional class path resources.
> However there is different needs for reloading instance class loader. That is 
> re-initializing static state in dependencies.
> For example, ReportLineageToAtlas reporting task uses Atlas client library, 
> and the library has static code blocks to initialize its dependencies, such 
> as Kafka client with parameters passed by NiFi reporting task based on user 
> inputs. Once the initialization is done, there is no way to re-initialize 
> those objects, meaning even if NiFi user changes reporting task 
> configuration, those will not be able to reflected. 
> https://github.com/apache/atlas/blob/master/notification/src/main/java/org/apache/atlas/hook/AtlasHook.java#L63
> The only possible operations to reset such static state are, creating another 
> instance, configure the instance again, remove the old one and start new one. 
> Or restart NiFi process. This limitation affects UX negatively.
> h2. Improvement design
> Possible approaches are:
> # Enhance RequiresInstanceClassLoading annotation so that custom components 
> can specify whether it needs to be reloaded on every component restart 
> without condition.
> # Add new PropertyDescriptor instance field to denote that instance class 
> loader needs to be reloaded if a property is changed. Similar to 
> 'dynamicallyModifiesClasspath' but for configurations other than URI. Then, 
> include those property value to 
> AbstractConfiguredComponent.additionalResourcesFingerprint
> No.1 is more naive approach, but require less framework code change. No.2 
> looks redundant, also could be error-prone, e.g. by forgetting to mark some 
> properties.

This message was sent by Atlassian JIRA

Reply via email to