[jira] [Commented] (NIFI-2078) State management for processors whose states are managed externally

[ASF GitHub Bot (JIRA)]

    [ 
https://issues.apache.org/jira/browse/NIFI-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15402855#comment-15402855
 ] 

ASF GitHub Bot commented on NIFI-2078:
--------------------------------------

Github user JPercivall commented on the issue:

    https://github.com/apache/nifi/pull/563
  
    @olegz @ijokarumawak I don't think "Queue Buffering Max Time" is working 
properly. I was getting super poor performance out of PutKafka and couldn't 
understand why it would take ~5 seconds for a single message to be sent. I then 
found this property and when I adjusted it lower it fixed my performance 
problems (and higher according made it worse).
    
    Looking at the property descriptor and corresponding property in Kafka 
([seach for "linger.ms"](http://kafka.apache.org/documentation.html)), I would 
expect it to batch multiple flowfiles but instead it just hung and only 
processed one flowfile per batch. Below are a couple screenshots of the 
configuration and the stats of running 3 node cluster (saw the saw phenomenon 
running primary node only and the scheduling period was "0 secs"):
    
    ![screen shot 2016-08-01 at 5 22 23 
pm](https://cloud.githubusercontent.com/assets/11302527/17309375/8d1189e0-580c-11e6-9c58-cf1a8f9bbc99.png)
    
    ![screen shot 2016-08-01 at 5 12 26 
pm](https://cloud.githubusercontent.com/assets/11302527/17309350/792e9512-580c-11e6-8d70-19233f7132e7.png)



> State management for processors whose states are managed externally
> -------------------------------------------------------------------
>
>                 Key: NIFI-2078
>                 URL: https://issues.apache.org/jira/browse/NIFI-2078
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Core Framework
>            Reporter: Koji Kawamura
>            Assignee: Koji Kawamura
>             Fix For: 1.0.0
>
>
> Inherently by the nature of a given processor it may involve state managed by 
> itself (using nifi state management), or can be managed by some external 
> service it interacts with (kafka's offset), and theoretically some might have 
> both going on. With the new state management, we're giving users a way to 
> reset state managed by nifi for a given processor. But it doesnt apply to 
> those processors who have external state.
> we should consider offering a way to reset state that allows a processor to 
> call out to whatever external store it impacts



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)