[
https://issues.apache.org/jira/browse/NIFI-14386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael W Moser reassigned NIFI-14386:
--------------------------------------
Assignee: Michael W Moser
> Remove ListenHTTP Max Data to Receive per Second capability
> -----------------------------------------------------------
>
> Key: NIFI-14386
> URL: https://issues.apache.org/jira/browse/NIFI-14386
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Michael W Moser
> Assignee: Michael W Moser
> Priority: Minor
>
> ListenHTTP can limit the rate of data that it accepts, which limits the rate
> that data is read from a socket. It will eventually read all data from the
> socket and commit the file to NiFi repositories. However, perhaps due to
> data buffering, the sending side can think it has sent the entire file but
> get a SocketTimeoutException because ListenHTTP hasn't finished reading yet.
> Typically, after encountering an exception, the sending side can try to send
> the file again. This causes duplicate files on the receiving end.
> Additionally, large files may never make it through the connection. Because
> of this we should remove the Max Data to Receive per Second capability.
> There is a viable replacement which should be documented. Follow ListenHTTP
> with a ControlRate processor where you set the desired data rate limit.
> Configure back pressure on the "success" connection between ListenHTTP and
> ControlRate. Back pressure should be limited to an acceptable data buffer
> size that is not too large to make the rate limit unnoticeable.
> When back pressure is in effect, ListenHTTP will send 503 Service Unavailable
> responses as soon as an HTTP connection is made. This indicates to the
> sender that the ListenHTTP is overloaded and to try again later.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
