[
https://issues.apache.org/jira/browse/NIFI-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17558228#comment-17558228
]
David Handermann commented on NIFI-8968:
----------------------------------------
Thanks for documenting these observations and providing examples flows
[~markbean].
As described in NIFI-10163, it appears that the framework was not properly
tracking bytes read in StandardProcessSession.exportTo(), which explains why
those statistics do not appear in the Status History.
After making those adjustments, throughput still remains higher in {{PostHTTP}}
in the default configuration. Disabling the {{Send as FlowFile}} property in
{{PostHTTP}} showed a performance similar to {{InvokeHTTP}}. The FlowFile
Packaging enabled in {{PostHTTP}} causes the processor to batch multiple
FlowFiles and stream them over a single HTTP request, instead of sending each
FlowFile in a separate HTTP request. This behavior is specific to {{PostHTTP}}
when paired with {{ListenHTTP}} since {{ListenHTTP}} is able to read multiple
FlowFiles from the stream. {{InvokeHTTP}} is already a complex Processor, so
attempting to implement a similar batch stream approach over a single HTTP
request could be challenging.
> Improve throughput performance for InvokeHTTP
> ---------------------------------------------
>
> Key: NIFI-8968
> URL: https://issues.apache.org/jira/browse/NIFI-8968
> Project: Apache NiFi
> Issue Type: Improvement
> Affects Versions: 1.14.0
> Reporter: Mark Bean
> Priority: Major
> Attachments: PostHTTP_vs_InvokeHTTP.json, PostHTTP_vs_InvokeHTTP.xml
>
>
> InvokeHTTP is the preferred processor to use over the deprecated PostHTTP.
> However, PostHTTP outperforms InvokeHTTP (at least in POST mode). A template
> and a JSON file have been attached to this ticket for benchmarking the two
> processors. Using this flow, PostHTTP was observed to have a throughput of
> approximately 5 times greater than InvokeHTTP.
> In addition, it was noted that InvokeHTTP had approximately 5 times as many
> tasks and 5 times the task duration for a given 5 minute stats window. And,
> the statistics of Bytes Read and Bytes Transferred remain at zero for
> InvokeHTTP; this area accurate statistics also needs to be addressed.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
