Adam, This question has come up in some Jira issues over the years, but the majority of community interest and effort continues to focus on InvokeHTTP.
There are several good reasons for removing PostHTTP, and recent versions of InvokeHTTP have included several important improvements to the InvokeHTTP-ListenHTTP flow. In addition to being deprecated for years, there are several technical reasons for removing in from the project in its current condition. 1. PostHTTP uses Apache HttpComponents HttpClient 4.5, which has been in maintenance mode for the last several years with the release of version 5 2. PostHTTP no longer has active test cases in Apache NiFi due its deprecation 3. PostHTTP includes an internal retry strategy that locks processing tasks with the use of Thread.sleep() As you noted, the Send as FlowFile feature of PostHTTP implements some specialized behavior for communicating with ListenHTTP. This is a convenient feature for sending both FlowFile content and attributes to another NiFi server running ListenHTTP, but it depends on specialized packaging. This same behavior can be achieved using the MergeContent Processor, and then sending the content through InvokeHTTP. Similar results can be achieved through sending FlowFile attributes as HTTP headers using InvokeHTTP. The custom processing in PostHTTP also performs Gzip compression during transmission. NiFi 1.17.0 introduced several improvements to InvokeHTTP and ListenHTTP that provide more generalized performance improvements. InvokeHTTP has a Request Content-Type property that can be set to GZIP, which implements a standards-based approach to compressing content during transmission. ListenHTTP automatically reads the Content-Type header and handles decompression as needed. NiFi 1.17.0 also added supporting for HTTP/2 in ListenHTTP, which uses binary formatting for HTTP headers, and supports multiplexing requests, as opposed to connection pooling strategies necessary for HTTP/1.1. InvokeHTTP and the OkHttp library have supported HTTP/2 in the default configuration for a number of versions. These improvements address several of the theoretical gaps between PostHTTP and InvokeHTTP sending to ListenHTTP. Nothing in the scope of NiFi 2.0 includes directly breaking support for older processors, so running older components is a last resort option. Regards, David Handermann On Wed, Jan 11, 2023 at 5:09 PM Adam Taft <a...@adamtaft.com> wrote: > Just wanted to note a concern on the deprecation (and presumed removal) of > the PostHTTP processor in the upcoming 2.0 release. > > While yes, for traditional client interactions with an external HTTP > service, utilizing InvokeHTTP for your POST operation is probably sensible. > The concern is that there are a number of NiFi-to-NiFi transfers that > leverage the "special sauce" that exists between PostHTTP and ListenHTTP. > > What special sauce? Namely, the extra negotiation that enables an automated > serialization of NiFi flowfiles from one system to another. InvokeHTTP is > just a "raw" HTTP client and doesn't share any special concern or support > for NiFi-to-NiFi data transfer. > > Of course, if you remember the history, before there was any site-to-site > functionality built into processor groups, the primary means of flowfile > transport between NiFi systems was the PostHTTP / ListenHTTP combo. It was > an easy way to facilitate transfer between two nifi systems. > > And from what I can tell, this "legacy" approach to NiFi data transfer is > still being used heavily in certain operational contexts. Why? Because > often it's the case that the _only_ traffic allowed between network > boundaries is done via HTTPS. The site-to-site protocol provides its own > ports and protocol operations that don't necessarily comply with such a > network policy. And I believe there's still some lingering and/or > demonstrated concern over the performance characteristics of the > site-to-site protocol by dataflow managers. They have often reverted to > using PostHTTP / ListenHTTP instead. > > While many of the other deprecated components seem logical, getting rid of > this one just seems like change-for-the-sake-of-change. > > Is there any actual technical reason to deprecate and remove PostHTTP from > the standard nar? Is it causing a burden to the product itself? Or was the > decision just more like, "hey it's dumb not to use InvokeHTTP for all HTTP > client operations" and maybe not realize the alternative use case that > PostHTTP enables? > > Thanks for any feedback. > > /Adam >
