Hi Adam, PostHTTP was marked deprecated 3 years ago (aka six technology lifetimes). The successive technologies to replace it's functionality are well documented and proven in production. The technical reason to remove it is that it is superfluous code that has a cost to maintain and zero benefit. Backwards compatibility is never guaranteed for components marked deprecated for such a long length of time in any software product let alone nifi specifically.
Your organisation is free to continue using the version of nifi it is on today and not take any further action. It is unhelpful to suggest every other organisation should be held back in progress because yours refuses to take the necessary flow maintenance action. One of the impetus for a major version upgrade is to specifically jettison deprecated components. It's quite remarkable you're advocating against standard practice presumably for your own convenience. Site to site connectivity is conducted with either raw sockets or http (which is https on secured nifi) so I'm highly skeptical there is any performance deprecation in InvokeHTTP or S2S over PostHTTP, given the former can take advantage of http/2 and the latter not. It's easy to monitor nifi and prove through metrics in any case. Sadly in enterprise environments this is sometimes necessary to defeat the political layer around change management. You can run records-based processing over either current method and it is ridiculously fast. The bottleneck in my last engagement ended up being network hardware limitations, not nifi. Having contributed in this domain, I also recommend tossing CompressContent into the flow to minimise bandwidth. On modern hardware the decompression is minimal time and you can plug a *lot* more data through in less CPU and wall clock time. It's easy to bench with DuplicateFlowfile on your production flow and metrics analysis, just make sure your provenance db has sufficient space. Kind regards, On Thu, Jan 12, 2023, 10:09 Adam Taft <a...@adamtaft.com> wrote: > Just wanted to note a concern on the deprecation (and presumed removal) of > the PostHTTP processor in the upcoming 2.0 release. > > While yes, for traditional client interactions with an external HTTP > service, utilizing InvokeHTTP for your POST operation is probably sensible. > The concern is that there are a number of NiFi-to-NiFi transfers that > leverage the "special sauce" that exists between PostHTTP and ListenHTTP. > > What special sauce? Namely, the extra negotiation that enables an automated > serialization of NiFi flowfiles from one system to another. InvokeHTTP is > just a "raw" HTTP client and doesn't share any special concern or support > for NiFi-to-NiFi data transfer. > > Of course, if you remember the history, before there was any site-to-site > functionality built into processor groups, the primary means of flowfile > transport between NiFi systems was the PostHTTP / ListenHTTP combo. It was > an easy way to facilitate transfer between two nifi systems. > > And from what I can tell, this "legacy" approach to NiFi data transfer is > still being used heavily in certain operational contexts. Why? Because > often it's the case that the _only_ traffic allowed between network > boundaries is done via HTTPS. The site-to-site protocol provides its own > ports and protocol operations that don't necessarily comply with such a > network policy. And I believe there's still some lingering and/or > demonstrated concern over the performance characteristics of the > site-to-site protocol by dataflow managers. They have often reverted to > using PostHTTP / ListenHTTP instead. > > While many of the other deprecated components seem logical, getting rid of > this one just seems like change-for-the-sake-of-change. > > Is there any actual technical reason to deprecate and remove PostHTTP from > the standard nar? Is it causing a burden to the product itself? Or was the > decision just more like, "hey it's dumb not to use InvokeHTTP for all HTTP > client operations" and maybe not realize the alternative use case that > PostHTTP enables? > > Thanks for any feedback. > > /Adam >
