I find myself wishing I had a list of all the jiras / issues that have been put off for a 2.0 release because they required some change or another :(
From: Joe Witt <[email protected]> <[email protected]> Reply: [email protected] <[email protected]> <[email protected]> Date: July 27, 2021 at 12:30:35 To: [email protected] <[email protected]> <[email protected]> Subject: Re: [DISCUSS] NiFi 2.0 Release Goals A few thoughts: 1. I would love to see deprecation notices show up in the UI in various ways to help motivate users to move off things to more supportable things. That is not a prerequisite for anything happening however. Just a good feature/nice thing to do for users when someone is able to tackle it. 2. The decision to deprecate something and to further remove it need not mean there is a superior solution available. If that thing itself isn't getting the love/attention it needs to be maintained/supported/healthy going forward that alone is enough to remove it. That might well be the case with PostHTTP [1] and for comparison you can see how much effort has gone into InvokeHTTP [2]. 3. When discussing a 2.0 release each thing we add as a 'must do' the further away from reality such a release will become. We'll have to get very specific about 'musts' vs 'wants'. [1] https://github.com/apache/nifi/commits/11e9ff377333784974fa55f41483c4281d80da50/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PostHTTP.java [2] https://github.com/apache/nifi/commits/cc554a6b110dfa45767bcb13d834ea4265d6dfe6/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java On Tue, Jul 27, 2021 at 8:47 AM David Handermann <[email protected]> wrote: > > Thanks Mark, providing a template or comparison statistics with Java > versions and component configuration details would be very helpful. If it > is possible to run tests using a public API or deployable service, that > would also help confirm potential differences. > > Showing a deprecation notice in the UI could be helpful, perhaps as a > configurable option. NIFI-8650 describes a general Flow Analysis > capability, and it seems like that might be one possible way to surface > deprecation warnings. For something more specific to component deprecation, > it seems important to find a balance between making it obvious and making > it something that ends up getting ignored. > > Regards, > David Handermann > > On Tue, Jul 27, 2021 at 10:28 AM Mark Bean <[email protected]> wrote: > > > I'll start a new thread for PostHTTP when I get a template and/or detailed > > stats. > > > > I know the deprecation is noted in the documentation. That's a necessary > > and minimum level of notification. I was suggesting it be more obvious in > > the UI. I think it would be beneficial to somehow be aware of the > > deprecation status simply by looking at the flow (perhaps even on the > > summary pages too), and without having to open the documentation for every > > processor to confirm whether or not a component is marked as deprecated. > > > > Thanks, > > Mark > > > > > > On Tue, Jul 27, 2021 at 11:16 AM David Handermann < > > [email protected]> wrote: > > > > > Mark, > > > > > > Thanks for the feedback. It may be better to start a separate thread on > > > PostHTTP, but can you provide an example flow demonstrating the > > performance > > > differences between PostHTTP and InvokeHTTP? > > > > > > PostHTTP uses the Apache HttpComponents library, whereas InvokeHTTP uses > > > OkHttp. NiFi 1.13.2 and 1.14.0 included major version updates for OkHttp, > > > so it would be important to test with the most recent version. It is also > > > worth noting that test classes for PostHTTP are now integration tests as > > > opposed to unit tests, which means they are not executed as part of the > > > automated builds. > > > > > > The NiFi documentation includes the contents of the DeprecationNotice for > > > PostHTTP: > > > > > > > > > > > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.14.0/org.apache.nifi.processors.standard.PostHTTP/index.html > > > > > > Regards, > > > David Handermann > > > > > > On Tue, Jul 27, 2021 at 9:56 AM Mark Bean <[email protected]> wrote: > > > > > > > I'm strongly in favor of reducing tech debt, and moving deliberately > > to a > > > > 2.0 release. I have a concern with just one processor that is currently > > > > marked as deprecated: PostHTTP. (I have not evaluated specifically any > > > > other deprecated components; I cannot say if there are or are not > > similar > > > > issues elsewhere.) I understand the rationale for deprecating this > > > > processor in that it eliminates a processor whose functionality is > > > > available elsewhere, specifically in the more flexible InvokeHTTP. > > > However, > > > > in my experience and testing, PostHTTP performs transfers with far > > > greater > > > > throughput than InvokeHTTP. I would not be in favor of removing > > PostHTTP > > > > unless/until InvokeHTTP is refactored to increase its throughput > > > > capability. > > > > > > > > Has anyone else continued to use PostHTTP over InvokeHTTP for similar > > > > reasons? Or, is there a performance-related configuration for > > InvokeHTTP > > > I > > > > may have missed? > > > > > > > > Also, in order to help facilitate a smooth transition to 2.0 from a > > user > > > > perspective, would it be advisable to add some sort of visual indicator > > > in > > > > the UI for components that are currently annotated as @Deprecated? This > > > > might help users proactively modify their flow prior to a release that > > > > would otherwise break it. > > > > > > > > > > > > > > > > > > > > On Mon, Jul 26, 2021 at 5:02 PM Bryan Bende <[email protected]> wrote: > > > > > > > > > With the merging of MiNiFi and registry into the NiFi repo, we've > > > > > actually gone more towards a mono-repo where everything is released > > > > > together. Eventually it would still be nice to have a smaller base > > > > > distribution containing just the framework and standard NARs, but it > > > > > is hard to tackle that until we provide a way for users to easily > > > > > obtain the NARs in some other way. > > > > > > > > > > On Mon, Jul 26, 2021 at 4:20 PM Edward Armes < [email protected] > > > > > > > > wrote: > > > > > > > > > > > > Given the major version number shift and the spliting up of > > > processors > > > > > into > > > > > > multiple NAR's. I'd like to suggest that we start individually > > > > versioning > > > > > > individual NARs/ bundles. > > > > > > > > > > > > I can see this bringing a large number of benifits including making > > > > Nifi > > > > > > more deployable with things RPM, but also potentially allowing > > those > > > > that > > > > > > have to deploy managed Nifi instances easier to upgrade. > > > > > > > > > > > > Edward > > > > > > > > > > > > On Mon, 26 Jul 2021, 20:42 Otto Fowler, <[email protected]> > > > > wrote: > > > > > > > > > > > > > The issue with updating the aws sdk is if it breaks any one of > > the > > > > > > > processors. > > > > > > > the Web Gateway API invoke processor for example is not using a > > > high > > > > > level > > > > > > > purpose build client and may break. > > > > > > > > > > > > > > If we change the aws version, we need to coordinate in such a way > > > > that > > > > > they > > > > > > > all > > > > > > > can come along reasonably. > > > > > > > IE: what happens if 1 or 2 break but the rest or OK? > > > > > > > > > > > > > > > > > > > > > > > > > > > > From: David Handermann <[email protected]> > > > > > > > <[email protected]> > > > > > > > Reply: [email protected] <[email protected]> < > > > > [email protected]> > > > > > > > Date: July 26, 2021 at 09:33:42 > > > > > > > To: [email protected] <[email protected]> < > > [email protected] > > > > > > > > > > > Subject: Re: [DISCUSS] NiFi 2.0 Release Goals > > > > > > > > > > > > > > Chris, > > > > > > > > > > > > > > Thanks for the reply and recommendations. It seems like some of > > the > > > > > work to > > > > > > > reorganize the module structure could be done outside of a major > > > > > release, > > > > > > > but it would be great to target any breaking changes for 2.0. > > > > Perhaps a > > > > > > > separate feature proposal on module restructuring, with the goal > > of > > > > > > > supporting optimized builds, would be a helpful way to move that > > > part > > > > > of > > > > > > > the discussion forward. > > > > > > > > > > > > > > Regarding updating AWS SDK to version 2, it seems like that might > > > be > > > > > > > possible now. I haven't taken a close look at the referencing > > > > > components, > > > > > > > so I'm not sure about the level of effort involved. Minor NiFi > > > > version > > > > > > > updates have incorporated major new versions of dependencies. For > > > > > example, > > > > > > > NiFi 1.14 included an upgrade from Spring Framework 4 to 5. On > > the > > > > one > > > > > > > hand, including the AWS SDK update as part of a major release > > seems > > > > > > > helpful, but unless there are changes that break existing > > component > > > > > > > properties, upgrading the AWS SDK could be worked independently. > > > > > Others may > > > > > > > have more insight into particular usage of that library. > > > > > > > > > > > > > > Regards, > > > > > > > David Handermann > > > > > > > > > > > > > > On Sun, Jul 25, 2021 at 2:12 AM Chris Sampson > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > Might be worth considering refactoring the build as part of > > this > > > > work > > > > > > > too, > > > > > > > > e.g. only building the bits of the repo affected by a commit, > > > etc. > > > > - > > > > > > > > discussed briefly in previous threads but don't think any > > changes > > > > > made > > > > > > > yet. > > > > > > > > If NARs/components are likely to be split up and refactored > > then > > > > such > > > > > > > work > > > > > > > > around the build probably makes sense to consider. > > > > > > > > > > > > > > > > I've a couple of PRs open that include updates to Elasticsearch > > > > > versions > > > > > > > > already, although I stopped at 7.10.2 (after which Elastic > > > changed > > > > > > > licence) > > > > > > > > in case there were licence concerns. But more work can be done > > to > > > > > tidy up > > > > > > > > the processors, absolutely. > > > > > > > > > > > > > > > > AWS libraries to v2 would seem a sensible move and a refactor > > of > > > > > those > > > > > > > > processors as well. > > > > > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > > > > > > Chris Sampson > > > > > > > > > > > > > > > > On Sat, 24 Jul 2021, 17:47 David Handermann, < > > > > > > > [email protected]> > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Thanks for pointing out the standard NAR bundles, Mark. There > > > > are a > > > > > > > > number > > > > > > > > > of components in the standard NAR bundles with particular > > > > > dependencies > > > > > > > > that > > > > > > > > > would make more sense in separate NARs. Reorganizing the > > > standard > > > > > NAR > > > > > > > to > > > > > > > > > components with limited dependencies and wide applicability > > > would > > > > > > > > > definitely help with future maintenance. > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > David Handermann > > > > > > > > > > > > > > > > > > On Sat, Jul 24, 2021 at 10:57 AM Mark Payne < > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > There’s also some code that exists in order to maintain > > > > backward > > > > > > > > > > compatibility in the repositories. I would very much like > > the > > > > > > > > > repositories > > > > > > > > > > to contain no unnecessary code. And swap file format > > supports > > > > > really > > > > > > > > old > > > > > > > > > > formats. And the old impls of the repositories themselves, > > > like > > > > > > > > > > PersistentProvRepo instead of WriteAheadProv Repo, etc. > > Lots > > > of > > > > > stuff > > > > > > > > > there > > > > > > > > > > that can be removed. And some methods in ProcessSession > > that > > > > are > > > > > > > never > > > > > > > > > used > > > > > > > > > > by any processor in the codebase but exists in the public > > API > > > > so > > > > > > > can’t > > > > > > > > be > > > > > > > > > > removed till 2.0. > > > > > > > > > > > > > > > > > > > > I think his is also a great time to clean up the “standard > > > > nar.” > > > > > At > > > > > > > > this > > > > > > > > > > point, it’s something like 70 MB. And many of the > > components > > > > > there > > > > > > > are > > > > > > > > > not > > > > > > > > > > really “standard” - things like connecting to FTP & SFTP > > > > > servers, XML > > > > > > > > > > processing, Jolt transform, etc. could potentially be moved > > > > into > > > > > > > other > > > > > > > > > > nars. The nifi-standard-content-viewer-1.15.0-SNAPSHOT.war > > is > > > > > 6.9 MB > > > > > > > is > > > > > > > > > not > > > > > > > > > > necessary for stateless or minifi java. Lots of things > > > probably > > > > > to > > > > > > > > > > reconsider within the standard nar. > > > > > > > > > > > > > > > > > > > > I definitely think this is a reasonable approach, to allow > > > for > > > > a > > > > > 2.0 > > > > > > > > that > > > > > > > > > > is not a huge feature release but allows the project to be > > > > > simpler > > > > > > > and > > > > > > > > > more > > > > > > > > > > nimble. > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > -Mark > > > > > > > > > > > > > > > > > > > > On Jul 24, 2021, at 10:59 AM, Mike Thomsen < > > > > > [email protected] > > > > > > > > > <mailto: > > > > > > > > > > [email protected]>> wrote: > > > > > > > > > > > > > > > > > > > > Russell, > > > > > > > > > > > > > > > > > > > > AFAICT from looking at Elastic's repos, the low level REST > > > > > client is > > > > > > > > > > still fine. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/elastic/elasticsearch/blob/e5518e07f13701e3bb3dcc6842b9023966752497/client/rest/src/main/java/org/elasticsearch/client/RestClient.java > > > > > > > > > > > > > > > > > > > > Our Elasticsearch support is spread over two NARs at > > present. > > > > One > > > > > > > uses > > > > > > > > > > OkHttp and the other uses that low level Elastic REST > > client. > > > > > > > > > > Therefore, I think we're fine on licensing for the moment. > > > > > > > > > > > > > > > > > > > > Mike > > > > > > > > > > > > > > > > > > > > On Fri, Jul 23, 2021 at 1:10 PM Russell Bateman < > > > > > > > [email protected] > > > > > > > > > > <mailto:[email protected]>> wrote: > > > > > > > > > > > > > > > > > > > > Bringing up Elastic also reminds me that the Elastic > > > framework > > > > > has > > > > > > > just > > > > > > > > > > recently transitioned out of Open Source, so to acknowledge > > > > that, > > > > > > > maybe > > > > > > > > > > some effort toward OpenSearch--I say this not understanding > > > > > exactly > > > > > > > how > > > > > > > > > > this sort of thing is considered in a large-scale, > > > world-class > > > > > > > software > > > > > > > > > > project like Apache NiFi. (I'm not a contributor, just a > > > > grateful > > > > > > > > > > consumer.) > > > > > > > > > > > > > > > > > > > > Russ > > > > > > > > > > > > > > > > > > > > On 7/23/21 10:28 AM, Matt Burgess wrote: > > > > > > > > > > Along with the itemized list for ancient components we > > should > > > > > look at > > > > > > > > > > updating versions of drivers, SDKs, etc. for external > > systems > > > > > such as > > > > > > > > > > Elasticsearch, Cassandra, etc. There may be breaking > > changes > > > > but > > > > > 2.0 > > > > > > > > > > is probably the right time to get things up to date to make > > > > them > > > > > more > > > > > > > > > > useful to more people. > > > > > > > > > > > > > > > > > > > > On Fri, Jul 23, 2021 at 12:21 PM Nathan Gough < > > > > > [email protected] > > > > > > > > > <mailto: > > > > > > > > > > [email protected]>> wrote: > > > > > > > > > > I'm a +1 for removing pretty much all of this stuff. There > > > are > > > > > > > security > > > > > > > > > > implications to keeping old dependencies around, so the > > more > > > > old > > > > > code > > > > > > > > we > > > > > > > > > > can remove the better. I agree that eventually we need to > > > move > > > > to > > > > > > > > > > supporting only Java 11+, and as our next release will > > > probably > > > > > be > > > > > > > > about > > > > > > > > > 4 > > > > > > > > > > - 6 months from now that doesn't seem too soon. We could > > > > > potentially > > > > > > > > > break > > > > > > > > > > this in two and remove the deprecated processors and leave > > > 1.x > > > > on > > > > > > > Java > > > > > > > > 8, > > > > > > > > > > and finally start on 2.x which would support only Java 11. > > > I'm > > > > > unsure > > > > > > > > of > > > > > > > > > > what implications changing the date and time handling would > > > > have > > > > > - > > > > > > > for > > > > > > > > > > running systems that use long term historical logs, > > > unexpected > > > > > > > impacts > > > > > > > > to > > > > > > > > > > time logging could be a problem. > > > > > > > > > > > > > > > > > > > > As Joe says I think feature work will have to be dedicated > > to > > > > > 2.x and > > > > > > > > we > > > > > > > > > > could support 1.x for security fixes for some period of > > time. > > > > 2.x > > > > > > > seems > > > > > > > > > > like a gargantuan task but it's probably time to get > > started. > > > > Not > > > > > > > sure > > > > > > > > > how > > > > > > > > > > we handle all open PRs and the transition between 1.x and > > > 2.x. > > > > > > > > > > > > > > > > > > > > On Fri, Jul 23, 2021 at 10:57 AM Joe Witt < > > > [email protected] > > > > > > > <mailto: > > > > > > > > > > [email protected]>> wrote: > > > > > > > > > > > > > > > > > > > > Jon > > > > > > > > > > > > > > > > > > > > You're right we have to be careful and you're right there > > are > > > > > still > > > > > > > > > > significant Java 8 users out there. But we also have to be > > > > > careful > > > > > > > > > > about security and sustainability of the codebase. If we > > had > > > > > talked > > > > > > > > > > about this last year when that article came out I'd have > > > agreed > > > > > it is > > > > > > > > > > too early. Interestingly that link seems to get updated > > and I > > > > > tried > > > > > > > > > > [1] and found more recent data (not sure how recent). > > Anyway > > > it > > > > > > > > > > suggests Java 8 is still the top dog but we see good growth > > > on > > > > > 11. In > > > > > > > > > > my $dayjob this aligns to what I'm seeing too. Customers > > > didn't > > > > > seem > > > > > > > > > > to care about Java 11 until later half last year and now > > > > > suddenly it > > > > > > > > > > is all over the place. > > > > > > > > > > > > > > > > > > > > I think once we put out a NiFi 2.0 release we'd see rapid > > > > > decrease in > > > > > > > > > > work on the 1.x line just being blunt. We did this many > > years > > > > ago > > > > > > > > > > with 0.x to 1.x and we stood behind 0.x for a while (maybe > > a > > > > > year or > > > > > > > > > > so) but it was purely bug fix/security related bits. We > > would > > > > > need to > > > > > > > > > > do something similar. But feature work would almost > > certainly > > > > go > > > > > to > > > > > > > > > > the 2.x line. Maybe there are other workable models but my > > > > > instinct > > > > > > > > > > suggests this is likely to follow a similar path. > > > > > > > > > > > > > > > > > > > > ...anyway I agree it isn't that easy of a call to dump Java > > > 8. > > > > We > > > > > > > > > > need to make the call in both the interests of the user > > base > > > > and > > > > > the > > > > > > > > > > contributor base of the community. > > > > > > > > > > > > > > > > > > > > [1] https://www.jetbrains.com/lp/devecosystem-2021/java/ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > Joe > > > > > > > > > > > > > > > > > > > > On Fri, Jul 23, 2021 at 7:46 AM Joe Witt < > > [email protected] > > > > > <mailto: > > > > > > > > > > [email protected]>> wrote: > > > > > > > > > > Russ > > > > > > > > > > > > > > > > > > > > Yeah the flow registry is a key part of it. But also now > > you > > > > can > > > > > > > > > > download the flow definition in JSON (upload i think is > > there > > > > now > > > > > > > > > > too). Templates offered a series of challenges such as we > > > store > > > > > them > > > > > > > > > > in the flow definition which has made flows massive in an > > > > > unintended > > > > > > > > > > way which isn't fun for cluster behavior. > > > > > > > > > > > > > > > > > > > > We have a couple cases where we headed down a particular > > > > concept > > > > > and > > > > > > > > > > came up with better approaches later. We need to reconcile > > > > these > > > > > with > > > > > > > > > > the benefit of hindsight, and while being careful to be not > > > > > overly > > > > > > > > > > disruptive to existing users, to reduce the > > > > codebase/maintenance > > > > > > > > > > burden and allow continued evolution of the project. > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > On Fri, Jul 23, 2021 at 7:43 AM Russell Bateman < > > > > > > > [email protected] > > > > > > > > > > <mailto:[email protected]>> > > > > > > > > > > wrote: > > > > > > > > > > Joe, > > > > > > > > > > > > > > > > > > > > I apologize for the off-topic intrusion, but what replaces > > > > > templates? > > > > > > > > > > The Registry? Templates rocked and we have used them since > > > > 0.5.x. > > > > > > > > > > > > > > > > > > > > Russ > > > > > > > > > > > > > > > > > > > > On 7/23/21 8:31 AM, Joe Witt wrote: > > > > > > > > > > David, > > > > > > > > > > > > > > > > > > > > I think this is a highly reasonable approach and such a > > focus > > > > > will > > > > > > > > > > greatly help make a 2.0 release far more approachable to > > > knock > > > > > out. > > > > > > > > > > Not only that but tech debt reduction would help make work > > > > > towards > > > > > > > > > > major features we'd think about in a 'major release' sense > > > more > > > > > > > > > > approachable. > > > > > > > > > > > > > > > > > > > > We should remove all deprecated things (as well as verify > > we > > > > > have the > > > > > > > > > > right list). We should remove/consider removal of > > deprecated > > > > > > > > > > concepts > > > > > > > > > > like templates. We should consider whether we can resolve > > the > > > > > > > > > > various > > > > > > > > > > ways we've handled what are now parameters down to one > > clean > > > > > > > > > > approach. > > > > > > > > > > We should remove options in the nifi.properties which turn > > > out > > > > to > > > > > > > > > > never be used quite right (if there are). There is quite a > > > bit > > > > we > > > > > > > > > > can > > > > > > > > > > do purely in the name of tech debt reduction. > > > > > > > > > > > > > > > > > > > > Lots to consider here but I think this is the right > > > discussion. > > > > > > > > > > > > > > > > > > > > Than ks > > > > > > > > > > > > > > > > > > > > On Fri, Jul 23, 2021 at 7:26 AM Bryan Bende < > > > [email protected] > > > > > > > <mailto: > > > > > > > > > > [email protected]>> > > > > > > > > > > wrote: > > > > > > > > > > I'm a +1 for this... Not sure if this falls under "Removing > > > > > > > > > > Deprecated > > > > > > > > > > Components", but I think we should also look at anything > > that > > > > has > > > > > > > > > > been > > > > > > > > > > marked as deprecated throughout the code base as a > > candidate > > > > for > > > > > > > > > > removal. There are quite a few classes, methods, > > properties, > > > > etc > > > > > > > > > > that > > > > > > > > > > have been waiting for a chance to be removed. > > > > > > > > > > > > > > > > > > > > On Fri, Jul 23, 2021 at 10:13 AM David Handermann > > > > > > > > > > <[email protected]<mailto: > > > > [email protected] > > > > > >> > > > > > > > > wrote: > > > > > > > > > > Team, > > > > > > > > > > > > > > > > > > > > With all of the excellent work that many have contributed > > to > > > > NiFi > > > > > > > > > > over the > > > > > > > > > > years, the code base has also accumulated some amount of > > > > > technical > > > > > > > > > > debt. A > > > > > > > > > > handful of components have been marked as deprecated, and > > > some > > > > > > > > > > components > > > > > > > > > > remain in the code base to support integration with old > > > > versions > > > > > > > > > > of various > > > > > > > > > > products. Following the principles of semantic versioning, > > > > > > > > > > introducing a > > > > > > > > > > major release would provide the opportunity to remove these > > > > > > > > > > deprecated and > > > > > > > > > > unsupported components. > > > > > > > > > > > > > > > > > > > > Rather than focusing the next major release on new > > features, > > > > what > > > > > > > > > > do you > > > > > > > > > > think about focusing on technical debt removal? This > > approach > > > > > > > > > > would not > > > > > > > > > > make for the most interesting release, but it provides the > > > > > > > > > > opportunity to > > > > > > > > > > clean up elements that involve breaking changes. > > > > > > > > > > > > > > > > > > > > Focusing on technical debt, at least three primary goals > > come > > > > to > > > > > > > > > > mind for > > > > > > > > > > the next major release: > > > > > > > > > > > > > > > > > > > > 1. Removal of deprecated and unmaintained components > > > > > > > > > > 2. Require Java 11 as the minimum supported version > > > > > > > > > > 3. Transition internal date and time handling to JSR 310 > > > > > java.time > > > > > > > > > > components > > > > > > > > > > > > > > > > > > > > *Removing Deprecated Components* > > > > > > > > > > > > > > > > > > > > Removing support for older and deprecated components > > > provides a > > > > > > > > > > great > > > > > > > > > > opportunity to improve the overall security posture when it > > > > comes > > > > > > > > > > to > > > > > > > > > > maintaining dependencies. The OWASP dependency plugin > > report > > > > > > > > > > currently > > > > > > > > > > generates 50 MB of HTML for questionable dependencies, many > > > of > > > > > > > > > > which are > > > > > > > > > > related to old versions of various libraries. > > > > > > > > > > > > > > > > > > > > As a starting point, here are a handful of components and > > > > > > > > > > extension modules > > > > > > > > > > that could be targeted for removal in a major version: > > > > > > > > > > > > > > > > > > > > - PostHTTP and GetHTTP > > > > > > > > > > - ListenLumberjack and the entire nifi-lumberjack-bundle > > > > > > > > > > - ListenBeats and the entire nifi-beats-bundle > > > > > > > > > > - Elasticsearch 5 components > > > > > > > > > > - Hive 1 and 2 components > > > > > > > > > > > > > > > > > > > > *Requiring Java 11* > > > > > > > > > > > > > > > > > > > > Java 8 is now over seven years old, and NiFi has supported > > > > > general > > > > > > > > > > compatibility with Java 11 for several years. NiFi 1.14.0 > > > > > > > > > > incorporated > > > > > > > > > > internal improvements specifically related to TLS 1.3, > > which > > > > > > > > > > allowed > > > > > > > > > > closing out the long-running Java 11 compatibility epic > > > > > NIFI-5174. > > > > > > > > > > Making > > > > > > > > > > Java 11 the minimum required version provides the > > opportunity > > > > to > > > > > > > > > > address > > > > > > > > > > any lingering edge cases and put NiFi in a better position > > to > > > > > > > > > > support > > > > > > > > > > current Java versions. > > > > > > > > > > > > > > > > > > > > *JSR 310 for Date and Time Handling* > > > > > > > > > > > > > > > > > > > > Without making the scope too broad, transitioning internal > > > date > > > > > > > > > > and time > > > > > > > > > > handling to use DateTimeFormatter instead of > > SimpleDateFormat > > > > > > > > > > would provide > > > > > > > > > > a number of advantages. The Java Time components provide > > much > > > > > > > > > > better > > > > > > > > > > clarity when it comes to handling localized date and time > > > > > > > > > > representations, > > > > > > > > > > and also avoid the inherent confusion of java.sql.Date > > > > extending > > > > > > > > > > java.util.Date. Many internal components, specifically > > > > > > > > > > Record-oriented > > > > > > > > > > processors and services, rely on date parsing, leading to > > > > > > > > > > confusion and > > > > > > > > > > various workarounds. The pattern formats of > > SimpleDateFormat > > > > and > > > > > > > > > > DateTimeFormatter are very similar, but there are a few > > > subtle > > > > > > > > > > differences. > > > > > > > > > > Making this transition would provide a much better > > foundation > > > > > > > > > > going forward. > > > > > > > > > > *Conclusion* > > > > > > > > > > > > > > > > > > > > Thanks for giving this proposal some consideration. Many of > > > you > > > > > > > > > > have been > > > > > > > > > > developing NiFi for years and I look forward to your > > > feedback. > > > > I > > > > > > > > > > would be > > > > > > > > > > glad to put together a more formalized recommendation on > > > > > > > > > > Confluence and > > > > > > > > > > write up Jira epics if this general approach sounds > > agreeable > > > > to > > > > > > > > > > the > > > > > > > > > > community. > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > David Handermann > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
