Jeff / all, I ran into an issue with NIFI-375 and re-opened the ticket. If a processor is stopped or started in a cluster, the stats that come back in the response are incorrect because the response is not being properly merged from all nodes in the cluster.
I also have run into a couple of other issues that appear to have been introduced in 1.8. I have not yet created JIRA's for them because I want to understand the issues better before trying to write it up. I am seeing, for example, some errors in the logs indicating that provenance events don't have a FlowFile UUID assigned to them when being read from the repository. This seems to have been introduced in 1.8 so would like to get a resolution before releasing. Thanks -Mark > On Oct 15, 2018, at 6:33 PM, Jeff <jtsw...@gmail.com> wrote: > > NiFi Devs, > > The Release page [1] for 1.8.0 now reports that all issues are done! I'd > like to start the release candidate preparation tomorrow, around 1200 EST. > > Thanks to everyone for all the great work that's been done! 196 issues > resolved in this version with some great new features! > > [1] https://issues.apache.org/jira/projects/NIFI/versions/12343482 > > On Mon, Oct 15, 2018 at 7:30 AM Sivaprasanna <sivaprasanna...@gmail.com> > wrote: > >> Great. Thanks. :) >> >> - >> Sivaprasanna >> >> On Mon, Oct 15, 2018 at 7:09 AM Koji Kawamura <ijokaruma...@gmail.com> >> wrote: >> >>> Jeff, Sivasprasanna, >>> >>> NIFI-5698 (PR3073) Fixing DeleteAzureBlob bug is merged. >>> >>> Thanks, >>> Koji >>> On Mon, Oct 15, 2018 at 10:18 AM Koji Kawamura <ijokaruma...@gmail.com> >>> wrote: >>>> >>>> Thank you for the fix Sivaprasanna, >>>> I have Azure account. Reviewing it now. >>>> >>>> Koji >>>> On Sun, Oct 14, 2018 at 11:21 PM Jeff <jtsw...@gmail.com> wrote: >>>>> >>>>> Sivaprasanna, >>>>> >>>>> Thanks for submitting a pull request for that issue! Later today or >>>>> tomorrow I'll have to check to see if I've already used up my >> free-tier >>>>> access to Azure. If I still have access, I can review your PR and >>> we'll >>>>> get it into 1.8.0. >>>>> >>>>> On Sun, Oct 14, 2018 at 4:30 AM Sivaprasanna < >>> sivaprasanna...@gmail.com> >>>>> wrote: >>>>> >>>>>> All - Just found one bug with DeleteAzureBlobStorage processor. It >>> was >>>>>> shared by one user on StackOverflow [1] and I later confirmed it. >> It >>> looks >>>>>> to be introduced by NIFI-4199. I have created a Jira [2] and made >> the >>>>>> necessary changes (not huge, just few lines) and raised a PR [3]. I >>> think, >>>>>> if we can spend a little time in getting it reviewed, we can mark >> it >>> for >>>>>> 1.8.0. Thoughts? >>>>>> >>>>>> [1] - >>>>>> >>>>>> >>> >> https://stackoverflow.com/questions/52766991/apache-nifi-deleteazureblobstorage-processor-is-throwing-an-error >>>>>> [2] - https://issues.apache.org/jira/browse/NIFI-5698 >>>>>> [3] - https://github.com/apache/nifi/pull/3073 >>>>>> >>>>>> - >>>>>> Sivaprasanna >>>>>> >>>>>> On Fri, Oct 12, 2018 at 9:05 PM Mike Thomsen < >> mikerthom...@gmail.com >>>> >>>>>> wrote: >>>>>> >>>>>>> 4811 should be ready for review now. Rebased and cleaned it up >>> with a >>>>>> full >>>>>>> listing of the Spring dependencies. >>>>>>> >>>>>>> On Fri, Oct 12, 2018 at 11:23 AM Joe Witt <joe.w...@gmail.com> >>> wrote: >>>>>>> >>>>>>>> Jeff, >>>>>>>> >>>>>>>> I think for anything not tagged to 1.8.0 we just keep rolling. >>> For >>>>>>>> anything tagged 1.8.0 that should not be we should remove it >>> until >>>>>>>> ready. For things tagged to 1.8.0 that cannot be moved we >> should >>>>>>>> resolve. For the tagged 1.8.0 section you had. >>>>>>>> >>>>>>>> - NIFI-4811 < >> https://issues.apache.org/jira/browse/NIFI-4811> >>> - >>>>>> Use a >>>>>>>> newer version of spring-data-redis >>>>>>>> - PR 2856 <https://github.com/apache/nifi/pull/2856> >>>>>>>> *This needs to be resolved by either reverting the commit or >>> ensuring >>>>>>>> L&N accurately reflects all. We have to do this always and for >>> every >>>>>>>> nar. The process isnt easy or fun but it is necessary to >> produce >>>>>>>> valid ASF releases. Landing commits which change dependencies >>>>>>>> requires this due diligence. Now, we've put a lot of energy >> into >>>>>>>> updating Spring dependencies because some older Spring libs had >>>>>>>> vulnerabilities which while we likely aren't exposed to them we >>> want >>>>>>>> to fix in due course. So reverting may require more analysis >>> than if >>>>>>>> we were just get L&N fixed with this new change. I commented >> on >>> the >>>>>>>> JIRA. But this needs to be resolved. >>>>>>>> >>>>>>>> >>>>>>>> - NIFI-5426 < >> https://issues.apache.org/jira/browse/NIFI-5426> >>> - Use >>>>>>>> NIO.2 API for ListFile to avoid multiple disk reads >>>>>>>> - PR 2889 <https://github.com/apache/nifi/pull/2889> >>>>>>>> *This just needed to be marked resolved. The commit went in >> the >>> day >>>>>>>> after we cut 1.7.1. So this one is sorted. >>>>>>>> >>>>>>>> - NIFI-5448 < >> https://issues.apache.org/jira/browse/NIFI-5448> >>> - >>>>>>> Failed >>>>>>>> EL date parsing live-locks processors without a failure >>> relationship >>>>>>>> * The commit needs to be reverted. I'm working on that now. >>> Once the >>>>>>>> discsusion/concerns are addressed this can get dealt with. >>>>>>>> >>>>>>>> - NIFI-5665 < >> https://issues.apache.org/jira/browse/NIFI-5665> >>> - >>>>>>> Upgrade >>>>>>>> io.netty dependencies >>>>>>>> * This looks important to get resolved if possible as old netty >>> libs >>>>>>>> are on the list of things with vulnerabilities. >>>>>>>> >>>>>>>> - NIFI-5686 < >> https://issues.apache.org/jira/browse/NIFI-5686> >>> - >>>>>> Test >>>>>>>> failure in TestStandardProcessScheduler >>>>>>>> - PR 3062 <https://github.com/apache/nifi/pull/3062> >>>>>>>> * This has a PR but a test, possibly two, failed in one of the >>> travis >>>>>>>> runs and it is clearly related. I ignored one of those tests >> in >>> a >>>>>>>> previous run. We must deal with brittle tests. But the >>> underlying >>>>>>>> problem is important to solve here so either the tests needs >>> improved >>>>>>>> or we still have an issue. Not clear but worth some focus. >>>>>>>> >>>>>>>> note: I intend to reference updates to libraries that have >> known >>>>>>>> vulnerabilities and do so in a far less subtle manner than we >>> had. We >>>>>>>> aren't acknowledging that NiFi is or exposes vulnerabilities >> but >>> we >>>>>>>> are and should be clear when we're updating dependencies that >> do >>> have >>>>>>>> them (even if we're not exposed to them) so that some of these >>> commits >>>>>>>> aren't so mysterious. It creates far more confusion than is >>> worth. >>>>>>>> We still will follow the ASF/NiFi security handling policy but >> I >>> no >>>>>>>> longer intend to treat due course dependency updates as if they >>> need >>>>>>>> to be a secret. >>>>>>>> >>>>>>>> Thanks >>>>>>>> Joe >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Oct 12, 2018 at 3:32 AM Jeff <jtsw...@gmail.com> >> wrote: >>>>>>>>> >>>>>>>>> Hello everyone! Next week is probably a good timeframe to >> aim >>> for a >>>>>>>>> release candidate, with two major feature PRs recently merged >>> to >>>>>>> master: >>>>>>>>> >>>>>>>>> - NIFI-5516 < >>> https://issues.apache.org/jira/browse/NIFI-5516> - >>>>>>> Allow >>>>>>>>> data in a Connection to be Load-Balanced across cluster >>>>>>>>> - NIFI-5585 < >>> https://issues.apache.org/jira/browse/NIFI-5585> - >>>>>>>> Prepare >>>>>>>>> Nodes to be Offloaded >>>>>>>>> >>>>>>>>> >>>>>>>>> To recap, here's a list of other JIRAs mentioned in this >>> thread: >>>>>>>>> >>>>>>>>> - NIFI-5402 < >>> https://issues.apache.org/jira/browse/NIFI-5402> - >>>>>>>> Reduce >>>>>>>>> artifact size by only building .zip archive >>>>>>>>> - NIFI-5462 < >>> https://issues.apache.org/jira/browse/NIFI-5462> - >>>>>>>> Refactor >>>>>>>>> TLS Toolkit >>>>>>>>> - NIFI-5485 < >>> https://issues.apache.org/jira/browse/NIFI-5485> - >>>>>>>> Enable >>>>>>>>> TLS Toolkit (client/server) to sign certificates with >>> external CA >>>>>>>>> certificate >>>>>>>>> - NIFI-5537 < >>> https://issues.apache.org/jira/browse/NIFI-5537> - >>>>>>>> Create >>>>>>>>> Neo4J cypher execution processor >>>>>>>>> - PR 2956 <https://github.com/apache/nifi/pull/2956> >>>>>>>>> - Mike Thomsen, this was the specific JIRA to which you >>> were >>>>>>>>> referring, right? >>>>>>>>> - NIFI-5582 < >>> https://issues.apache.org/jira/browse/NIFI-5582> - >>>>>>>> Integrate >>>>>>>>> legacy behavior of HashAttribute into >>> CryptographicHashAttribute >>>>>>>>> >>>>>>>>> >>>>>>>>> These JIRAs are marked with a fix version of 1.8.0 that are >> not >>>>>>> currently >>>>>>>>> resolved: >>>>>>>>> >>>>>>>>> - NIFI-4811 < >>> https://issues.apache.org/jira/browse/NIFI-4811> - >>>>>>> Use a >>>>>>>>> newer version of spring-data-redis >>>>>>>>> - PR 2856 <https://github.com/apache/nifi/pull/2856> >>>>>>>>> - NIFI-5426 < >>> https://issues.apache.org/jira/browse/NIFI-5426> - >>>>>> Use >>>>>>>>> NIO.2 API for ListFile to avoid multiple disk reads >>>>>>>>> - PR 2889 <https://github.com/apache/nifi/pull/2889> >>>>>>>>> - NIFI-5448 < >>> https://issues.apache.org/jira/browse/NIFI-5448> - >>>>>>>> Failed >>>>>>>>> EL date parsing live-locks processors without a failure >>>>>> relationship >>>>>>>>> - NIFI-5665 < >>> https://issues.apache.org/jira/browse/NIFI-5665> - >>>>>>>> Upgrade >>>>>>>>> io.netty dependencies >>>>>>>>> - NIFI-5686 < >>> https://issues.apache.org/jira/browse/NIFI-5686> - >>>>>>> Test >>>>>>>>> failure in TestStandardProcessScheduler >>>>>>>>> - PR 3062 <https://github.com/apache/nifi/pull/3062> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Oct 4, 2018 at 6:39 AM Mike Thomsen < >>> mikerthom...@gmail.com> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> That's a fair point. Only thing I could add there is that I >>> think >>>>>> we >>>>>>>> should >>>>>>>>>> consider a targeted burn down on the PR list as part of >> 1.9. >>> There >>>>>>> are >>>>>>>> a >>>>>>>>>> lot of PRs from the last several months that would be good >>>>>> candidates >>>>>>>> to >>>>>>>>>> see if we can close them out like MarkLogic and Pulsar. >>>>>>>>>> >>>>>>>>>> On Wed, Oct 3, 2018 at 10:34 PM Joe Witt < >> joe.w...@gmail.com >>>> >>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Mike, >>>>>>>>>>> >>>>>>>>>>> Processors in particularly are among the toughest at this >>> point. >>>>>>> We >>>>>>>>>>> have very very little headroom on dependency size for the >>> full >>>>>>> build >>>>>>>>>>> size that we upload to ASF infra and mirrors. That and >> the >>>>>> license >>>>>>>>>>> review work involved in each... >>>>>>>>>>> >>>>>>>>>>> We should really create a way to publish processors on >> more >>>>>>> frequent, >>>>>>>>>>> irregular intervals where the release work and size/etc.. >>> are far >>>>>>>> less >>>>>>>>>>> problematic. We have another discuss thread on that so >>> I'll >>>>>> leave >>>>>>> it >>>>>>>>>>> there for discussion. I do share your view that this >>> processor >>>>>>>> (among >>>>>>>>>>> several others outstanding) would be really useful but i >> am >>>>>>>> definitely >>>>>>>>>>> thinking we should keep release pace up. Release more >>>>>>>> often...release >>>>>>>>>>> processors separately, etc.. >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> Joe >>>>>>>>>>> On Wed, Oct 3, 2018 at 9:30 PM Mike Thomsen < >>>>>>> mikerthom...@gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> I would like to see the Neo4J work that mans2singh is >>> doing get >>>>>>>>>> included. >>>>>>>>>>>> Being able to at least partially support a popular >> graph >>>>>> database >>>>>>>> would >>>>>>>>>>> be >>>>>>>>>>>> a nice feather in our cap. >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Oct 3, 2018 at 5:12 PM Andy LoPresto < >>>>>>> alopre...@apache.org >>>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I am currently working on a TLS Toolkit refactor >>> (NIFI-5462 & >>>>>>>>>>> NIFI-5485) >>>>>>>>>>>>> and HashAttribute updates (NIFI-5582). I believe >> there >>> are a >>>>>>>> couple >>>>>>>>>>> upgrade >>>>>>>>>>>>> PRs open, and I would really like to see NIFI-5402 >> (no >>>>>> .tar.gz >>>>>>>> in the >>>>>>>>>>>>> build) tackled for this release as well. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Andy LoPresto >>>>>>>>>>>>> alopre...@apache.org >>>>>>>>>>>>> *alopresto.apa...@gmail.com < >>> alopresto.apa...@gmail.com>* >>>>>>>>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E >>> F65B >>>>>> 2F7D >>>>>>>> EF69 >>>>>>>>>>>>> >>>>>>>>>>>>> On Oct 3, 2018, at 11:16 AM, Joe Witt < >>> joew...@apache.org> >>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Jeff - thanks again for volunteering. I just went >>> through >>>>>> the >>>>>>>> open >>>>>>>>>>>>> items tagged to 1.8.0 to try and shake some loose, >>> close down >>>>>>>> ones >>>>>>>>>>>>> that appear to be done but forgotten, and initiate >>> resolution >>>>>>> on >>>>>>>> one >>>>>>>>>>>>> that is in a dangling state. >>>>>>>>>>>>> >>>>>>>>>>>>> Another very nice release shaping up here. All the >>> work >>>>>> around >>>>>>>> load >>>>>>>>>>>>> balancing and node offloading is awesome. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks >>>>>>>>>>>>> On Wed, Oct 3, 2018 at 2:06 PM Jeff < >>> jsto...@apache.org> >>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> It looks like we're getting close to a point where we >>> could >>>>>>>> release >>>>>>>>>>> NiFi >>>>>>>>>>>>> 1.8.0The release tracking page for version 1.8.0 [1] >>> shows 3 >>>>>>> "in >>>>>>>>>>> progress" >>>>>>>>>>>>> and 9 "to do" issues. In addition to what has been >>> tagged >>>>>> with >>>>>>> a >>>>>>>> fix >>>>>>>>>>>>> version of 1.8.0, it looks like NIFI-5516 and >>> NIFI-5585 are >>>>>>>> close to >>>>>>>>>>>>> completion. >>>>>>>>>>>>> >>>>>>>>>>>>> Are there other JIRAs that the community considers >>> necessary >>>>>>> for >>>>>>>> the >>>>>>>>>>>>> release that are close to being resolved, with the >>> goal of >>>>>>>> getting a >>>>>>>>>>>>> release candidate out in the next couple of weeks? >>>>>>>>>>>>> >>>>>>>>>>>>> I'm happy to perform the release manager duties! >>>>>>>>>>>>> >>>>>>>>>>>>> [1] >>>>>>>> https://issues.apache.org/jira/projects/NIFI/versions/12343482 >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>> >>
