I've reviewed NIFI-3064 and it's good to go, would like to add that one as well.
Thanks, Matt On Tue, Nov 22, 2016 at 10:51 AM, Joe Witt <joe.w...@gmail.com> wrote: > thanks Joe and Andy. Have those on the list now. > > On Tue, Nov 22, 2016 at 5:09 AM, Joe Gresock <jgres...@gmail.com> wrote: >> I'm biased, but I think the new Query / Scroll Elasticsearch processors are >> pretty big ;) >> >> On Tue, Nov 22, 2016 at 9:02 AM, Andy LoPresto <alopre...@apache.org> wrote: >> >>> Important to call out in the security section that >>> login-identity-providers.xml is now covered by encrypted config, and >>> flow.xml.gz processor property encryption/nifi.sensitive.props.key value >>> can now be migrated by the command-line tool (second part pending >>> completion of NIFI-3024). >>> >>> Andy LoPresto >>> alopre...@apache.org >>> *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* >>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>> >>> On Nov 21, 2016, at 9:30 PM, Joe Witt <joe.w...@gmail.com> wrote: >>> >>> matt >>> >>> i'll add a wiki page or set of instructions linked from the release notes. >>> >>> all, >>> >>> walked through the 250 or so JIRAs in the 1.1.0 release and pulled out >>> highlights. The items noted are as follows. Will likely reduce this >>> down further for the release notes but wanted to put this out in case >>> folks have things they think are really important to highlight. >>> >>> - Core Improvements: >>> - Performance: Session Migration >>> - Stability: Cluster Management >>> - Developer: Framework supports easy user driven classloader extension >>> - Expression Language: Now supports base64 and hex encoded values >>> and Math functions >>> - Repositories now support rollback >>> - Faster startup due to more efficient state restoration algorithm >>> - UX Improvements: >>> - Visual Backpressure Indicator >>> - Introduced more colors to better highlight actions and components >>> - Performance: Validate non-running components >>> - Provenance graph image can be exported >>> - Cron Scheduling for Primary node tasks now supported >>> - Updated versions >>> - Azure Event Hub 0.9.0 >>> - Spark 2.0.1 >>> - Hadoop 2.7.x >>> - New/Improved Processors >>> - new Fetch/Put Elastic Search 5.0 >>> - new ParseCEF to parse CEF formatted logs >>> - improve ExtractEmail now supports TNEF files >>> - new Validate CSV >>> - improved Solr processors now support SSL and Kerberos >>> - new Websocket client and server processors >>> - New Utility >>> - Zookeeper Migrator (move from one zookeeper to another) >>> - Security >>> - Restricted Processors >>> - Site-to-site now supports port forwarding >>> - Improved Policy Management UX >>> - Migration Notes: >>> - Restricted Processors >>> - Twitter Processor Removed >>> >>> On Mon, Nov 21, 2016 at 10:17 PM, Matt Burgess <mattyb...@apache.org> >>> wrote: >>> >>> Is there a good spot for us to put instructions on how to build the >>> Twitter processor and/or the Social Media NAR in the meantime? Maybe a >>> Wiki page or something simple to say "go to this directory, run this >>> Maven command, drop the NAR into your deployment..." ? >>> >>> On Mon, Nov 21, 2016 at 9:34 PM, Joe Witt <joe.w...@gmail.com> wrote: >>> >>> Team, >>> >>> We appear to be very close. Andy is working NIFI-3024 but otherwise >>> it is focus on testing. >>> >>> I'm going to prep the RC and release notes now. Unfortunately the >>> twitter changes for json.org will need to remain. Consensus forming >>> on the legal-discuss thread regarding a grace period has been elusive >>> and we're already prepared to make the right steps so we'll just need >>> to take that on by being empathetic to the user base. >>> >>> Thanks >>> Joe >>> >>> On Thu, Nov 17, 2016 at 7:37 AM, Andre <andre-li...@fucs.org> wrote: >>> >>> Andy, >>> >>> Great to see NIFI-3050 implemented and certainly good news that NiFi 1.1.0 >>> is set to include a number of security related improvements. >>> >>> >>> >>> On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <alopre...@apache.org> >>> wrote: >>> >>> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have been >>> added to my plate for this release. Coordinated with Joe Witt and they >>> should both be included. >>> >>> [1] https://issues.apache.org/jira/browse/NIFI-3050 >>> [2] https://issues.apache.org/jira/browse/NIFI-3051 >>> >>> Andy LoPresto >>> alopre...@apache.org >>> *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* >>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>> >>> On Nov 16, 2016, at 12:08 PM, Joe Witt <joe.w...@gmail.com> wrote: >>> >>> Team >>> >>> There is a thread on apache legal-discuss that might allow for a >>> graceperiod of continued usage of the json library. Am going to keep >>> a close eye on this and if VP Legal approves we'll be able to keep the >>> twitter processors in which is definitely a good thing. Will advise >>> >>> Thanks >>> Joe >>> >>> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <bbe...@gmail.com> wrote: >>> >>> I've noticed an issue with the per-instance class loading capability >>> introduced in NIFI-2909 where the additional classpath resources can get >>> incorrectly removed from the class loader. >>> >>> I was able to reproduced this with a unit test and have a fix ready. I >>> believe this is important and needs to go in for the 1.1 release, going to >>> re-open NIFI-2909 and submit a PR shortly. >>> >>> -Bryan >>> >>> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <matt.c.gil...@gmail.com> >>> wrote: >>> >>> I have two items that I would like to wrap up prior to creating an RC for >>> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group port >>> configuration. The work is already completed and I will be reviewing it >>> this today. Additionally, following recent interest on the mailing list, >>> I'd like to knock out NIFI-3020. This will allow an admin to configure a >>> strategy for user identity when logging in via LDAP. Specifically, it will >>> support usage of the DN (the default and current implementation) as well as >>> the username the user logged in as. I should be able to have a PR up for >>> this work later today. >>> >>> Thanks! >>> >>> Matt >>> >>> [1] https://issues.apache.org/jira/browse/NIFI-2949 >>> [2] https://issues.apache.org/jira/browse/NIFI-3020 >>> >>> >>> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <joe.w...@gmail.com> wrote: >>> >>> The code is within the twitter4j library itself. I filed a request to >>> twitter4jg. The most likely case is we will need to submit a PR to them. >>> However, I don't see this as something that should delay the release. We >>> can provide instructions for folks wanting to use the processor during >>> >>> the >>> >>> time we cannot make it available in a convenient manner. I will provide >>> >>> a >>> >>> meaningful comment about this in release notes and pointers on what folks >>> can do in the meantime. >>> >>> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <alopre...@apache.org> wrote: >>> >>> I understand there was a discussion thread within the NiFi community >>> >>> for >>> >>> this as well and I missed responding to that at that time. It just >>> >>> seems >>> >>> to >>> >>> me like JSON processing is necessary for GetTwitter, which is >>> >>> incredibly >>> >>> useful for demonstrating NiFi’s ability to read from a high volume >>> >>> stream >>> >>> out of the box. With NIFI-3019 (Remove GetTwitter from default build), >>> >>> is >>> >>> there any related effort to substitute an acceptable replacement JSON >>> library to restore this functionality? >>> >>> [1] https://issues.apache.org/jira/browse/NIFI-3019 >>> >>> Andy LoPresto >>> alopre...@apache.org >>> *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* >>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>> >>> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <alopre...@apache.org> >>> >>> wrote: >>> >>> >>> I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655, and >>> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we >>> investigated some alternate TLS config options for the new version of >>> >>> the >>> >>> client library. >>> >>> Is there any alternative to excluding the GetTwitter processor? Using >>> Johnzon [1] or the Android re-implementation [2] discussed in the >>> >>> mailing >>> >>> list thread? >>> >>> [1] https://johnzon.apache.org/ >>> [2] https://developer.android.com/reference/org/json/package- >>> >>> summary.html >>> >>> >>> >>> Andy LoPresto >>> alopre...@apache.org >>> *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* >>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>> >>> On Nov 15, 2016, at 3:58 PM, Joe Witt <joe.w...@gmail.com> wrote: >>> >>> Team >>> >>> Very happy to see that we are down to three items remaining tagged to >>> 1.1.0. Solid effort over the recent weeks to close the gap including >>> >>> work >>> >>> to get past the now category x Jason dependency we had. The most >>> >>> notable >>> >>> impact from that is the wildly popular GetTwitter processor, the fav >>> >>> new >>> >>> nifi user and demo processor, can no longer be included in the default >>> build. It is optionally available if users choose to build and use it >>> >>> but >>> >>> we won't distribute binaries that have it. >>> >>> I see some review movement on some patch available but untagged items. >>> >>> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri. Anyone >>> have any outstanding items? >>> >>> Thanks >>> Joe >>> >>> On Nov 8, 2016 2:12 PM, "Joe Witt" <joe.w...@gmail.com> wrote: >>> >>> Ryan >>> >>> Not officially but I think we should try to close this thing out and >>> start a vote in the next week or two at most. >>> >>> I'm going through the tickets again now. There is also a new issue of >>> the json-p license falling out of favor in Apache legal terms and >>> becoming Category-X. Am looking into that now. >>> >>> Thanks >>> Joe >>> >>> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <ryan.wa...@gmail.com> >>> >>> wrote: >>> >>> >>> Joe - Is there a target date for 1.1? >>> >>> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <joe.w...@gmail.com> wrote: >>> >>> Team, >>> >>> Just an update on things with working toward an Apache NiFi 1.1.0 >>> release. There are still about 33 JIRAs there now and some are >>> awaiting review and are some are under active progress. Yet there is >>> good traction and progress. I think we should just stay vigilant with >>> what makes it in and keep working it down. So let's please shoot for >>> a couple weeks from now. If it is ready sooner I'll jump on it. >>> >>> Thanks >>> Joe >>> >>> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <joe.w...@gmail.com> wrote: >>> >>> Team, >>> >>> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0. Let's >>> avoiding putting more in there for now at least without a discussion. >>> Of the 31 JIRAs there the vast majority need review so we should be >>> able to close these down fairly quickly as long as we don't let the >>> list grow. >>> >>> Thanks >>> joe >>> >>> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <edgardo.v...@gmail.com> >>> >>> wrote: >>> >>> Joe, >>> >>> Appreciate the offer it isn't my PR. I was just using it as an >>> >>> example. >>> >>> All >>> >>> mine are currently closed, which I greatly appreciate. >>> >>> Cheers, >>> >>> Edgardo >>> >>> On Friday, October 14, 2016, Joe Witt <joe.w...@gmail.com> wrote: >>> >>> Edgardo, >>> >>> You mentioned a PR from August. I'd be happy to help you work that >>> through review. >>> >>> Thanks >>> Joe >>> >>> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega < >>> >>> edgardo.v...@gmail.com >>> >>> <javascript:;>> wrote: >>> >>> I have agreed that at this point a release is important. My goal >>> >>> was >>> >>> try >>> >>> to >>> >>> squeeze in a much goodness as possible into the release, but the >>> >>> important >>> >>> bug fixes should come first. Getting 1.x into a state where the >>> >>> release >>> >>> notes don't say that it is geared toward developers and testers is >>> >>> really >>> >>> huge. >>> >>> I think Nifi is a great community otherwise I would participate in >>> >>> the >>> >>> mailing list, create Jira tickets and pull requests. I am only >>> >>> trying to >>> >>> strengthen the great thing that is going on here. We can always do >>> >>> better. >>> >>> I was not trying to put down this community only to participate and >>> >>> make >>> >>> it >>> >>> better. I think this conversation is an indication of how great >>> >>> this >>> >>> community is. >>> >>> Maybe I am being sensitive about this issue and trying to >>> >>> strengthen >>> >>> the >>> >>> nifi community even more, after coming from a conference where it >>> >>> was >>> >>> reported there was lots of excitement at first and now the >>> >>> participation >>> >>> in >>> >>> the community has really died down and they are struggling. I don't >>> >>> want >>> >>> to >>> >>> see that happen here. >>> >>> Cheers, >>> >>> Edgardo >>> >>> >>> >>> >>> On Fri, Oct 14, 2016 at 9:37 AM, Andre <andre-li...@fucs.org >>> >>> <javascript:;>> wrote: >>> >>> >>> Edgardo, >>> >>> Thank you for your feedback. We hear your comments and as a >>> >>> committer I >>> >>> can >>> >>> share we are constantly looking to improve the PR process, having >>> >>> already >>> >>> taken many of the steps you suggest. >>> >>> However, it is important to notice that the number of PRs should >>> >>> not be >>> >>> seen as a metric of engagement by the development community: Most >>> >>> of us >>> >>> will submit PRs so that our work can be carefully reviewed by our >>> >>> peers >>> >>> and >>> >>> some of us will use JIRA patches to provide contributions. >>> >>> Having said that, it is true that some PRs may sit idle for a long >>> >>> time >>> >>> and >>> >>> we are working to improve this pipeline. >>> >>> It was therefore no coincidence that I browsed most of the PRs >>> >>> performing >>> >>> a triage of items that have been superseded or diverged from the >>> >>> current >>> >>> code base. >>> >>> In fact, less than a month ago the dev team closed a number of >>> >>> stalled >>> >>> and >>> >>> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8). >>> >>> Despite all the above, I think Joe has a point. The master >>> >>> contain a >>> >>> series >>> >>> of important bug fixes and suspect the community would benefit >>> >>> from >>> >>> a >>> >>> release sooner rather than later. >>> >>> Once again, thank you for your feedback and contribution. It is >>> >>> good to >>> >>> have you here. >>> >>> Andre >>> >>> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega < >>> >>> edgardo.v...@gmail.com >>> >>> <javascript:;>> >>> >>> wrote: >>> >>> Joe - You are correct I was mentioning the PRs that are >>> >>> currently >>> >>> open. >>> >>> >>> Regardless of how it happens reducing the count of open PRs I >>> >>> believe >>> >>> to >>> >>> be >>> >>> extremely important. Maybe I was hoping that the release could >>> >>> be >>> >>> a >>> >>> forcing >>> >>> function to make that happen. I believe that developers are more >>> >>> willing >>> >>> to >>> >>> contribute when they see that their PRs will actually be able >>> >>> accepted >>> >>> and >>> >>> merged into the code base. Having a low number of open PRs in >>> >>> progress >>> >>> is a >>> >>> great indication that the main nifi developers are fully engaged >>> >>> with >>> >>> the >>> >>> community. >>> >>> There are a few PRs that don't have any comments from committers >>> >>> at >>> >>> all. >>> >>> I >>> >>> found one from August in that state. If that was my PR I don't >>> >>> think I >>> >>> would be so willing to put another one in anytime soon. I do get >>> >>> that >>> >>> sometime PRs get stalled by the originator, if so maybe a rule >>> >>> about >>> >>> closing them after a certain amount of time or being taken over >>> >>> by a >>> >>> core >>> >>> contributor if they think it worthwhile. >>> >>> I would like to shoutout to James Wing on my last PR he was >>> >>> quick >>> >>> to >>> >>> review, provided great comments, testing, and even some >>> >>> additional >>> >>> code. >>> >>> It >>> >>> was a great PR experience. >>> >>> Cheers, >>> >>> Edgardo >>> >>> >>> >>> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall < >>> >>> joeperciv...@yahoo.com <javascript:;>. >>> >>> invalid> wrote: >>> >>> Joe, I think you misread. Edgardo is referring to the Pull >>> >>> Requests >>> >>> that >>> >>> are currently open, not the tickets assigned to the 1.1.0 >>> >>> version. >>> >>> >>> I think these goals (releasing 1.1.0 and cutting down the PR >>> >>> count) >>> >>> should >>> >>> be two different efforts. Doing a thorough job reviewing >>> >>> takes a >>> >>> significant amount of time from both the reviewer and >>> >>> contributor. >>> >>> In >>> >>> order >>> >>> to cut it down significantly would take much longer than a >>> >>> couple >>> >>> days. >>> >>> >>> Also there has already been a lot of great new features and >>> >>> bug >>> >>> fixes >>> >>> contributed to the 1.X line and I don't think it's worth >>> >>> holding up >>> >>> a >>> >>> 1.1.0 >>> >>> release for tickets not assigned to this fix version. As an >>> >>> added >>> >>> bonus >>> >>> though, I think many of the tickets tagged as 1.1.0 have PRs >>> >>> already >>> >>> open >>> >>> so closing those will make a large dent in the PR count. >>> >>> >>> Joe >>> >>> - - - - - - >>> Joseph Percivall >>> linkedin.com/in/Percivall >>> e: joeperciv...@yahoo.com <javascript:;> >>> >>> >>> >>> On Thursday, October 13, 2016 3:58 PM, Joe Witt < >>> >>> joe.w...@gmail.com >>> >>> <javascript:;>> >>> >>> wrote: >>> >>> >>> >>> There are less than 30 right now. Many of the roughly 90+ >>> >>> JIRAs >>> >>> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed >>> >>> or >>> >>> just >>> >>> had fix versions removed. >>> >>> We will need to have a push over the next bunch of days to >>> >>> deal >>> >>> with >>> >>> reviewing/merging/moving the remaining items. >>> >>> Thanks >>> Joe >>> >>> >>> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega < >>> >>> edgardo.v...@gmail.com <javascript:;>> >>> >>> wrote: >>> >>> Joe, >>> >>> There are 75 PRs currently open. Why not make a push over >>> >>> the >>> >>> next >>> >>> bunch >>> >>> of >>> >>> days to get them closed and then cut the release after that. >>> >>> Cheers, >>> >>> Edgardo >>> >>> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt < >>> >>> joe.w...@gmail.com >>> >>> <javascript:;>> >>> >>> wrote: >>> >>> >>> Team, >>> >>> There have been a ton of bugs fixed a few nice features. I >>> >>> would >>> >>> like >>> >>> to move to get Apache NiFi 1.1.0 release going pretty much >>> >>> based >>> >>> on >>> >>> where we are now and plan to move most tickets to a new >>> >>> Apache >>> >>> NiFi >>> >>> 1.2.0 version. We can try to get back on our roughly 6-8 >>> >>> week >>> >>> release >>> >>> schedule and shoot for a mid to late Nov release for NiFi >>> >>> 1.2.0 >>> >>> this >>> >>> way as well. Please advise if anyone has any other views on >>> >>> this. In >>> >>> the mean time I'll get the wheels in motion so you'll be >>> >>> seeing a >>> >>> lot >>> >>> of JIRA/issue updates to move version around. >>> >>> Thanks >>> Joe >>> >>> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc < >>> >>> trk...@gmail.com >>> >>> <javascript:;>> >>> >>> wrote: >>> >>> Sounds good Joe. I have no issue to you doing the rm'ing >>> >>> for >>> >>> it. >>> >>> >>> On Oct 13, 2016 8:19 AM, "Joe Witt" <joe.w...@gmail.com >>> >>> <javascript:;>> wrote: >>> >>> >>> Team, >>> >>> There are a lot of great fixes and improvements on the >>> >>> master >>> >>> line >>> >>> now >>> >>> and we're at a good time window to start pushing for a >>> >>> release. >>> >>> There >>> >>> are, however, about 90+ JIRAs assigned to 1.1.0 which >>> >>> are >>> >>> open. >>> >>> I'm >>> >>> going to go through them and remove fix versions where >>> >>> appropriate. >>> >>> >>> I'm happy to take on RM task for this release though if >>> >>> someone >>> >>> else >>> >>> would like to take that on please advise. >>> >>> Thanks >>> Joe >>> >>> >>> >>> >>> >>> -- >>> Cheers, >>> >>> Edgardo >>> >>> >>> >>> >>> >>> -- >>> Cheers, >>> >>> Edgardo >>> >>> >>> >>> >>> >>> -- >>> Cheers, >>> >>> Edgardo >>> >>> >>> >>> >>> -- >>> Cheers, >>> >>> Edgardo >>> >>> Sent from Gmail Mobile >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >> >> >> -- >> I know what it is to be in need, and I know what it is to have plenty. I >> have learned the secret of being content in any and every situation, >> whether well fed or hungry, whether living in plenty or in want. I can do >> all this through him who gives me strength. *-Philippians 4:12-13*
