thanks Joe and Andy. Have those on the list now.
On Tue, Nov 22, 2016 at 5:09 AM, Joe Gresock <jgres...@gmail.com> wrote: > I'm biased, but I think the new Query / Scroll Elasticsearch processors are > pretty big ;) > > On Tue, Nov 22, 2016 at 9:02 AM, Andy LoPresto <alopre...@apache.org> wrote: > >> Important to call out in the security section that >> login-identity-providers.xml is now covered by encrypted config, and >> flow.xml.gz processor property encryption/nifi.sensitive.props.key value >> can now be migrated by the command-line tool (second part pending >> completion of NIFI-3024). >> >> Andy LoPresto >> alopre...@apache.org >> *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >> On Nov 21, 2016, at 9:30 PM, Joe Witt <joe.w...@gmail.com> wrote: >> >> matt >> >> i'll add a wiki page or set of instructions linked from the release notes. >> >> all, >> >> walked through the 250 or so JIRAs in the 1.1.0 release and pulled out >> highlights. The items noted are as follows. Will likely reduce this >> down further for the release notes but wanted to put this out in case >> folks have things they think are really important to highlight. >> >> - Core Improvements: >> - Performance: Session Migration >> - Stability: Cluster Management >> - Developer: Framework supports easy user driven classloader extension >> - Expression Language: Now supports base64 and hex encoded values >> and Math functions >> - Repositories now support rollback >> - Faster startup due to more efficient state restoration algorithm >> - UX Improvements: >> - Visual Backpressure Indicator >> - Introduced more colors to better highlight actions and components >> - Performance: Validate non-running components >> - Provenance graph image can be exported >> - Cron Scheduling for Primary node tasks now supported >> - Updated versions >> - Azure Event Hub 0.9.0 >> - Spark 2.0.1 >> - Hadoop 2.7.x >> - New/Improved Processors >> - new Fetch/Put Elastic Search 5.0 >> - new ParseCEF to parse CEF formatted logs >> - improve ExtractEmail now supports TNEF files >> - new Validate CSV >> - improved Solr processors now support SSL and Kerberos >> - new Websocket client and server processors >> - New Utility >> - Zookeeper Migrator (move from one zookeeper to another) >> - Security >> - Restricted Processors >> - Site-to-site now supports port forwarding >> - Improved Policy Management UX >> - Migration Notes: >> - Restricted Processors >> - Twitter Processor Removed >> >> On Mon, Nov 21, 2016 at 10:17 PM, Matt Burgess <mattyb...@apache.org> >> wrote: >> >> Is there a good spot for us to put instructions on how to build the >> Twitter processor and/or the Social Media NAR in the meantime? Maybe a >> Wiki page or something simple to say "go to this directory, run this >> Maven command, drop the NAR into your deployment..." ? >> >> On Mon, Nov 21, 2016 at 9:34 PM, Joe Witt <joe.w...@gmail.com> wrote: >> >> Team, >> >> We appear to be very close. Andy is working NIFI-3024 but otherwise >> it is focus on testing. >> >> I'm going to prep the RC and release notes now. Unfortunately the >> twitter changes for json.org will need to remain. Consensus forming >> on the legal-discuss thread regarding a grace period has been elusive >> and we're already prepared to make the right steps so we'll just need >> to take that on by being empathetic to the user base. >> >> Thanks >> Joe >> >> On Thu, Nov 17, 2016 at 7:37 AM, Andre <andre-li...@fucs.org> wrote: >> >> Andy, >> >> Great to see NIFI-3050 implemented and certainly good news that NiFi 1.1.0 >> is set to include a number of security related improvements. >> >> >> >> On Thu, Nov 17, 2016 at 2:38 PM, Andy LoPresto <alopre...@apache.org> >> wrote: >> >> Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have been >> added to my plate for this release. Coordinated with Joe Witt and they >> should both be included. >> >> [1] https://issues.apache.org/jira/browse/NIFI-3050 >> [2] https://issues.apache.org/jira/browse/NIFI-3051 >> >> Andy LoPresto >> alopre...@apache.org >> *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >> On Nov 16, 2016, at 12:08 PM, Joe Witt <joe.w...@gmail.com> wrote: >> >> Team >> >> There is a thread on apache legal-discuss that might allow for a >> graceperiod of continued usage of the json library. Am going to keep >> a close eye on this and if VP Legal approves we'll be able to keep the >> twitter processors in which is definitely a good thing. Will advise >> >> Thanks >> Joe >> >> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <bbe...@gmail.com> wrote: >> >> I've noticed an issue with the per-instance class loading capability >> introduced in NIFI-2909 where the additional classpath resources can get >> incorrectly removed from the class loader. >> >> I was able to reproduced this with a unit test and have a fix ready. I >> believe this is important and needs to go in for the 1.1 release, going to >> re-open NIFI-2909 and submit a PR shortly. >> >> -Bryan >> >> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <matt.c.gil...@gmail.com> >> wrote: >> >> I have two items that I would like to wrap up prior to creating an RC for >> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group port >> configuration. The work is already completed and I will be reviewing it >> this today. Additionally, following recent interest on the mailing list, >> I'd like to knock out NIFI-3020. This will allow an admin to configure a >> strategy for user identity when logging in via LDAP. Specifically, it will >> support usage of the DN (the default and current implementation) as well as >> the username the user logged in as. I should be able to have a PR up for >> this work later today. >> >> Thanks! >> >> Matt >> >> [1] https://issues.apache.org/jira/browse/NIFI-2949 >> [2] https://issues.apache.org/jira/browse/NIFI-3020 >> >> >> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <joe.w...@gmail.com> wrote: >> >> The code is within the twitter4j library itself. I filed a request to >> twitter4jg. The most likely case is we will need to submit a PR to them. >> However, I don't see this as something that should delay the release. We >> can provide instructions for folks wanting to use the processor during >> >> the >> >> time we cannot make it available in a convenient manner. I will provide >> >> a >> >> meaningful comment about this in release notes and pointers on what folks >> can do in the meantime. >> >> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <alopre...@apache.org> wrote: >> >> I understand there was a discussion thread within the NiFi community >> >> for >> >> this as well and I missed responding to that at that time. It just >> >> seems >> >> to >> >> me like JSON processing is necessary for GetTwitter, which is >> >> incredibly >> >> useful for demonstrating NiFi’s ability to read from a high volume >> >> stream >> >> out of the box. With NIFI-3019 (Remove GetTwitter from default build), >> >> is >> >> there any related effort to substitute an acceptable replacement JSON >> library to restore this functionality? >> >> [1] https://issues.apache.org/jira/browse/NIFI-3019 >> >> Andy LoPresto >> alopre...@apache.org >> *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <alopre...@apache.org> >> >> wrote: >> >> >> I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655, and >> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we >> investigated some alternate TLS config options for the new version of >> >> the >> >> client library. >> >> Is there any alternative to excluding the GetTwitter processor? Using >> Johnzon [1] or the Android re-implementation [2] discussed in the >> >> mailing >> >> list thread? >> >> [1] https://johnzon.apache.org/ >> [2] https://developer.android.com/reference/org/json/package- >> >> summary.html >> >> >> >> Andy LoPresto >> alopre...@apache.org >> *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >> On Nov 15, 2016, at 3:58 PM, Joe Witt <joe.w...@gmail.com> wrote: >> >> Team >> >> Very happy to see that we are down to three items remaining tagged to >> 1.1.0. Solid effort over the recent weeks to close the gap including >> >> work >> >> to get past the now category x Jason dependency we had. The most >> >> notable >> >> impact from that is the wildly popular GetTwitter processor, the fav >> >> new >> >> nifi user and demo processor, can no longer be included in the default >> build. It is optionally available if users choose to build and use it >> >> but >> >> we won't distribute binaries that have it. >> >> I see some review movement on some patch available but untagged items. >> >> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri. Anyone >> have any outstanding items? >> >> Thanks >> Joe >> >> On Nov 8, 2016 2:12 PM, "Joe Witt" <joe.w...@gmail.com> wrote: >> >> Ryan >> >> Not officially but I think we should try to close this thing out and >> start a vote in the next week or two at most. >> >> I'm going through the tickets again now. There is also a new issue of >> the json-p license falling out of favor in Apache legal terms and >> becoming Category-X. Am looking into that now. >> >> Thanks >> Joe >> >> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <ryan.wa...@gmail.com> >> >> wrote: >> >> >> Joe - Is there a target date for 1.1? >> >> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <joe.w...@gmail.com> wrote: >> >> Team, >> >> Just an update on things with working toward an Apache NiFi 1.1.0 >> release. There are still about 33 JIRAs there now and some are >> awaiting review and are some are under active progress. Yet there is >> good traction and progress. I think we should just stay vigilant with >> what makes it in and keep working it down. So let's please shoot for >> a couple weeks from now. If it is ready sooner I'll jump on it. >> >> Thanks >> Joe >> >> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <joe.w...@gmail.com> wrote: >> >> Team, >> >> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0. Let's >> avoiding putting more in there for now at least without a discussion. >> Of the 31 JIRAs there the vast majority need review so we should be >> able to close these down fairly quickly as long as we don't let the >> list grow. >> >> Thanks >> joe >> >> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <edgardo.v...@gmail.com> >> >> wrote: >> >> Joe, >> >> Appreciate the offer it isn't my PR. I was just using it as an >> >> example. >> >> All >> >> mine are currently closed, which I greatly appreciate. >> >> Cheers, >> >> Edgardo >> >> On Friday, October 14, 2016, Joe Witt <joe.w...@gmail.com> wrote: >> >> Edgardo, >> >> You mentioned a PR from August. I'd be happy to help you work that >> through review. >> >> Thanks >> Joe >> >> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega < >> >> edgardo.v...@gmail.com >> >> <javascript:;>> wrote: >> >> I have agreed that at this point a release is important. My goal >> >> was >> >> try >> >> to >> >> squeeze in a much goodness as possible into the release, but the >> >> important >> >> bug fixes should come first. Getting 1.x into a state where the >> >> release >> >> notes don't say that it is geared toward developers and testers is >> >> really >> >> huge. >> >> I think Nifi is a great community otherwise I would participate in >> >> the >> >> mailing list, create Jira tickets and pull requests. I am only >> >> trying to >> >> strengthen the great thing that is going on here. We can always do >> >> better. >> >> I was not trying to put down this community only to participate and >> >> make >> >> it >> >> better. I think this conversation is an indication of how great >> >> this >> >> community is. >> >> Maybe I am being sensitive about this issue and trying to >> >> strengthen >> >> the >> >> nifi community even more, after coming from a conference where it >> >> was >> >> reported there was lots of excitement at first and now the >> >> participation >> >> in >> >> the community has really died down and they are struggling. I don't >> >> want >> >> to >> >> see that happen here. >> >> Cheers, >> >> Edgardo >> >> >> >> >> On Fri, Oct 14, 2016 at 9:37 AM, Andre <andre-li...@fucs.org >> >> <javascript:;>> wrote: >> >> >> Edgardo, >> >> Thank you for your feedback. We hear your comments and as a >> >> committer I >> >> can >> >> share we are constantly looking to improve the PR process, having >> >> already >> >> taken many of the steps you suggest. >> >> However, it is important to notice that the number of PRs should >> >> not be >> >> seen as a metric of engagement by the development community: Most >> >> of us >> >> will submit PRs so that our work can be carefully reviewed by our >> >> peers >> >> and >> >> some of us will use JIRA patches to provide contributions. >> >> Having said that, it is true that some PRs may sit idle for a long >> >> time >> >> and >> >> we are working to improve this pipeline. >> >> It was therefore no coincidence that I browsed most of the PRs >> >> performing >> >> a triage of items that have been superseded or diverged from the >> >> current >> >> code base. >> >> In fact, less than a month ago the dev team closed a number of >> >> stalled >> >> and >> >> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8). >> >> Despite all the above, I think Joe has a point. The master >> >> contain a >> >> series >> >> of important bug fixes and suspect the community would benefit >> >> from >> >> a >> >> release sooner rather than later. >> >> Once again, thank you for your feedback and contribution. It is >> >> good to >> >> have you here. >> >> Andre >> >> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega < >> >> edgardo.v...@gmail.com >> >> <javascript:;>> >> >> wrote: >> >> Joe - You are correct I was mentioning the PRs that are >> >> currently >> >> open. >> >> >> Regardless of how it happens reducing the count of open PRs I >> >> believe >> >> to >> >> be >> >> extremely important. Maybe I was hoping that the release could >> >> be >> >> a >> >> forcing >> >> function to make that happen. I believe that developers are more >> >> willing >> >> to >> >> contribute when they see that their PRs will actually be able >> >> accepted >> >> and >> >> merged into the code base. Having a low number of open PRs in >> >> progress >> >> is a >> >> great indication that the main nifi developers are fully engaged >> >> with >> >> the >> >> community. >> >> There are a few PRs that don't have any comments from committers >> >> at >> >> all. >> >> I >> >> found one from August in that state. If that was my PR I don't >> >> think I >> >> would be so willing to put another one in anytime soon. I do get >> >> that >> >> sometime PRs get stalled by the originator, if so maybe a rule >> >> about >> >> closing them after a certain amount of time or being taken over >> >> by a >> >> core >> >> contributor if they think it worthwhile. >> >> I would like to shoutout to James Wing on my last PR he was >> >> quick >> >> to >> >> review, provided great comments, testing, and even some >> >> additional >> >> code. >> >> It >> >> was a great PR experience. >> >> Cheers, >> >> Edgardo >> >> >> >> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall < >> >> joeperciv...@yahoo.com <javascript:;>. >> >> invalid> wrote: >> >> Joe, I think you misread. Edgardo is referring to the Pull >> >> Requests >> >> that >> >> are currently open, not the tickets assigned to the 1.1.0 >> >> version. >> >> >> I think these goals (releasing 1.1.0 and cutting down the PR >> >> count) >> >> should >> >> be two different efforts. Doing a thorough job reviewing >> >> takes a >> >> significant amount of time from both the reviewer and >> >> contributor. >> >> In >> >> order >> >> to cut it down significantly would take much longer than a >> >> couple >> >> days. >> >> >> Also there has already been a lot of great new features and >> >> bug >> >> fixes >> >> contributed to the 1.X line and I don't think it's worth >> >> holding up >> >> a >> >> 1.1.0 >> >> release for tickets not assigned to this fix version. As an >> >> added >> >> bonus >> >> though, I think many of the tickets tagged as 1.1.0 have PRs >> >> already >> >> open >> >> so closing those will make a large dent in the PR count. >> >> >> Joe >> >> - - - - - - >> Joseph Percivall >> linkedin.com/in/Percivall >> e: joeperciv...@yahoo.com <javascript:;> >> >> >> >> On Thursday, October 13, 2016 3:58 PM, Joe Witt < >> >> joe.w...@gmail.com >> >> <javascript:;>> >> >> wrote: >> >> >> >> There are less than 30 right now. Many of the roughly 90+ >> >> JIRAs >> >> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed >> >> or >> >> just >> >> had fix versions removed. >> >> We will need to have a push over the next bunch of days to >> >> deal >> >> with >> >> reviewing/merging/moving the remaining items. >> >> Thanks >> Joe >> >> >> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega < >> >> edgardo.v...@gmail.com <javascript:;>> >> >> wrote: >> >> Joe, >> >> There are 75 PRs currently open. Why not make a push over >> >> the >> >> next >> >> bunch >> >> of >> >> days to get them closed and then cut the release after that. >> >> Cheers, >> >> Edgardo >> >> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt < >> >> joe.w...@gmail.com >> >> <javascript:;>> >> >> wrote: >> >> >> Team, >> >> There have been a ton of bugs fixed a few nice features. I >> >> would >> >> like >> >> to move to get Apache NiFi 1.1.0 release going pretty much >> >> based >> >> on >> >> where we are now and plan to move most tickets to a new >> >> Apache >> >> NiFi >> >> 1.2.0 version. We can try to get back on our roughly 6-8 >> >> week >> >> release >> >> schedule and shoot for a mid to late Nov release for NiFi >> >> 1.2.0 >> >> this >> >> way as well. Please advise if anyone has any other views on >> >> this. In >> >> the mean time I'll get the wheels in motion so you'll be >> >> seeing a >> >> lot >> >> of JIRA/issue updates to move version around. >> >> Thanks >> Joe >> >> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc < >> >> trk...@gmail.com >> >> <javascript:;>> >> >> wrote: >> >> Sounds good Joe. I have no issue to you doing the rm'ing >> >> for >> >> it. >> >> >> On Oct 13, 2016 8:19 AM, "Joe Witt" <joe.w...@gmail.com >> >> <javascript:;>> wrote: >> >> >> Team, >> >> There are a lot of great fixes and improvements on the >> >> master >> >> line >> >> now >> >> and we're at a good time window to start pushing for a >> >> release. >> >> There >> >> are, however, about 90+ JIRAs assigned to 1.1.0 which >> >> are >> >> open. >> >> I'm >> >> going to go through them and remove fix versions where >> >> appropriate. >> >> >> I'm happy to take on RM task for this release though if >> >> someone >> >> else >> >> would like to take that on please advise. >> >> Thanks >> Joe >> >> >> >> >> >> -- >> Cheers, >> >> Edgardo >> >> >> >> >> >> -- >> Cheers, >> >> Edgardo >> >> >> >> >> >> -- >> Cheers, >> >> Edgardo >> >> >> >> >> -- >> Cheers, >> >> Edgardo >> >> Sent from Gmail Mobile >> >> >> >> >> >> >> >> >> >> >> > > > -- > I know what it is to be in need, and I know what it is to have plenty. I > have learned the secret of being content in any and every situation, > whether well fed or hungry, whether living in plenty or in want. I can do > all this through him who gives me strength. *-Philippians 4:12-13*
