Re: How to dynamically update URL in GetHTTP processor ?

[Brandon DeVries]

Shweta,

 Take a look at InvokeHTTP[1] instead of GetHTTP.  InvokeHTTP allows
Expression Language in the URL, so you can specify the page number.  Let us
know if you have any other questions.  Thanks.

[1]
http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.InvokeHTTP/index.html

Brandon

On Thu, Dec 17, 2015 at 4:13 AM, shweta  wrote:

> Hi All,
>
> I have a requirement wherein I have to update  getHTTP URL dynamically.
> Within that url I have
> a parameter called page whose value can vary from 1 to N number depending
> upon no. of pages present.
>
> My requirement is as such that  "nextInt()" would not be useful. I want to
> define my own parameter like page.counter and increment it by 1 based on
> some condition. For example in following URL page=1
>
>
> https://api.stackexchange.com/2.2/posts?*page=1*=1450224000=1450310400=desc=activity=stackoverflow
>
> I want to dynamically update page no. and fetch the result.
>
> Any pointers would be useful.
>
> Thanks,
> Shweta
>
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/How-to-dynamically-update-URL-in-GetHTTP-processor-tp5826.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>

Re: JSON / Avro issues

[Tony Kurc]

Ian,
Excellent catch, I was referring to the ConvertAvroToJSON processor, which
can EMIT json with either representation, which is obvious in retrospect
*not* what was being asked:

http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.avro.ConvertAvroToJSON/index.html

On Thu, Dec 17, 2015 at 11:25 AM, ianwork  wrote:

> trkurc, Am i missing something, I do not see the functionality to toggle
> between the two json representations in the latest build of jsontoavro
> processor?
>
> Ian
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/JSON-Avro-issues-tp3923p5828.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>

[GitHub] nifi pull request: NIFI-1300 - Penalize flowfiles when message sen...

[jskora]

GitHub user jskora opened a pull request:

https://github.com/apache/nifi/pull/145

NIFI-1300 - Penalize flowfiles when message send or commit exceptions occur 
and m…

…ake commit exception handler route to failure instead of rolling back.  
Moved producer queue creation into a method to support testing.  Created 
comprehensive PutJMS test suite.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jskora/nifi NIFI-1300

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/145.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #145


commit 218e5fdcd5d4dd2513c42f699cbf7b19cc7710a2
Author: Joe Skora 
Date:   2015-12-17T17:27:56Z

Penalize flowfiles when message send or commit exceptions occur and make 
commit exception handler route to failure instead of rolling back.  Moved 
producer queue creation into a method to support testing.  Created 
comprehensive PutJMS test suite.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: JSON / Avro issues

[ianwork]

trkurc, Am i missing something, I do not see the functionality to toggle
between the two json representations in the latest build of jsontoavro
processor?

Ian



--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/JSON-Avro-issues-tp3923p5828.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: JSON / Avro issues

[Ryan Blue]

Jeff, I've answered inline. Thanks for using the processor, sorry it 
isn't clear what's happening.


rb

On 11/05/2015 01:59 PM, Jeff wrote:

I built a simple flow that reads a tab separated file and attempts to convert 
to Avro.

ConvertCSVtoAvro just says that the conversion failed.

Where can I find more information on what the failure was?


Information about failures is added to the "errors" attribute on files 
emitted to the failure relationship. Unfortunately, right now the files 
aren't filtered to just the failed rows. That's something we need to 
fix, but it does accumulate error messages so you get something like:


  "NumberFormatException: 'turkey' is not an integer (1,234 similar 
errors)"



Using the same sample tab separated file, I create a JSON file out of it.

The JSON to Avro processor also fails with very little explication.


These processors are basically the same on the inside. :)

Same place for errors. I think the problem is likely that some of the 
values are failing to convert to the Avro type you've selected.




With regard to the ConvertCSVtoAvro processor
Since my file is tab  delimited, do I simple open the "CSV delimiter” 
property, delete , and hit the tab key or is there a special syntax like ^t?
My data has no CSV quote character so do I leave this as “or delete it 
or check the empty box?


This could definitely be a problem. The delimiter is what you want. It 
works with both a tab character (I usually paste it in since the browser 
uses it as a movement key) and with \t, though I think there's a bug 
where you can't have 2-character delimiters in the validation. I should 
fix that.



With regard to the ConvertJSONtoAvro
What is the expected JSON source file to look like?
[
 {fields values … },
 {fields values …}
]
Or
 {fields values … }
 {fields values …}
or something else.


This should be the second case. the JSON to Avro processor can't handle 
JSON lists as the root just yet. You should simply concatenate JSON. The 
whitespace doesn't matter.


rb


--
Ryan Blue
Software Engineer
Cloudera, Inc.

Re: discuss nifi 0.4.1

[Sean Busbey]

Sounds reasonable to me. branching off of the last release tag and then
cherry picking a conservative subset of fixes for a patch release has
worked well for me on another project.

It's implied in your email, but just to confirm, you're only suggesting
grabbing *some* of the currently-in-0.5.0 issues right? specifically just
those that you added a 0.4.1 fixversion to?


On Thu, Dec 17, 2015 at 2:29 PM, Joe Witt  wrote:

> team,
>
> matt clarke just discovered an interesting case that appears to expose
> a defect in site-to-site.  The details of it are still being worked
> out as you can see in NIFI-1301.  And this issue has been around for a
> very long time but it still feels like something worth addressing in
> an incremental/bug release (0.4.1).
>
> I looked at already addressed bugs on 050 and added the to fix
> versions of 041 as well.  What I am wondering here is a bit of a
> proper usage and thinking with Git.  Would it make sense to branch off
> master right where 0.4.1-SNAPSHOT started, then cherry pick the
> commits into this new branch, and just release that branch never
> needing then to merge that back to master since these fixes are all
> already on master anyway?
>
>
> https://issues.apache.org/jira/browse/NIFI-1301?jql=project%20%3D%20NIFI%20AND%20fixVersion%20%3D%200.4.1%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
>
> Thanks
> Joe
>



-- 
Sean

Re: discuss nifi 0.4.1

[Matt Gilman]

Are there some commits on master that we don't want included in 0.4.1? If
not, wouldn't we just follow our normal release process?

Matt

On Thu, Dec 17, 2015 at 6:15 PM, Ryan Blue  wrote:

> Branching from master at the start of 0.4.1-SNAPSHOT and cherry-picking
> makes sense to me.
>
> rb
>
>
> On 12/17/2015 12:29 PM, Joe Witt wrote:
>
>> team,
>>
>> matt clarke just discovered an interesting case that appears to expose
>> a defect in site-to-site.  The details of it are still being worked
>> out as you can see in NIFI-1301.  And this issue has been around for a
>> very long time but it still feels like something worth addressing in
>> an incremental/bug release (0.4.1).
>>
>> I looked at already addressed bugs on 050 and added the to fix
>> versions of 041 as well.  What I am wondering here is a bit of a
>> proper usage and thinking with Git.  Would it make sense to branch off
>> master right where 0.4.1-SNAPSHOT started, then cherry pick the
>> commits into this new branch, and just release that branch never
>> needing then to merge that back to master since these fixes are all
>> already on master anyway?
>>
>>
>> https://issues.apache.org/jira/browse/NIFI-1301?jql=project%20%3D%20NIFI%20AND%20fixVersion%20%3D%200.4.1%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
>>
>> Thanks
>> Joe
>>
>>
>
> --
> Ryan Blue
> Software Engineer
> Cloudera, Inc.
>

Re: discuss nifi 0.4.1

[Sean Busbey]

I wouldn't want to see the kafka client upgrade in a patch release.

On Thu, Dec 17, 2015 at 5:37 PM, Joe Witt  wrote:

> OK thanks for confirming proper git fu.
>
> Yeah I was meaning to just grab bugs.  The master branch already has stuff
> that seems to warrant a minor bump (maybe) so wanted to understand a bug
> only route.
>
> Matt understand your point on dependent commits.  Will check that out.
>
> Thanks
> Joe
> On Dec 17, 2015 6:32 PM, "Matt Gilman"  wrote:
>
> > I see, that does appear to be the case. What your suggesting sounds good.
> > Though we should review the tickets that addressed UI bugs/improvements.
> I
> > realize you probably specifically just chose the JIRAs that addressed
> bugs.
> > I'd want to make sure that the tickets included don't have a dependency
> on
> > the tickets excluded. For instance, merge conflicts because a commit in
> an
> > included ticket is based off the code base after a commit in an excluded
> > ticket.
> >
> > Matt
> >
> > On Thu, Dec 17, 2015 at 6:17 PM, Matt Gilman 
> > wrote:
> >
> > > Are there some commits on master that we don't want included in 0.4.1?
> If
> > > not, wouldn't we just follow our normal release process?
> > >
> > > Matt
> > >
> > >
> > > On Thu, Dec 17, 2015 at 6:15 PM, Ryan Blue  wrote:
> > >
> > >> Branching from master at the start of 0.4.1-SNAPSHOT and
> cherry-picking
> > >> makes sense to me.
> > >>
> > >> rb
> > >>
> > >>
> > >> On 12/17/2015 12:29 PM, Joe Witt wrote:
> > >>
> > >>> team,
> > >>>
> > >>> matt clarke just discovered an interesting case that appears to
> expose
> > >>> a defect in site-to-site.  The details of it are still being worked
> > >>> out as you can see in NIFI-1301.  And this issue has been around for
> a
> > >>> very long time but it still feels like something worth addressing in
> > >>> an incremental/bug release (0.4.1).
> > >>>
> > >>> I looked at already addressed bugs on 050 and added the to fix
> > >>> versions of 041 as well.  What I am wondering here is a bit of a
> > >>> proper usage and thinking with Git.  Would it make sense to branch
> off
> > >>> master right where 0.4.1-SNAPSHOT started, then cherry pick the
> > >>> commits into this new branch, and just release that branch never
> > >>> needing then to merge that back to master since these fixes are all
> > >>> already on master anyway?
> > >>>
> > >>>
> > >>>
> >
> https://issues.apache.org/jira/browse/NIFI-1301?jql=project%20%3D%20NIFI%20AND%20fixVersion%20%3D%200.4.1%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
> > >>>
> > >>> Thanks
> > >>> Joe
> > >>>
> > >>>
> > >>
> > >> --
> > >> Ryan Blue
> > >> Software Engineer
> > >> Cloudera, Inc.
> > >>
> > >
> > >
> >
>



-- 
Sean

Re: discuss nifi 0.4.1

[Oleg Zhurakousky]

I think we want to exclude new features and make it a true maintenance release, 
so only bugs should go into 0.4.1   

> On Dec 17, 2015, at 6:17 PM, Matt Gilman  wrote:
> 
> Are there some commits on master that we don't want included in 0.4.1? If
> not, wouldn't we just follow our normal release process?
> 
> Matt
> 
> On Thu, Dec 17, 2015 at 6:15 PM, Ryan Blue  wrote:
> 
>> Branching from master at the start of 0.4.1-SNAPSHOT and cherry-picking
>> makes sense to me.
>> 
>> rb
>> 
>> 
>> On 12/17/2015 12:29 PM, Joe Witt wrote:
>> 
>>> team,
>>> 
>>> matt clarke just discovered an interesting case that appears to expose
>>> a defect in site-to-site.  The details of it are still being worked
>>> out as you can see in NIFI-1301.  And this issue has been around for a
>>> very long time but it still feels like something worth addressing in
>>> an incremental/bug release (0.4.1).
>>> 
>>> I looked at already addressed bugs on 050 and added the to fix
>>> versions of 041 as well.  What I am wondering here is a bit of a
>>> proper usage and thinking with Git.  Would it make sense to branch off
>>> master right where 0.4.1-SNAPSHOT started, then cherry pick the
>>> commits into this new branch, and just release that branch never
>>> needing then to merge that back to master since these fixes are all
>>> already on master anyway?
>>> 
>>> 
>>> https://issues.apache.org/jira/browse/NIFI-1301?jql=project%20%3D%20NIFI%20AND%20fixVersion%20%3D%200.4.1%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
>>> 
>>> Thanks
>>> Joe
>>> 
>>> 
>> 
>> --
>> Ryan Blue
>> Software Engineer
>> Cloudera, Inc.
>> 

Re: discuss nifi 0.4.1

[Tony Kurc]

s/features/buxfixes/

On Thu, Dec 17, 2015 at 6:50 PM, Tony Kurc  wrote:

> Is there a reason to not just cut a 0.5.0 instead of grafting 0.5.0
> features onto 0.4.1?
>
> On Thu, Dec 17, 2015 at 6:38 PM, Oleg Zhurakousky <
> ozhurakou...@hortonworks.com> wrote:
>
>> I think we want to exclude new features and make it a true maintenance
>> release, so only bugs should go into 0.4.1
>>
>> > On Dec 17, 2015, at 6:17 PM, Matt Gilman 
>> wrote:
>> >
>> > Are there some commits on master that we don't want included in 0.4.1?
>> If
>> > not, wouldn't we just follow our normal release process?
>> >
>> > Matt
>> >
>> > On Thu, Dec 17, 2015 at 6:15 PM, Ryan Blue  wrote:
>> >
>> >> Branching from master at the start of 0.4.1-SNAPSHOT and cherry-picking
>> >> makes sense to me.
>> >>
>> >> rb
>> >>
>> >>
>> >> On 12/17/2015 12:29 PM, Joe Witt wrote:
>> >>
>> >>> team,
>> >>>
>> >>> matt clarke just discovered an interesting case that appears to expose
>> >>> a defect in site-to-site.  The details of it are still being worked
>> >>> out as you can see in NIFI-1301.  And this issue has been around for a
>> >>> very long time but it still feels like something worth addressing in
>> >>> an incremental/bug release (0.4.1).
>> >>>
>> >>> I looked at already addressed bugs on 050 and added the to fix
>> >>> versions of 041 as well.  What I am wondering here is a bit of a
>> >>> proper usage and thinking with Git.  Would it make sense to branch off
>> >>> master right where 0.4.1-SNAPSHOT started, then cherry pick the
>> >>> commits into this new branch, and just release that branch never
>> >>> needing then to merge that back to master since these fixes are all
>> >>> already on master anyway?
>> >>>
>> >>>
>> >>>
>> https://issues.apache.org/jira/browse/NIFI-1301?jql=project%20%3D%20NIFI%20AND%20fixVersion%20%3D%200.4.1%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
>> >>>
>> >>> Thanks
>> >>> Joe
>> >>>
>> >>>
>> >>
>> >> --
>> >> Ryan Blue
>> >> Software Engineer
>> >> Cloudera, Inc.
>> >>
>>
>>
>

Re: discuss nifi 0.4.1

[Tony Kurc]

Is there a reason to not just cut a 0.5.0 instead of grafting 0.5.0
features onto 0.4.1?

On Thu, Dec 17, 2015 at 6:38 PM, Oleg Zhurakousky <
ozhurakou...@hortonworks.com> wrote:

> I think we want to exclude new features and make it a true maintenance
> release, so only bugs should go into 0.4.1
>
> > On Dec 17, 2015, at 6:17 PM, Matt Gilman 
> wrote:
> >
> > Are there some commits on master that we don't want included in 0.4.1? If
> > not, wouldn't we just follow our normal release process?
> >
> > Matt
> >
> > On Thu, Dec 17, 2015 at 6:15 PM, Ryan Blue  wrote:
> >
> >> Branching from master at the start of 0.4.1-SNAPSHOT and cherry-picking
> >> makes sense to me.
> >>
> >> rb
> >>
> >>
> >> On 12/17/2015 12:29 PM, Joe Witt wrote:
> >>
> >>> team,
> >>>
> >>> matt clarke just discovered an interesting case that appears to expose
> >>> a defect in site-to-site.  The details of it are still being worked
> >>> out as you can see in NIFI-1301.  And this issue has been around for a
> >>> very long time but it still feels like something worth addressing in
> >>> an incremental/bug release (0.4.1).
> >>>
> >>> I looked at already addressed bugs on 050 and added the to fix
> >>> versions of 041 as well.  What I am wondering here is a bit of a
> >>> proper usage and thinking with Git.  Would it make sense to branch off
> >>> master right where 0.4.1-SNAPSHOT started, then cherry pick the
> >>> commits into this new branch, and just release that branch never
> >>> needing then to merge that back to master since these fixes are all
> >>> already on master anyway?
> >>>
> >>>
> >>>
> https://issues.apache.org/jira/browse/NIFI-1301?jql=project%20%3D%20NIFI%20AND%20fixVersion%20%3D%200.4.1%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
> >>>
> >>> Thanks
> >>> Joe
> >>>
> >>>
> >>
> >> --
> >> Ryan Blue
> >> Software Engineer
> >> Cloudera, Inc.
> >>
>
>

Re: discuss nifi 0.4.1

[Ryan Blue]

Branching from master at the start of 0.4.1-SNAPSHOT and cherry-picking 
makes sense to me.


rb

On 12/17/2015 12:29 PM, Joe Witt wrote:

team,

matt clarke just discovered an interesting case that appears to expose
a defect in site-to-site.  The details of it are still being worked
out as you can see in NIFI-1301.  And this issue has been around for a
very long time but it still feels like something worth addressing in
an incremental/bug release (0.4.1).

I looked at already addressed bugs on 050 and added the to fix
versions of 041 as well.  What I am wondering here is a bit of a
proper usage and thinking with Git.  Would it make sense to branch off
master right where 0.4.1-SNAPSHOT started, then cherry pick the
commits into this new branch, and just release that branch never
needing then to merge that back to master since these fixes are all
already on master anyway?

https://issues.apache.org/jira/browse/NIFI-1301?jql=project%20%3D%20NIFI%20AND%20fixVersion%20%3D%200.4.1%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC

Thanks
Joe




--
Ryan Blue
Software Engineer
Cloudera, Inc.

Re: discuss nifi 0.4.1

[Matt Gilman]

I see, that does appear to be the case. What your suggesting sounds good.
Though we should review the tickets that addressed UI bugs/improvements. I
realize you probably specifically just chose the JIRAs that addressed bugs.
I'd want to make sure that the tickets included don't have a dependency on
the tickets excluded. For instance, merge conflicts because a commit in an
included ticket is based off the code base after a commit in an excluded
ticket.

Matt

On Thu, Dec 17, 2015 at 6:17 PM, Matt Gilman 
wrote:

> Are there some commits on master that we don't want included in 0.4.1? If
> not, wouldn't we just follow our normal release process?
>
> Matt
>
>
> On Thu, Dec 17, 2015 at 6:15 PM, Ryan Blue  wrote:
>
>> Branching from master at the start of 0.4.1-SNAPSHOT and cherry-picking
>> makes sense to me.
>>
>> rb
>>
>>
>> On 12/17/2015 12:29 PM, Joe Witt wrote:
>>
>>> team,
>>>
>>> matt clarke just discovered an interesting case that appears to expose
>>> a defect in site-to-site.  The details of it are still being worked
>>> out as you can see in NIFI-1301.  And this issue has been around for a
>>> very long time but it still feels like something worth addressing in
>>> an incremental/bug release (0.4.1).
>>>
>>> I looked at already addressed bugs on 050 and added the to fix
>>> versions of 041 as well.  What I am wondering here is a bit of a
>>> proper usage and thinking with Git.  Would it make sense to branch off
>>> master right where 0.4.1-SNAPSHOT started, then cherry pick the
>>> commits into this new branch, and just release that branch never
>>> needing then to merge that back to master since these fixes are all
>>> already on master anyway?
>>>
>>>
>>> https://issues.apache.org/jira/browse/NIFI-1301?jql=project%20%3D%20NIFI%20AND%20fixVersion%20%3D%200.4.1%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
>>>
>>> Thanks
>>> Joe
>>>
>>>
>>
>> --
>> Ryan Blue
>> Software Engineer
>> Cloudera, Inc.
>>
>
>

Re: discuss nifi 0.4.1

[Sean Busbey]

On Thu, Dec 17, 2015 at 5:50 PM, Tony Kurc  wrote:

> s/features/buxfixes/
>
> On Thu, Dec 17, 2015 at 6:50 PM, Tony Kurc  wrote:
>
> > Is there a reason to not just cut a 0.5.0 instead of grafting 0.5.0
> > features onto 0.4.1?
> >
>

This is a good question.

Some downstream users might have different change processes for a patch vs
minor release, so making a 0.4.1 that fixes what we determine to be a
substantial gap in the 0.4 line would be nice for them.

While we might be a young project now, it would be good to already have the
habit practiced for when we have more users in enterprise settings.

On the other hand, 0.4.0 just happened, so a release in 3 days would
minimize the number of "stuck on 0.4.0" folks.

-- 
Sean

How to dynamically update URL in GetHTTP processor ?

[shweta]

Hi All,

I have a requirement wherein I have to update  getHTTP URL dynamically.
Within that url I have 
a parameter called page whose value can vary from 1 to N number depending
upon no. of pages present.

My requirement is as such that  "nextInt()" would not be useful. I want to
define my own parameter like page.counter and increment it by 1 based on
some condition. For example in following URL page=1

https://api.stackexchange.com/2.2/posts?*page=1*=1450224000=1450310400=desc=activity=stackoverflow

I want to dynamically update page no. and fetch the result.

Any pointers would be useful.

Thanks,
Shweta




--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/How-to-dynamically-update-URL-in-GetHTTP-processor-tp5826.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

discuss nifi 0.4.1

[Joe Witt]

team,

matt clarke just discovered an interesting case that appears to expose
a defect in site-to-site.  The details of it are still being worked
out as you can see in NIFI-1301.  And this issue has been around for a
very long time but it still feels like something worth addressing in
an incremental/bug release (0.4.1).

I looked at already addressed bugs on 050 and added the to fix
versions of 041 as well.  What I am wondering here is a bit of a
proper usage and thinking with Git.  Would it make sense to branch off
master right where 0.4.1-SNAPSHOT started, then cherry pick the
commits into this new branch, and just release that branch never
needing then to merge that back to master since these fixes are all
already on master anyway?

https://issues.apache.org/jira/browse/NIFI-1301?jql=project%20%3D%20NIFI%20AND%20fixVersion%20%3D%200.4.1%20ORDER%20BY%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC

Thanks
Joe

Re: discuss nifi 0.4.1

[Ryan Blue]

Another reason to release 0.4.1 is to allow the additions that warrant 
0.5.0 to have more validation before release. With a regular release 
cycle, features can go in at the beginning to have more time for 
catching bugs in them. I also agree with what Sean said below.


rb

On 12/17/2015 04:00 PM, Sean Busbey wrote:

On Thu, Dec 17, 2015 at 5:50 PM, Tony Kurc  wrote:


s/features/buxfixes/

On Thu, Dec 17, 2015 at 6:50 PM, Tony Kurc  wrote:


Is there a reason to not just cut a 0.5.0 instead of grafting 0.5.0
features onto 0.4.1?





This is a good question.

Some downstream users might have different change processes for a patch vs
minor release, so making a 0.4.1 that fixes what we determine to be a
substantial gap in the 0.4 line would be nice for them.

While we might be a young project now, it would be good to already have the
habit practiced for when we have more users in enterprise settings.

On the other hand, 0.4.0 just happened, so a release in 3 days would
minimize the number of "stuck on 0.4.0" folks.




--
Ryan Blue
Software Engineer
Cloudera, Inc.

Re: discuss nifi 0.4.1

[Tony Kurc]

I'm not sure I understand "more validation" reasoning - won't features at
the end have very little validation?

On Thu, Dec 17, 2015 at 7:26 PM, Ryan Blue  wrote:

> Another reason to release 0.4.1 is to allow the additions that warrant
> 0.5.0 to have more validation before release. With a regular release cycle,
> features can go in at the beginning to have more time for catching bugs in
> them. I also agree with what Sean said below.
>
> rb
>
> On 12/17/2015 04:00 PM, Sean Busbey wrote:
>
>> On Thu, Dec 17, 2015 at 5:50 PM, Tony Kurc  wrote:
>>
>> s/features/buxfixes/
>>>
>>> On Thu, Dec 17, 2015 at 6:50 PM, Tony Kurc  wrote:
>>>
>>> Is there a reason to not just cut a 0.5.0 instead of grafting 0.5.0
 features onto 0.4.1?


>>>
>> This is a good question.
>>
>> Some downstream users might have different change processes for a patch vs
>> minor release, so making a 0.4.1 that fixes what we determine to be a
>> substantial gap in the 0.4 line would be nice for them.
>>
>> While we might be a young project now, it would be good to already have
>> the
>> habit practiced for when we have more users in enterprise settings.
>>
>> On the other hand, 0.4.0 just happened, so a release in 3 days would
>> minimize the number of "stuck on 0.4.0" folks.
>>
>>
>
> --
> Ryan Blue
> Software Engineer
> Cloudera, Inc.
>

Re: discuss nifi 0.4.1

[Ryan Blue]

Yes, which affects when you time getting something into master. Larger 
features that are done just before a release (more risk) can get pushed 
so that they are committed after a release instead of just before one. 
Regular releases ensure the penalty for choosing to get into the next 
release aren't too high.


You could make the argument that master should always be in a releasable 
state, but I think that even when reviews are done right there is risk 
for some features. All I want to note is that a regular release cadence 
helps mitigate that risk when we stick to it.


rb

On 12/17/2015 04:32 PM, Tony Kurc wrote:

I'm not sure I understand "more validation" reasoning - won't features at
the end have very little validation?

On Thu, Dec 17, 2015 at 7:26 PM, Ryan Blue  wrote:


Another reason to release 0.4.1 is to allow the additions that warrant
0.5.0 to have more validation before release. With a regular release cycle,
features can go in at the beginning to have more time for catching bugs in
them. I also agree with what Sean said below.

rb

On 12/17/2015 04:00 PM, Sean Busbey wrote:


On Thu, Dec 17, 2015 at 5:50 PM, Tony Kurc  wrote:

s/features/buxfixes/


On Thu, Dec 17, 2015 at 6:50 PM, Tony Kurc  wrote:

Is there a reason to not just cut a 0.5.0 instead of grafting 0.5.0

features onto 0.4.1?





This is a good question.

Some downstream users might have different change processes for a patch vs
minor release, so making a 0.4.1 that fixes what we determine to be a
substantial gap in the 0.4 line would be nice for them.

While we might be a young project now, it would be good to already have
the
habit practiced for when we have more users in enterprise settings.

On the other hand, 0.4.0 just happened, so a release in 3 days would
minimize the number of "stuck on 0.4.0" folks.




--
Ryan Blue
Software Engineer
Cloudera, Inc.






--
Ryan Blue
Software Engineer
Cloudera, Inc.

Re: discuss nifi 0.4.1

[Sean Busbey]

On Thu, Dec 17, 2015 at 6:32 PM, Tony Kurc  wrote:

> I'm not sure I understand "more validation" reasoning - won't features at
> the end have very little validation?
>
> On Thu, Dec 17, 2015 at 7:26 PM, Ryan Blue  wrote:
>
> > Another reason to release 0.4.1 is to allow the additions that warrant
> > 0.5.0 to have more validation before release. With a regular release
> cycle,
> > features can go in at the beginning to have more time for catching bugs
> in
> > them. I also agree with what Sean said below.
> >
>

I presume Ryan just meant that the things that have gone in now would have
more time.

This is the current history since 0.4.0 branched for RC:

* 04e9606 - (HEAD, origin/master, origin/HEAD, master) NIFI-1290 Document th
* 7c87968 - NIFI-1218 addressed PR comments (29 hours ago)
* 3763523 - NIFI-1218 upgraded Kafka to 0.9.0.0 client API Tested and valida
* b19ff7c - NIFI-1215: - Only showing the run duration setting when applicab
* 51b8ecd - NIFI-1185: - Using banners from the NCM rather than replicating
* c75b5cf - NIFI-1119: - Addressing race condition that caused the revision
* 17be1c2 - NIFI-1206: - Only enabling the enable/disable toolbar icon when
* f9f0443 - NIFI-1119: - Also refreshing flow revision when the user clicks
*   a7b09a5 - NIFI-1122 release vote passess. Merge branch 'NIFI-1122_nifi-0
|\
| * d755e43 - (origin/NIFI-1122_nifi-0.4.0-RC2) NIFI-1122_nifi-0.4.0-RC2prep
| * b66c029 - (nifi-0.4.0-RC2, nifi-0.4.0) NIFI-1122_nifi-0.4.0-RC2prepare r
* | 8070a9f - NIFI-1104: - Using the appropriate attributes based on the con
* | 854c667 - NIFI-1211 Adding a .travis.yml to provide CI and adding an exc
|/
* fb65cf1 - NIFI-1271: Yield funnels and ports for nifi.bored.yield.duration

Nothing here gives me heartburn about 0.5.0 going out on short release
cycle.


-- 
Sean