Re: [DISCUSS] Minimize use of MINOR, BUILD, and HOTFIX w/ no JIRA

[Tom Graves]

I think the problems comes in with your definition as well as peoples 
interpretation of that.  I don't agree with your statement of "where the "how" 
is different from the "what"".  
This could apply to a lot of things.  I could easily file a jira that says 
remove synchronization on routine x, then change a lock.  No discussion needed 
and the how is the same as the what.  But it could have huge impact on the code 
and definitely should have a jira.  It may be a contrived example but there is 
a lot of leeway in that.  Which is why I think Patrick originally sent this 
email and you said it yourself above that some of the things reverted  weren't 
trivial enough to begin with.  That just proves people can't make that 
judgement call themselves.  So why not just file a jira for everything?  
Another example of this could be doc changes.  you may think they are trivial 
but if someone changes the docs and remove configs or change the wording such 
that users don't understand, then to me it should have had a jira and possibly 
discussion before changing.  
So based on that it seems like spending the 5 to 30 seconds to file a jira 
would only help in tracking things and isn't much overhead.  
We also base our release notes and other things on jira. 
Also for hotfixes I think they should have the original jira or a separate jira 
(with brokenby linked to original), again for tracking purposes. If we check 
something into master and then later want to cherry-pick it back, I might just 
pick the original commit and totally miss this "HOTFIX" that was required if 
they aren't properly linked.
Tom 

On Thursday, July 7, 2016 2:56 PM, Sean Owen  wrote:
 

 I don't agree that every change needs a JIRA, myself. Really, we
didn't choose to have this system split across JIRA and Github PRs.
It's necessitated by how the ASF works (and with some good reasons).
But while we have this dual system, I figure, let's try to make some
sense of it.

I think it makes sense to make a JIRA for any non-trivial change.
What's non-trivial? where the "how" is different from the "what". That
is, if the JIRA is not just a repeat of the pull request, they should
probably be separate. But, if the change is so simple that describing
it amounts to dictating how it's implemented -- well, seems like a
JIRA is just overhead.

ONe problem that I think happened above was: pretty non-trivial things
were being merged without a JIRA. The evidence? they were reverted.
That means their effect was not quite obvious. They probably deserved
more discussion. Anything that needs some discussion probably deserves
a JIRA.

Also: we have some hot-fixes here that aren't connected to JIRAs.
Either they belong with an existing JIRA and aren't tagged correctly,
or, again, are patching changes that weren't really trivial enough to
skip a JIRA to begin with.

On Thu, Jul 7, 2016 at 7:47 PM, Tom Graves  wrote:
> Popping this back up to the dev list again.  I see a bunch of checkins with
> minor or hotfix.
>
> It seems to me we shouldn't be doing this, but I would like to hear thoughts
> from others.  I see no reason we can't have a jira for each of those issues,
> it only takes a few seconds to file one and it makes things much easier to
> track.
>
> For instance, I tend to watch the jiras on the mailing list and if I hit an
> issue I search jira to see if there is existing one for it, but if there
> isn't jira then I don't see and can't find what someone perhaps already
> fixed with a [MINOR] checkin.
>
> Tom
>
>
> On Saturday, June 6, 2015 11:02 AM, Patrick Wendell 
> wrote:
>
>
> Hey All,
>
> Just a request here - it would be great if people could create JIRA's
> for any and all merged pull requests. The reason is that when patches
> get reverted due to build breaks or other issues, it is very difficult
> to keep track of what is going on if there is no JIRA. Here is a list
> of 5 patches we had to revert recently that didn't include a JIRA:
>
>    Revert "[MINOR] [BUILD] Use custom temp directory during build."
>    Revert "[SQL] [TEST] [MINOR] Uses a temporary log4j.properties in
> HiveThriftServer2Test to ensure expected logging behavior"
>    Revert "[BUILD] Always run SQL tests in master build."
>    Revert "[MINOR] [CORE] Warn users who try to cache RDDs with
> dynamic allocation on."
>    Revert "[HOT FIX] [YARN] Check whether `/lib` exists before
> listing its files"
>
> The cost overhead of creating a JIRA relative to other aspects of
> development is very small. If it's *really* a documentation change or
> something small, that's okay.
>
> But anything affecting the build, packaging, etc. These all need to
> have a JIRA to ensure that follow-up can be well communicated to all
> Spark developers.
>
> Hopefully this is something everyone can get behind, but opened a
> discussion here in case others feel differently.
>
> - Patrick
>
> 

Re: [DISCUSS] Minimize use of MINOR, BUILD, and HOTFIX w/ no JIRA

[Sean Owen]

I don't agree that every change needs a JIRA, myself. Really, we
didn't choose to have this system split across JIRA and Github PRs.
It's necessitated by how the ASF works (and with some good reasons).
But while we have this dual system, I figure, let's try to make some
sense of it.

I think it makes sense to make a JIRA for any non-trivial change.
What's non-trivial? where the "how" is different from the "what". That
is, if the JIRA is not just a repeat of the pull request, they should
probably be separate. But, if the change is so simple that describing
it amounts to dictating how it's implemented -- well, seems like a
JIRA is just overhead.

ONe problem that I think happened above was: pretty non-trivial things
were being merged without a JIRA. The evidence? they were reverted.
That means their effect was not quite obvious. They probably deserved
more discussion. Anything that needs some discussion probably deserves
a JIRA.

Also: we have some hot-fixes here that aren't connected to JIRAs.
Either they belong with an existing JIRA and aren't tagged correctly,
or, again, are patching changes that weren't really trivial enough to
skip a JIRA to begin with.

On Thu, Jul 7, 2016 at 7:47 PM, Tom Graves  wrote:
> Popping this back up to the dev list again.  I see a bunch of checkins with
> minor or hotfix.
>
> It seems to me we shouldn't be doing this, but I would like to hear thoughts
> from others.  I see no reason we can't have a jira for each of those issues,
> it only takes a few seconds to file one and it makes things much easier to
> track.
>
> For instance, I tend to watch the jiras on the mailing list and if I hit an
> issue I search jira to see if there is existing one for it, but if there
> isn't jira then I don't see and can't find what someone perhaps already
> fixed with a [MINOR] checkin.
>
> Tom
>
>
> On Saturday, June 6, 2015 11:02 AM, Patrick Wendell 
> wrote:
>
>
> Hey All,
>
> Just a request here - it would be great if people could create JIRA's
> for any and all merged pull requests. The reason is that when patches
> get reverted due to build breaks or other issues, it is very difficult
> to keep track of what is going on if there is no JIRA. Here is a list
> of 5 patches we had to revert recently that didn't include a JIRA:
>
> Revert "[MINOR] [BUILD] Use custom temp directory during build."
> Revert "[SQL] [TEST] [MINOR] Uses a temporary log4j.properties in
> HiveThriftServer2Test to ensure expected logging behavior"
> Revert "[BUILD] Always run SQL tests in master build."
> Revert "[MINOR] [CORE] Warn users who try to cache RDDs with
> dynamic allocation on."
> Revert "[HOT FIX] [YARN] Check whether `/lib` exists before
> listing its files"
>
> The cost overhead of creating a JIRA relative to other aspects of
> development is very small. If it's *really* a documentation change or
> something small, that's okay.
>
> But anything affecting the build, packaging, etc. These all need to
> have a JIRA to ensure that follow-up can be well communicated to all
> Spark developers.
>
> Hopefully this is something everyone can get behind, but opened a
> discussion here in case others feel differently.
>
> - Patrick
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
> For additional commands, e-mail: dev-h...@spark.apache.org
>
>
>

-
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [DISCUSS] Minimize use of MINOR, BUILD, and HOTFIX w/ no JIRA

[Tom Graves]

Popping this back up to the dev list again.  I see a bunch of checkins with 
minor or hotfix.  
It seems to me we shouldn't be doing this, but I would like to hear thoughts 
from others.  I see no reason we can't have a jira for each of those issues, it 
only takes a few seconds to file one and it makes things much easier to track.
For instance, I tend to watch the jiras on the mailing list and if I hit an 
issue I search jira to see if there is existing one for it, but if there isn't 
jira then I don't see and can't find what someone perhaps already fixed with a 
[MINOR] checkin.
Tom 

On Saturday, June 6, 2015 11:02 AM, Patrick Wendell  
wrote:
 

 Hey All,

Just a request here - it would be great if people could create JIRA's
for any and all merged pull requests. The reason is that when patches
get reverted due to build breaks or other issues, it is very difficult
to keep track of what is going on if there is no JIRA. Here is a list
of 5 patches we had to revert recently that didn't include a JIRA:

    Revert "[MINOR] [BUILD] Use custom temp directory during build."
    Revert "[SQL] [TEST] [MINOR] Uses a temporary log4j.properties in
HiveThriftServer2Test to ensure expected logging behavior"
    Revert "[BUILD] Always run SQL tests in master build."
    Revert "[MINOR] [CORE] Warn users who try to cache RDDs with
dynamic allocation on."
    Revert "[HOT FIX] [YARN] Check whether `/lib` exists before
listing its files"

The cost overhead of creating a JIRA relative to other aspects of
development is very small. If it's *really* a documentation change or
something small, that's okay.

But anything affecting the build, packaging, etc. These all need to
have a JIRA to ensure that follow-up can be well communicated to all
Spark developers.

Hopefully this is something everyone can get behind, but opened a
discussion here in case others feel differently.

- Patrick

-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org



  

Re: [DISCUSS] Minimize use of MINOR, BUILD, and HOTFIX w/ no JIRA

[shane knapp]

+1, and i know i've been guilty of this in the past.  :)

On Wed, Jun 10, 2015 at 10:20 PM, Joseph Bradley jos...@databricks.com
wrote:

 +1

 On Sat, Jun 6, 2015 at 9:01 AM, Patrick Wendell pwend...@gmail.com
 wrote:

 Hey All,

 Just a request here - it would be great if people could create JIRA's
 for any and all merged pull requests. The reason is that when patches
 get reverted due to build breaks or other issues, it is very difficult
 to keep track of what is going on if there is no JIRA. Here is a list
 of 5 patches we had to revert recently that didn't include a JIRA:

 Revert [MINOR] [BUILD] Use custom temp directory during build.
 Revert [SQL] [TEST] [MINOR] Uses a temporary log4j.properties in
 HiveThriftServer2Test to ensure expected logging behavior
 Revert [BUILD] Always run SQL tests in master build.
 Revert [MINOR] [CORE] Warn users who try to cache RDDs with
 dynamic allocation on.
 Revert [HOT FIX] [YARN] Check whether `/lib` exists before
 listing its files

 The cost overhead of creating a JIRA relative to other aspects of
 development is very small. If it's *really* a documentation change or
 something small, that's okay.

 But anything affecting the build, packaging, etc. These all need to
 have a JIRA to ensure that follow-up can be well communicated to all
 Spark developers.

 Hopefully this is something everyone can get behind, but opened a
 discussion here in case others feel differently.

 - Patrick

 -
 To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
 For additional commands, e-mail: dev-h...@spark.apache.org

Re: [DISCUSS] Minimize use of MINOR, BUILD, and HOTFIX w/ no JIRA

[Joseph Bradley]

+1

On Sat, Jun 6, 2015 at 9:01 AM, Patrick Wendell pwend...@gmail.com wrote:

 Hey All,

 Just a request here - it would be great if people could create JIRA's
 for any and all merged pull requests. The reason is that when patches
 get reverted due to build breaks or other issues, it is very difficult
 to keep track of what is going on if there is no JIRA. Here is a list
 of 5 patches we had to revert recently that didn't include a JIRA:

 Revert [MINOR] [BUILD] Use custom temp directory during build.
 Revert [SQL] [TEST] [MINOR] Uses a temporary log4j.properties in
 HiveThriftServer2Test to ensure expected logging behavior
 Revert [BUILD] Always run SQL tests in master build.
 Revert [MINOR] [CORE] Warn users who try to cache RDDs with
 dynamic allocation on.
 Revert [HOT FIX] [YARN] Check whether `/lib` exists before
 listing its files

 The cost overhead of creating a JIRA relative to other aspects of
 development is very small. If it's *really* a documentation change or
 something small, that's okay.

 But anything affecting the build, packaging, etc. These all need to
 have a JIRA to ensure that follow-up can be well communicated to all
 Spark developers.

 Hopefully this is something everyone can get behind, but opened a
 discussion here in case others feel differently.

 - Patrick

 -
 To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
 For additional commands, e-mail: dev-h...@spark.apache.org

[DISCUSS] Minimize use of MINOR, BUILD, and HOTFIX w/ no JIRA

[Patrick Wendell]

Hey All,

Just a request here - it would be great if people could create JIRA's
for any and all merged pull requests. The reason is that when patches
get reverted due to build breaks or other issues, it is very difficult
to keep track of what is going on if there is no JIRA. Here is a list
of 5 patches we had to revert recently that didn't include a JIRA:

Revert [MINOR] [BUILD] Use custom temp directory during build.
Revert [SQL] [TEST] [MINOR] Uses a temporary log4j.properties in
HiveThriftServer2Test to ensure expected logging behavior
Revert [BUILD] Always run SQL tests in master build.
Revert [MINOR] [CORE] Warn users who try to cache RDDs with
dynamic allocation on.
Revert [HOT FIX] [YARN] Check whether `/lib` exists before
listing its files

The cost overhead of creating a JIRA relative to other aspects of
development is very small. If it's *really* a documentation change or
something small, that's okay.

But anything affecting the build, packaging, etc. These all need to
have a JIRA to ensure that follow-up can be well communicated to all
Spark developers.

Hopefully this is something everyone can get behind, but opened a
discussion here in case others feel differently.

- Patrick

-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org