subject:"Re\: Proposed stable release changes"

On Wed, Aug 21, 2013 at 01:16:00PM -0700, Greg KH wrote:
> And I pushed back on that. Which specific stable patch should _not_
> have been included?

Well, here's one for example:

https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/commit/?id=f0a56c480196a98479760862468cc95879df3de0
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=717473#54

I decided not to tag it for stable, even though Ben wanted it, just
because it is the first bug report for this and it was caused by a
pretty unusual hardware configuration. It simply wasn't important enough
to need to add it to stable, IMO.

So basically the rules at the beginning of
Documentation/stable_kernel_rules.txt didn't really apply and that's why
I held off on it.

And I'm pretty sure I've seen similar minor issues like that simply
"automatically" tagged for stable - I just don't have more concrete
examples right now.

> I am going to be pickier (and already have, as some maintainers have
> found out), with what I accept, but so far, the number of patches I've
> rejected can be counted on one hand, a very small percentage of the
> overall number of stable patches.

Ok, fair enough. I mean, in the end of the day, it is less work for you
and for distro people. And more importantly, less unnecessary work. :-)

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

2013-08-21 Thread Tony Luck

On Wed, Aug 21, 2013 at 1:00 PM, Borislav Petkov  wrote:
> We don't want to run daily snapshots of your tree though, right? Only
> -rcs because the daily states are kinda arbitrary and they can be broken
> in various ways. Or are we at a point in time where we can amend that
> rule?

If *nobody* runs daily snapshots - then problems just sit latent all week until
the -rc is released and people start testing. Doesn't sound optimal.

Running daily git snapshots can be "exciting" during the merge window. But
I rarely see problems running a random build after -rc1.  If you are still
running that ancient 3.11-rc6 released on Sunday - then you are missing out
on 28 commits worth of goodness since then :-)

-Tony
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Wed, Aug 21, 2013 at 10:07:03PM +0200, Borislav Petkov wrote:
> On Wed, Aug 21, 2013 at 01:58:16PM -0400, Steven Rostedt wrote:
> > The point I'm making, we should be more reluctant in pulling patches
> > into stable as quick as we are. A patch ideally should simmer in
> > linux-next for a bit, then go into mainline.
> 
> Oh, and it is really debatable if the sheer volume of -stable patches is
> actually warranted - several people already raised the question whether
> we should be more conservative with the stable tag. But you're probably
> going to have this as one of the topics at KS...

And I pushed back on that.  Which specific stable patch should _not_
have been included?

I am going to be pickier (and already have, as some maintainers have
found out), with what I accept, but so far, the number of patches I've
rejected can be counted on one hand, a very small percentage of the
overall number of stable patches.

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Wed, Aug 21, 2013 at 01:58:16PM -0400, Steven Rostedt wrote:
> The point I'm making, we should be more reluctant in pulling patches
> into stable as quick as we are. A patch ideally should simmer in
> linux-next for a bit, then go into mainline.

Oh, and it is really debatable if the sheer volume of -stable patches is
actually warranted - several people already raised the question whether
we should be more conservative with the stable tag. But you're probably
going to have this as one of the topics at KS...

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Wed, Aug 21, 2013 at 11:36:05AM -0700, Linus Torvalds wrote:
> Will it catch all cases? Hell no. We don't have *that* many people who
> run git kernels, and even people who do don't tend to update daily
> anyway.

We don't want to run daily snapshots of your tree though, right? Only
-rcs because the daily states are kinda arbitrary and they can be broken
in various ways. Or are we at a point in time where we can amend that
rule?

For example, what I do currently is, I take your -rcX something and
merge tip/master ontop of it and this is running on my machines for that
week. Come next week, I rinse and repeat. Or does it make sense to do
that more than once a week?

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

2013-08-21 Thread Linus Torvalds

On Wed, Aug 21, 2013 at 11:20 AM, Stephen Warren  wrote:
> On 08/20/2013 04:40 PM, Greg KH wrote:
>
> Presumably the idea is that much useful testing only happens on -rc
> kernels rather than linux-next or arbitrary points in Linus' tree.

Linux-next gets little to no testing outside of compiles.

And I don't think the -rc releases are all that important either. The
important part is to _wait_. Not "wait for an -rc". There are
reasonable number of developers and users who actually run git
kernels, just because they want to help. At rc points, you tend to get
a few more of those.

In contrast, when patches get moved from the development tree to
stable within a day or two, that patch has gotten basically _no_
testing in some cases (or rather, it's been tested to fix the thing it
was supposed to fix, but then there are surprising new problems that
it introduces that nobody even though about, and wasn't tested for).

So I don't think "is in an rc release" is the important thing. I think
"has been in the standard git tree for at least a week" is what we
should aim for.

Will it catch all cases? Hell no. We don't have *that* many people who
run git kernels, and even people who do don't tend to update daily
anyway. But at least this kind of embarrassing "We found a bug within
almost minutes of it hitting mainline" should not make it into stable.

  Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

2013-08-21 Thread Stephen Warren

On 08/20/2013 04:40 PM, Greg KH wrote:
> Hi all,
> 
> Given that I had to just revert a patch in the recent stable releases
> that didn't get enough time to "bake" in Linus's tree (or in -next), I
> figured it was worth discussing some possible changes with how "fast" I
> pick up patches for stable releases.
> 
> So, how about this proposal:
> 
> - I will wait for a -rc to come out with the patch in it before putting
>   it into a stable release, unless: [...]

Presumably the idea is that much useful testing only happens on -rc
kernels rather than linux-next or arbitrary points in Linus' tree.

If so, then don't you want to wait for two -rc releases to come out that
include the patch?

When the first -rc that contains the patch is released, people will test
it then. This won't happen immediately, but might take a few days.

By the time the second -rc that contains the patch is released, that's
presumably enough time for people to have tested the first -rc, and
hence we then presume the patch doesn't cause too many issues.

That's still only an average of 1.5 weeks delay, with a min-max of 1-2,
ignoring the merge window and assuming bugfix patches go quickly
upstream from subsystem maintainers.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 09:03:12PM -0400, Josh Boyer wrote:
> Suspend/Resume is broken on a variety of Thinkpad T400 and T500
> machines in 3.10.  This was true with 3.10.0 afaik.  Current thinking
> is that it's related to the Intel mei/mei_me driver(s).  Blacklisting
> those seems to fix things for a number of users.  There are patches in
> 3.11-rcX, but the "fix" highlighted doesn't fix it.

I have heard of mei issues recently, but no real "this is a problem"
type thing.  There are some patches queued up for 3.12 in that area, if
they are needed earlier, that would be great for me, as a subsystem
maintainer, to know.

> I'm aware I'm reporting issues that you either already knew about or
> were already fixed.  The problem we have is that we roll out a new
> stable release and then we get bug reports for 2 weeks because not
> everyone updates as frequently as stable releases, etc.  So something
> that may seem to impact a small number of users at the time winds up
> actually impacting lots of users once it rolls out in a distro.  As
> far as I know, Fedora is possibly the only distro actually pushing
> stable release kernels out on a normal basis.  I'd love to be wrong on
> that point.

The openSUSE Tumbleweed disto also pushes out these stable kernels.  But
there's only an "estimated" 8-10 thousand users of that openSUSE
"flavor", while smaller than what Fedora has, it's better than nothing.

> In the future, if we can get the information from the end user in
> time, I'll be happy to forward issues that aren't already reported
> onwards.  Or if you still want to hear about it, I can chime in on the
> existing threads with bugzilla numbers.  I'm also willing to do a
> monthly "patches we're carrying not in stable" report if people find
> that helpful.

I would love that report, one of the things I keep asking for is for
people to send the patches that distros have that are not in stable to
me, as those obviously are things that are needed for a valid reason
that everyone should be able to benifit from.

> I'll likely be doing that within Fedora already and I'm happy to send
> it to stable@, even if those patches aren't exactly stable-rules
> matching.

If they aren't "allowed" by the current rules, I'd be interested to know
why, unless it's the "add a new feature" type thing, which makes sense
why I couldn't take them.

> We did that when kernel.org went down and it helped then, just not
> sure how much it would help now or if people care.

I care :)

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Wed, 21 Aug 2013 19:23:27 +0200
Jochen Striepe  wrote:

> ... people are very different. Many just update here and then, but
> there are those who do update on most stable releases. Those are the
> ones you get feedback and testing from, and I think you should encourage
> them to update as soon as a new -stable kernel is released. Getting
> regression fixes fast sounds like a good encouragement especially for
> people interested in the -stable series. Keeping the number of
> regressions low is the whole point of -stable, right?

Yes, but its even worse when we add a new regression to fix the old one.

Note, this will still encourage people to upgrade to stable, and the
delay should be even more encouragement. The point of the delay is to
catch regressions caused by fixes in mainline. We don't want that added
regression to trickle into stable like it has a few times.

As you say "Keeping the number of regressions low is the whole point of
-stable". Yes it is. By the delay, it should help stable from getting
as many regressions as it is today.

> 
> > Maybe people are rushing, but I don't update my main machines every
> > stable release because I can't always afford the down time it causes me
> > to do so.
> 
> On my server/router and on my desktop machine it's the same, those are
> for daily work. But I have a netbook for playing around. :)

Right, and after upgrading your server/router to a new stable, it will
suck that it hits a new regression and you have to update again. A
delay should help limit those occurrences even more.

For netbook that you can play with the -rc candidates and if you hit a
regression, report it, and go back to a more stable kernel.

The point I'm making, we should be more reluctant in pulling patches
into stable as quick as we are. A patch ideally should simmer in
linux-next for a bit, then go into mainline. It should also simmer
there for a bit before it goes into stable. Except for those very rare
patches that can cause the system to easily crash, let an exploit
happen, or corrupt data. Those do need to go into stable ASAP. But
luckily, those are also rare and far between.

-- Steve
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

2013-08-21 Thread Jochen Striepe

Hello,

just speaking as a user, not a developer.

On Wed, Aug 21, 2013 at 09:37:13AM -0400, Steven Rostedt wrote:
> First I want to say that I 100% support the idea of waiting at least one
> -rc. Maybe even two.

I think so, too. But ...

> Really, most fixes are for regressions.
[...]
> If people are not rushing to update their kernel to stable every time a
> new stable is released then why are we rushing to get them out?

... people are very different. Many just update here and then, but
there are those who do update on most stable releases. Those are the
ones you get feedback and testing from, and I think you should encourage
them to update as soon as a new -stable kernel is released. Getting
regression fixes fast sounds like a good encouragement especially for
people interested in the -stable series. Keeping the number of
regressions low is the whole point of -stable, right?

> Maybe people are rushing, but I don't update my main machines every
> stable release because I can't always afford the down time it causes me
> to do so.

On my server/router and on my desktop machine it's the same, those are
for daily work. But I have a netbook for playing around. :)

Have a nice day,
Jochen.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Wed, Aug 21, 2013 at 03:48:52PM +0200, Borislav Petkov wrote:
> On Tue, Aug 20, 2013 at 03:40:32PM -0700, Greg KH wrote:
> > - I will wait for a -rc to come out with the patch in it before putting
> >   it into a stable release, unless:
> 
> Question: what's the exact reasoning of that delay? To get more people
> who install -rc kernels to smoke-test patches tagged for stable.

Yes.

> What happens if the bug gets detected *after* you've taken the patch? We
> probably end up in the same situation.

Yes we will, but this does give us a bit more testing, right?  That's
the point of the delay.

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Wed, 21 Aug 2013 16:17:25 +0200
Willy Tarreau  wrote:

> This is also what I suspected though I have no data to confirm or deny that.
> If this happens to be the case, maybe then there should be some barrier such
> as :
>   - everything merged at -rc4 or before gets backported after the next -rc
>   - everything from -rc5 and upper waits for next -rc1 unless tagged urgent ?

No, I think a full -rc cycle would be good. All commits tagged for
stable that appeared in -rc4 (added from -rc3), must wait till -rc5
before it is added to stable. That way we have a week of full exposure
to find problems.

I guess the other question to ask is, how long does it take for a
problem to appear after hitting mainline? If a problem is found in -rc4
before -rc5 comes out, then this would be sufficient. But if the
problem from -rc4 isn't found till -rc6 then that tells us something
too.

We should be using pass data to determine these heuristics. But I don't
have that data, but Greg should.

-- Steve
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

2013-08-21 Thread Willy Tarreau

On Wed, Aug 21, 2013 at 09:42:48AM -0400, Steven Rostedt wrote:
> Personally, I still let my stable patches that go into later -rc sit
> in linux-next for a few days before pushing them to mainline. I may even
> wait for the next -rc to push it just to make sure the patch wont cause
> more issues. But I know others that just send the fix directly to
> mainline without going through linux-next. Those, I would think, are the
> most prone to cause issues in stable.

This is also what I suspected though I have no data to confirm or deny that.
If this happens to be the case, maybe then there should be some barrier such
as :
  - everything merged at -rc4 or before gets backported after the next -rc
  - everything from -rc5 and upper waits for next -rc1 unless tagged urgent ?

Willy

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 03:40:32PM -0700, Greg KH wrote:
> - I will wait for a -rc to come out with the patch in it before putting
>   it into a stable release, unless:

Question: what's the exact reasoning of that delay? To get more people
who install -rc kernels to smoke-test patches tagged for stable.

What happens if the bug gets detected *after* you've taken the patch? We
probably end up in the same situation.

I guess I'm questioning the usefullness of such a delay period...

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 08:41:23PM -0400, Josh Boyer wrote:
> 
> Let me phrase this as a question instead.  Is there something we can
> do to help catch the patches that get sucked into stable during the
> merge window and then wind up causing issues and reverted/fixed after
> things settle down in the -rc releases?

I'm curious. Of the patches that went into stable that caused problems,
how many were from the merge window?

Reason that I ask this, is because the patches I tag for stable that I
wait for a merge window for release, goes through the process of all my
patches that enter the merge window. It sits in linux-next for a while
and usually gets much more testing than a fix that may come later in the
-rc cycles.

Personally, I still let my stable patches that go into later -rc sit
in linux-next for a few days before pushing them to mainline. I may even
wait for the next -rc to push it just to make sure the patch wont cause
more issues. But I know others that just send the fix directly to
mainline without going through linux-next. Those, I would think, are the
most prone to cause issues in stable.

-- Steve

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

First I want to say that I 100% support the idea of waiting at least one
-rc. Maybe even two.

On Wed, Aug 21, 2013 at 07:38:36AM +0200, Willy Tarreau wrote:
> > Cc: stable  # after -rc5 is out
> > or
> > Cc: stable  # wait a -rc cycle
> > or
> > Cc: stable  # wait a few weeks to bake
> 
> That's where I think that the default one (with no indication) should
> be the higher delay. If the author has no clue about the emergency of
> his patch, who else can guess for him ?
> 
> It's too optimistic to consider that some code authors will be
> realist about the impacts of their code. We all create bugs and
> regressions everywhere because we're sure about what we do, until
> someone says "hey dude you broke this". So if we expect authors to
> say "look, I managed to get this merged into mainline but I'm still
> not sure about the risks", I suspect only a small fraction of the
> patches will be tagged this way. But I may be wrong, after all it
> already works well with -net.

Really, most fixes are for regressions. If it isn't something that can
let non root crash the kernel, or have access that they shouldn't have,
then let it simmer in the kernel for a while. I don't see any reason to
rush out a regression fix if it just causes a device not to work
anymore. The user can go back to the old kernel for a week or two and
wait. Even with two weeks (or even a month) of waiting, we are still much
faster than Apple or Microsoft in getting regression fixes out.

If people are not rushing to update their kernel to stable every time a
new stable is released then why are we rushing to get them out?

Maybe people are rushing, but I don't update my main machines every
stable release because I can't always afford the down time it causes me
to do so.

-- Steve

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

First I want to say that I 100% support the idea of waiting at least one
-rc. Maybe even two.

On Wed, Aug 21, 2013 at 07:38:36AM +0200, Willy Tarreau wrote:
  Cc: stable sta...@vger.kernel.org # after -rc5 is out
  or
  Cc: stable sta...@vger.kernel.org # wait a -rc cycle
  or
  Cc: stable sta...@vger.kernel.org # wait a few weeks to bake

 That's where I think that the default one (with no indication) should
 be the higher delay. If the author has no clue about the emergency of
 his patch, who else can guess for him ?

 It's too optimistic to consider that some code authors will be
 realist about the impacts of their code. We all create bugs and
 regressions everywhere because we're sure about what we do, until
 someone says hey dude you broke this. So if we expect authors to
 say look, I managed to get this merged into mainline but I'm still
 not sure about the risks, I suspect only a small fraction of the
 patches will be tagged this way. But I may be wrong, after all it
 already works well with -net.

Really, most fixes are for regressions. If it isn't something that can
let non root crash the kernel, or have access that they shouldn't have,
then let it simmer in the kernel for a while. I don't see any reason to
rush out a regression fix if it just causes a device not to work
anymore. The user can go back to the old kernel for a week or two and
wait. Even with two weeks (or even a month) of waiting, we are still much
faster than Apple or Microsoft in getting regression fixes out.

If people are not rushing to update their kernel to stable every time a
new stable is released then why are we rushing to get them out?

Maybe people are rushing, but I don't update my main machines every
stable release because I can't always afford the down time it causes me
to do so.

-- Steve

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 08:41:23PM -0400, Josh Boyer wrote:
 
 Let me phrase this as a question instead.  Is there something we can
 do to help catch the patches that get sucked into stable during the
 merge window and then wind up causing issues and reverted/fixed after
 things settle down in the -rc releases?

I'm curious. Of the patches that went into stable that caused problems,
how many were from the merge window?

Reason that I ask this, is because the patches I tag for stable that I
wait for a merge window for release, goes through the process of all my
patches that enter the merge window. It sits in linux-next for a while
and usually gets much more testing than a fix that may come later in the
-rc cycles.

Personally, I still let my stable patches that go into later -rc sit
in linux-next for a few days before pushing them to mainline. I may even
wait for the next -rc to push it just to make sure the patch wont cause
more issues. But I know others that just send the fix directly to
mainline without going through linux-next. Those, I would think, are the
most prone to cause issues in stable.

-- Steve

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 03:40:32PM -0700, Greg KH wrote:
 - I will wait for a -rc to come out with the patch in it before putting
   it into a stable release, unless:

Question: what's the exact reasoning of that delay? To get more people
who install -rc kernels to smoke-test patches tagged for stable.

What happens if the bug gets detected *after* you've taken the patch? We
probably end up in the same situation.

I guess I'm questioning the usefullness of such a delay period...

Thanks.
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

2013-08-21 Thread Willy Tarreau

On Wed, Aug 21, 2013 at 09:42:48AM -0400, Steven Rostedt wrote:
 Personally, I still let my stable patches that go into later -rc sit
 in linux-next for a few days before pushing them to mainline. I may even
 wait for the next -rc to push it just to make sure the patch wont cause
 more issues. But I know others that just send the fix directly to
 mainline without going through linux-next. Those, I would think, are the
 most prone to cause issues in stable.

This is also what I suspected though I have no data to confirm or deny that.
If this happens to be the case, maybe then there should be some barrier such
as :
  - everything merged at -rc4 or before gets backported after the next -rc
  - everything from -rc5 and upper waits for next -rc1 unless tagged urgent ?

Willy

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Wed, 21 Aug 2013 16:17:25 +0200
Willy Tarreau w...@1wt.eu wrote:

 This is also what I suspected though I have no data to confirm or deny that.
 If this happens to be the case, maybe then there should be some barrier such
 as :
   - everything merged at -rc4 or before gets backported after the next -rc
   - everything from -rc5 and upper waits for next -rc1 unless tagged urgent ?

No, I think a full -rc cycle would be good. All commits tagged for
stable that appeared in -rc4 (added from -rc3), must wait till -rc5
before it is added to stable. That way we have a week of full exposure
to find problems.

I guess the other question to ask is, how long does it take for a
problem to appear after hitting mainline? If a problem is found in -rc4
before -rc5 comes out, then this would be sufficient. But if the
problem from -rc4 isn't found till -rc6 then that tells us something
too.

We should be using pass data to determine these heuristics. But I don't
have that data, but Greg should.


-- Steve
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Wed, Aug 21, 2013 at 03:48:52PM +0200, Borislav Petkov wrote:
 On Tue, Aug 20, 2013 at 03:40:32PM -0700, Greg KH wrote:
  - I will wait for a -rc to come out with the patch in it before putting
it into a stable release, unless:
 
 Question: what's the exact reasoning of that delay? To get more people
 who install -rc kernels to smoke-test patches tagged for stable.

Yes.

 What happens if the bug gets detected *after* you've taken the patch? We
 probably end up in the same situation.

Yes we will, but this does give us a bit more testing, right?  That's
the point of the delay.

thanks,

greg k-h
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

2013-08-21 Thread Jochen Striepe

Hello,

just speaking as a user, not a developer.

On Wed, Aug 21, 2013 at 09:37:13AM -0400, Steven Rostedt wrote:
 First I want to say that I 100% support the idea of waiting at least one
 -rc. Maybe even two.

I think so, too. But ...

 Really, most fixes are for regressions.
[...]
 If people are not rushing to update their kernel to stable every time a
 new stable is released then why are we rushing to get them out?

... people are very different. Many just update here and then, but
there are those who do update on most stable releases. Those are the
ones you get feedback and testing from, and I think you should encourage
them to update as soon as a new -stable kernel is released. Getting
regression fixes fast sounds like a good encouragement especially for
people interested in the -stable series. Keeping the number of
regressions low is the whole point of -stable, right?

 Maybe people are rushing, but I don't update my main machines every
 stable release because I can't always afford the down time it causes me
 to do so.

On my server/router and on my desktop machine it's the same, those are
for daily work. But I have a netbook for playing around. :)


Have a nice day,
Jochen.
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Wed, 21 Aug 2013 19:23:27 +0200
Jochen Striepe joc...@tolot.escape.de wrote:

 ... people are very different. Many just update here and then, but
 there are those who do update on most stable releases. Those are the
 ones you get feedback and testing from, and I think you should encourage
 them to update as soon as a new -stable kernel is released. Getting
 regression fixes fast sounds like a good encouragement especially for
 people interested in the -stable series. Keeping the number of
 regressions low is the whole point of -stable, right?

Yes, but its even worse when we add a new regression to fix the old one.

Note, this will still encourage people to upgrade to stable, and the
delay should be even more encouragement. The point of the delay is to
catch regressions caused by fixes in mainline. We don't want that added
regression to trickle into stable like it has a few times.

As you say Keeping the number of regressions low is the whole point of
-stable. Yes it is. By the delay, it should help stable from getting
as many regressions as it is today.

 
  Maybe people are rushing, but I don't update my main machines every
  stable release because I can't always afford the down time it causes me
  to do so.
 
 On my server/router and on my desktop machine it's the same, those are
 for daily work. But I have a netbook for playing around. :)

Right, and after upgrading your server/router to a new stable, it will
suck that it hits a new regression and you have to update again. A
delay should help limit those occurrences even more.

For netbook that you can play with the -rc candidates and if you hit a
regression, report it, and go back to a more stable kernel.

The point I'm making, we should be more reluctant in pulling patches
into stable as quick as we are. A patch ideally should simmer in
linux-next for a bit, then go into mainline. It should also simmer
there for a bit before it goes into stable. Except for those very rare
patches that can cause the system to easily crash, let an exploit
happen, or corrupt data. Those do need to go into stable ASAP. But
luckily, those are also rare and far between.

-- Steve
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 09:03:12PM -0400, Josh Boyer wrote:
 Suspend/Resume is broken on a variety of Thinkpad T400 and T500
 machines in 3.10.  This was true with 3.10.0 afaik.  Current thinking
 is that it's related to the Intel mei/mei_me driver(s).  Blacklisting
 those seems to fix things for a number of users.  There are patches in
 3.11-rcX, but the fix highlighted doesn't fix it.

I have heard of mei issues recently, but no real this is a problem
type thing.  There are some patches queued up for 3.12 in that area, if
they are needed earlier, that would be great for me, as a subsystem
maintainer, to know.

 I'm aware I'm reporting issues that you either already knew about or
 were already fixed.  The problem we have is that we roll out a new
 stable release and then we get bug reports for 2 weeks because not
 everyone updates as frequently as stable releases, etc.  So something
 that may seem to impact a small number of users at the time winds up
 actually impacting lots of users once it rolls out in a distro.  As
 far as I know, Fedora is possibly the only distro actually pushing
 stable release kernels out on a normal basis.  I'd love to be wrong on
 that point.

The openSUSE Tumbleweed disto also pushes out these stable kernels.  But
there's only an estimated 8-10 thousand users of that openSUSE
flavor, while smaller than what Fedora has, it's better than nothing.

 In the future, if we can get the information from the end user in
 time, I'll be happy to forward issues that aren't already reported
 onwards.  Or if you still want to hear about it, I can chime in on the
 existing threads with bugzilla numbers.  I'm also willing to do a
 monthly patches we're carrying not in stable report if people find
 that helpful.

I would love that report, one of the things I keep asking for is for
people to send the patches that distros have that are not in stable to
me, as those obviously are things that are needed for a valid reason
that everyone should be able to benifit from.

 I'll likely be doing that within Fedora already and I'm happy to send
 it to stable@, even if those patches aren't exactly stable-rules
 matching.

If they aren't allowed by the current rules, I'd be interested to know
why, unless it's the add a new feature type thing, which makes sense
why I couldn't take them.

 We did that when kernel.org went down and it helped then, just not
 sure how much it would help now or if people care.

I care :)

thanks,

greg k-h
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

2013-08-21 Thread Stephen Warren

On 08/20/2013 04:40 PM, Greg KH wrote:
 Hi all,
 
 Given that I had to just revert a patch in the recent stable releases
 that didn't get enough time to bake in Linus's tree (or in -next), I
 figured it was worth discussing some possible changes with how fast I
 pick up patches for stable releases.
 
 So, how about this proposal:
 
 - I will wait for a -rc to come out with the patch in it before putting
   it into a stable release, unless: [...]

Presumably the idea is that much useful testing only happens on -rc
kernels rather than linux-next or arbitrary points in Linus' tree.

If so, then don't you want to wait for two -rc releases to come out that
include the patch?

When the first -rc that contains the patch is released, people will test
it then. This won't happen immediately, but might take a few days.

By the time the second -rc that contains the patch is released, that's
presumably enough time for people to have tested the first -rc, and
hence we then presume the patch doesn't cause too many issues.

That's still only an average of 1.5 weeks delay, with a min-max of 1-2,
ignoring the merge window and assuming bugfix patches go quickly
upstream from subsystem maintainers.
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

2013-08-21 Thread Linus Torvalds

On Wed, Aug 21, 2013 at 11:20 AM, Stephen Warren swar...@wwwdotorg.org wrote:
 On 08/20/2013 04:40 PM, Greg KH wrote:

 Presumably the idea is that much useful testing only happens on -rc
 kernels rather than linux-next or arbitrary points in Linus' tree.

Linux-next gets little to no testing outside of compiles.

And I don't think the -rc releases are all that important either. The
important part is to _wait_. Not wait for an -rc. There are
reasonable number of developers and users who actually run git
kernels, just because they want to help. At rc points, you tend to get
a few more of those.

In contrast, when patches get moved from the development tree to
stable within a day or two, that patch has gotten basically _no_
testing in some cases (or rather, it's been tested to fix the thing it
was supposed to fix, but then there are surprising new problems that
it introduces that nobody even though about, and wasn't tested for).

So I don't think is in an rc release is the important thing. I think
has been in the standard git tree for at least a week is what we
should aim for.

Will it catch all cases? Hell no. We don't have *that* many people who
run git kernels, and even people who do don't tend to update daily
anyway. But at least this kind of embarrassing We found a bug within
almost minutes of it hitting mainline should not make it into stable.

  Linus
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Wed, Aug 21, 2013 at 11:36:05AM -0700, Linus Torvalds wrote:
 Will it catch all cases? Hell no. We don't have *that* many people who
 run git kernels, and even people who do don't tend to update daily
 anyway.

We don't want to run daily snapshots of your tree though, right? Only
-rcs because the daily states are kinda arbitrary and they can be broken
in various ways. Or are we at a point in time where we can amend that
rule?

For example, what I do currently is, I take your -rcX something and
merge tip/master ontop of it and this is running on my machines for that
week. Come next week, I rinse and repeat. Or does it make sense to do
that more than once a week?

Thanks.
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Wed, Aug 21, 2013 at 01:58:16PM -0400, Steven Rostedt wrote:
 The point I'm making, we should be more reluctant in pulling patches
 into stable as quick as we are. A patch ideally should simmer in
 linux-next for a bit, then go into mainline.

Oh, and it is really debatable if the sheer volume of -stable patches is
actually warranted - several people already raised the question whether
we should be more conservative with the stable tag. But you're probably
going to have this as one of the topics at KS...

Thanks.
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Wed, Aug 21, 2013 at 10:07:03PM +0200, Borislav Petkov wrote:
 On Wed, Aug 21, 2013 at 01:58:16PM -0400, Steven Rostedt wrote:
  The point I'm making, we should be more reluctant in pulling patches
  into stable as quick as we are. A patch ideally should simmer in
  linux-next for a bit, then go into mainline.
 
 Oh, and it is really debatable if the sheer volume of -stable patches is
 actually warranted - several people already raised the question whether
 we should be more conservative with the stable tag. But you're probably
 going to have this as one of the topics at KS...

And I pushed back on that.  Which specific stable patch should _not_
have been included?

I am going to be pickier (and already have, as some maintainers have
found out), with what I accept, but so far, the number of patches I've
rejected can be counted on one hand, a very small percentage of the
overall number of stable patches.

thanks,

greg k-h
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

2013-08-21 Thread Tony Luck

On Wed, Aug 21, 2013 at 1:00 PM, Borislav Petkov b...@alien8.de wrote:
 We don't want to run daily snapshots of your tree though, right? Only
 -rcs because the daily states are kinda arbitrary and they can be broken
 in various ways. Or are we at a point in time where we can amend that
 rule?

If *nobody* runs daily snapshots - then problems just sit latent all week until
the -rc is released and people start testing. Doesn't sound optimal.

Running daily git snapshots can be exciting during the merge window. But
I rarely see problems running a random build after -rc1.  If you are still
running that ancient 3.11-rc6 released on Sunday - then you are missing out
on 28 commits worth of goodness since then :-)

-Tony
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Wed, Aug 21, 2013 at 01:16:00PM -0700, Greg KH wrote:
And I pushed back on that. Which specific stable patch should _not_
have been included?

Well, here's one for example:

https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/commit/?id=f0a56c480196a98479760862468cc95879df3de0
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=717473#54

I decided not to tag it for stable, even though Ben wanted it, just
because it is the first bug report for this and it was caused by a
pretty unusual hardware configuration. It simply wasn't important enough
to need to add it to stable, IMO.

So basically the rules at the beginning of
Documentation/stable_kernel_rules.txt didn't really apply and that's why
I held off on it.

And I'm pretty sure I've seen similar minor issues like that simply
automatically tagged for stable - I just don't have more concrete
examples right now.

I am going to be pickier (and already have, as some maintainers have
found out), with what I accept, but so far, the number of patches I've
rejected can be counted on one hand, a very small percentage of the
overall number of stable patches.

Ok, fair enough. I mean, in the end of the day, it is less work for you
and for distro people. And more importantly, less unnecessary work. :-)

Thanks.
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Re: Proposed stable release changes

2013-08-21 Thread Stephen Rothwell

On Wed, 21 Aug 2013 11:36:05 -0700 Linus Torvalds 
torva...@linux-foundation.org wrote:

 On Wed, Aug 21, 2013 at 11:20 AM, Stephen Warren swar...@wwwdotorg.org 
 wrote:
  On 08/20/2013 04:40 PM, Greg KH wrote:
 
  Presumably the idea is that much useful testing only happens on -rc
  kernels rather than linux-next or arbitrary points in Linus' tree.
 
 Linux-next gets little to no testing outside of compiles.

Besides which, regression fixes often get no exposure in linux-next anyway.

-- 
Cheers,
Stephen Rothwells...@canb.auug.org.au


pgpkvGvNXKkz_.pgp
Description: PGP signature

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 05:49:24PM -0700, Greg KH wrote:
> On Tue, Aug 20, 2013 at 08:41:23PM -0400, Josh Boyer wrote:
> > On Tue, Aug 20, 2013 at 7:57 PM, Greg KH  wrote:
> > >> I like this overall.  The only thing I might change is "wait for -rc2"
> > >> for patches tagged with CC: stable that go in during the merge window.
> > >>  It seems those are the ones that tend to bite us.
> > >
> > > Maintainers can always tag their patches to have me hold off until -rc2
> > > for that.
> > 
> > They can (not immediately sure how though?)
> 
> Some do:
>   Cc: stable  # after -rc5 is out
> or
>   Cc: stable  # wait a -rc cycle
> or
>   Cc: stable  # wait a few weeks to bake

That's where I think that the default one (with no indication) should
be the higher delay. If the author has no clue about the emergency of
his patch, who else can guess for him ?

It's too optimistic to consider that some code authors will be
realist about the impacts of their code. We all create bugs and
regressions everywhere because we're sure about what we do, until
someone says "hey dude you broke this". So if we expect authors to
say "look, I managed to get this merged into mainline but I'm still
not sure about the risks", I suspect only a small fraction of the
patches will be tagged this way. But I may be wrong, after all it
already works well with -net.

Willy

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 04:12:32PM -0700, Greg KH wrote:
> On Wed, Aug 21, 2013 at 12:58:15AM +0200, Willy Tarreau wrote:
> > Hi Greg,
> > 
> > On Tue, Aug 20, 2013 at 03:40:32PM -0700, Greg KH wrote:
> > > Hi all,
> > > 
> > > Given that I had to just revert a patch in the recent stable releases
> > > that didn't get enough time to "bake" in Linus's tree (or in -next), I
> > > figured it was worth discussing some possible changes with how "fast" I
> > > pick up patches for stable releases.
> > > 
> > > So, how about this proposal:
> > > 
> > > - I will wait for a -rc to come out with the patch in it before putting
> > >   it into a stable release, unless:
> > >   - the maintainer ACKs it, or sends it directly (like DaveM does
> > > for networking patches)
> > >   - I have seen enough discussion about a patch to show that it
> > > really does fix something / is good / doesn't cause problems.
> > >   - obviously safe, i.e. "add a device id" type thing.
> > > 
> > > Given that we have -rc releases every week, except for the initial -rc1
> > > release, I don't think this will really cause any major delays.
> > 
> > In the last discussion you initiated on the subject, I proposed something
> > even more conservative which was the same as above except instead of
> > "wait for a -rc", it was "wait for rc1 after a full release containing
> > the patch", unless one of the conditions you proposed, or another one
> > which would be a tag "urgent" or something like this in the patch.
> 
> Waiting 3 months is too long, in my opinion, sorry.

I meant only for the non-important ones. Their authors will qualify the
ones that are important and must not wait. The same way as now many patches
are correctly tagged "cc: stable", I suspect that we could end up with maybe
80% of patches tagged as "must not wait", and the remaining 20% would indeed
wait up to 3 months, but if their authors think they should wait maybe we
should trust them.

Willy

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 09:03:12PM -0400, Josh Boyer wrote:
[ ... ]

> 
> Suspend/Resume is broken on a variety of Thinkpad T400 and T500
> machines in 3.10.  This was true with 3.10.0 afaik.  Current thinking
> is that it's related to the Intel mei/mei_me driver(s).  Blacklisting

Reminds me ... I had to disable MEI on my servers at home because 
it caused some instability (random system hangs) and the watchdog
didn't work. I thought that was only me, but maybe not.

Guenter
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 8:49 PM, Greg KH  wrote:
> On Tue, Aug 20, 2013 at 08:41:23PM -0400, Josh Boyer wrote:
>> On Tue, Aug 20, 2013 at 7:57 PM, Greg KH  wrote:
>> >> I like this overall.  The only thing I might change is "wait for -rc2"
>> >> for patches tagged with CC: stable that go in during the merge window.
>> >>  It seems those are the ones that tend to bite us.
>> >
>> > Maintainers can always tag their patches to have me hold off until -rc2
>> > for that.
>>
>> They can (not immediately sure how though?)
>
> Some do:
> Cc: stable  # after -rc5 is out
> or
> Cc: stable  # wait a -rc cycle
> or
> Cc: stable  # wait a few weeks to bake
>
>> , but they don't with the
>> exception of the few that don't tag at all and send you patches in
>> bundles.  I mean, that's what the huge thread about the stable trees
>> that hopefully leads to a conversation at KS is about, right?
>
> Hopefully, yes, but I don't know about that yet.
>
>> Let me phrase this as a question instead.  Is there something we can
>> do to help catch the patches that get sucked into stable during the
>> merge window and then wind up causing issues and reverted/fixed after
>> things settle down in the -rc releases?
>
> Test linux-next and Linus's tree-of-the-day better.  If problems happen,
> and a patch has a cc: stable@ on it, let stable@ know about it.

Ah, yes.  I forgot about the "look through linux-next for CC: stable".
 That could probably be automated to a degree even.

We already build a Linus kernel in rawhide at least once a day.  Two
flavors even, once with debug options enabled and once without.  The
three of us run it and have an autotest setup going and being
improved, but there is no substitute for wide-spread user testing.
Unfortunately, we have problems getting that with rawhide for various
reasons.

>> I'm offering to help in whatever way you think is best.  It's your
>> workflow (and sanity) that are the most impacted.  However, I share
>> the pain whenever something breaks in stable through the wonderful
>> place that is Fedora bugzilla so I'm looking for ways to reduce that.
>
> Letting me know when something breaks is always good as well.  Right now
> that doesn't seem to happen much, so either not much is breaking, or I'm
> just not told about it, I don't know which.

There's no need to reply specifically to these, but off the top of my
head just for 3.10.y:

brcmsmac explodes in 3.10.{6,7,8,9}.  I think you know about that one
already and Felix and Arend are working on it.

Suspend/Resume is broken on a variety of Thinkpad T400 and T500
machines in 3.10.  This was true with 3.10.0 afaik.  Current thinking
is that it's related to the Intel mei/mei_me driver(s).  Blacklisting
those seems to fix things for a number of users.  There are patches in
3.11-rcX, but the "fix" highlighted doesn't fix it.

There were various i915 backlight issues reported with 3.10.3/4.  I
believe they were fixed with 3.10.5.

USB storage was broken because of the mishandled VPD stuff.  Fixed in 3.10.7.

I'm aware I'm reporting issues that you either already knew about or
were already fixed.  The problem we have is that we roll out a new
stable release and then we get bug reports for 2 weeks because not
everyone updates as frequently as stable releases, etc.  So something
that may seem to impact a small number of users at the time winds up
actually impacting lots of users once it rolls out in a distro.  As
far as I know, Fedora is possibly the only distro actually pushing
stable release kernels out on a normal basis.  I'd love to be wrong on
that point.

In the future, if we can get the information from the end user in
time, I'll be happy to forward issues that aren't already reported
onwards.  Or if you still want to hear about it, I can chime in on the
existing threads with bugzilla numbers.  I'm also willing to do a
monthly "patches we're carrying not in stable" report if people find
that helpful.  I'll likely be doing that within Fedora already and I'm
happy to send it to stable@, even if those patches aren't exactly
stable-rules matching.  We did that when kernel.org went down and it
helped then, just not sure how much it would help now or if people
care.

josh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 08:41:23PM -0400, Josh Boyer wrote:
> On Tue, Aug 20, 2013 at 7:57 PM, Greg KH  wrote:
> >> I like this overall.  The only thing I might change is "wait for -rc2"
> >> for patches tagged with CC: stable that go in during the merge window.
> >>  It seems those are the ones that tend to bite us.
> >
> > Maintainers can always tag their patches to have me hold off until -rc2
> > for that.
> 
> They can (not immediately sure how though?)

Some do:
Cc: stable  # after -rc5 is out
or
Cc: stable  # wait a -rc cycle
or
Cc: stable  # wait a few weeks to bake

> , but they don't with the
> exception of the few that don't tag at all and send you patches in
> bundles.  I mean, that's what the huge thread about the stable trees
> that hopefully leads to a conversation at KS is about, right?

Hopefully, yes, but I don't know about that yet.

> Let me phrase this as a question instead.  Is there something we can
> do to help catch the patches that get sucked into stable during the
> merge window and then wind up causing issues and reverted/fixed after
> things settle down in the -rc releases?

Test linux-next and Linus's tree-of-the-day better.  If problems happen,
and a patch has a cc: stable@ on it, let stable@ know about it.

> I'm offering to help in whatever way you think is best.  It's your
> workflow (and sanity) that are the most impacted.  However, I share
> the pain whenever something breaks in stable through the wonderful
> place that is Fedora bugzilla so I'm looking for ways to reduce that.

Letting me know when something breaks is always good as well.  Right now
that doesn't seem to happen much, so either not much is breaking, or I'm
just not told about it, I don't know which.

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 7:57 PM, Greg KH  wrote:
> On Tue, Aug 20, 2013 at 07:40:49PM -0400, Josh Boyer wrote:
>> On Tue, Aug 20, 2013 at 6:40 PM, Greg KH  wrote:
>> > Hi all,
>> >
>> > Given that I had to just revert a patch in the recent stable releases
>> > that didn't get enough time to "bake" in Linus's tree (or in -next), I
>> > figured it was worth discussing some possible changes with how "fast" I
>> > pick up patches for stable releases.
>> >
>> > So, how about this proposal:
>> >
>> > - I will wait for a -rc to come out with the patch in it before putting
>> >   it into a stable release, unless:
>> > - the maintainer ACKs it, or sends it directly (like DaveM does
>> >   for networking patches)
>> > - I have seen enough discussion about a patch to show that it
>> >   really does fix something / is good / doesn't cause problems.
>> > - obviously safe, i.e. "add a device id" type thing.
>> >
>> > Given that we have -rc releases every week, except for the initial -rc1
>> > release, I don't think this will really cause any major delays.
>> >
>> > Also, now that we are about to head into my busy "travel season", odds
>> > are, I'll be at least a week behind anyway, so this would probably start
>> > happening without an "official" change.  It's been a boring summer, I've
>> > been able to keep up with the stable stuff really easily, causing
>> > problems like this :)
>> >
>> > Objections?  Comments?
>>
>> I like this overall.  The only thing I might change is "wait for -rc2"
>> for patches tagged with CC: stable that go in during the merge window.
>>  It seems those are the ones that tend to bite us.
>
> Maintainers can always tag their patches to have me hold off until -rc2
> for that.

They can (not immediately sure how though?), but they don't with the
exception of the few that don't tag at all and send you patches in
bundles.  I mean, that's what the huge thread about the stable trees
that hopefully leads to a conversation at KS is about, right?

> And, given the huge number of patches for stable that come in during
> the -rc1 merge window, there's no way I can get to them all before -rc2
> comes out, so this will probably end up being the case for most of them
> anyway.

Sheer numbers forcing some to be pushed out isn't really that great
when the problem is spread across the whole window.  I'm not saying
you aren't correct that a lot of them don't get pushed to stable
releases until post -rc2, but at that point you're playing catch up.
I was suggesting just sitting on them until -rc2 entirely.  Basically,
don't release a stable kernel until the next version is at -rc2.

Let me phrase this as a question instead.  Is there something we can
do to help catch the patches that get sucked into stable during the
merge window and then wind up causing issues and reverted/fixed after
things settle down in the -rc releases?

Reviewing the patches submitted for those stable kernels isn't
necessarily sufficient, since the issues I'm concerned about aren't
spotted until after the release anyway.  It's a two-fold problem.  The
first is one we're never going to fix in that people are human and
make mistakes and sometimes patches don't get tested well enough
before they're tagged for stable.  The second issue is a combination
of timing and volume.  I doubt we're going to fix the volume since we
keep touting the increasing change rate as a sign of success, but
could we fix the timing by forcing things to bake more in the -rc
releases?

I'm offering to help in whatever way you think is best.  It's your
workflow (and sanity) that are the most impacted.  However, I share
the pain whenever something breaks in stable through the wonderful
place that is Fedora bugzilla so I'm looking for ways to reduce that.

josh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 05:11:11PM -0700, Guenter Roeck wrote:
> On Tue, Aug 20, 2013 at 04:17:43PM -0700, Greg KH wrote:
> > On Tue, Aug 20, 2013 at 04:11:17PM -0700, Guenter Roeck wrote:
> > > It would be even better if you could find the time to push -rc releases
> > > into the stable repository before you accept patches into a stable branch.
> > 
> > What do you mean by this?
> >
> > The git tree?  I could do that, but when I have to drop a patch, it
> > would cause a mess if I had to always go forwards.
> > 
> That is not what I mean. I referred to the master branch of the stable
> repository, which you normally update to the most recent -rc when you
> publish a new -stable release.

Ah, ok, that makes sense.  I've updated it to 3.11-rc6 now.  Sorry for
the confusion.

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 04:17:43PM -0700, Greg KH wrote:
> On Tue, Aug 20, 2013 at 04:11:17PM -0700, Guenter Roeck wrote:
> > It would be even better if you could find the time to push -rc releases
> > into the stable repository before you accept patches into a stable branch.
> 
> What do you mean by this?
>
> The git tree?  I could do that, but when I have to drop a patch, it
> would cause a mess if I had to always go forwards.
> 
That is not what I mean. I referred to the master branch of the stable
repository, which you normally update to the most recent -rc when you
publish a new -stable release.

> I could do branches for -rc releases, that end up as the
> "end-of-the-line", and I create the next .y release on top of the
> previous one, not the -rc release.
> 
I would not ask for creating any new branches. The existing ones are good
enough.

> That's kind of what I do "internally" when I create the -rc releases in
> the first place, but I just delete those throw-away trees, and never
> push them publicly anywhere.
> 
> Does it really help anyone to do this, except for some automated
> testing?  Doesn't the -rc patch work good enough for that?
> 
> > I am now running my test suite on stable/master, so that would give it
> > some time to catch new problems before they make their way into a stable
> > branch.
> 
> I don't understand what you mean here.
> 
The master branch of linux-stable.git on kernel.org currently points to
v3.11-rc5. All I asked for is to update it to the latest -rc prior to
accepting patches from this -rc into a stable release.

I am running my test suite on linux-stable:master to have a reference
and comparison against the various -stable branches. As soon as you update
the master branch of linux-stable.git (eg after you push v3.11-rc6 into it),
the test suite will automatically run on it.

Obviously I could track linux-kernel.git itself instead, but there is much more
activity on it and I am really only interested in the most recent tagged 
version.

Hope that describes it better.

Guenter
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 04:12:32PM -0700, Greg KH wrote:
> On Wed, Aug 21, 2013 at 12:58:15AM +0200, Willy Tarreau wrote:
> > Hi Greg,
> > 
> > On Tue, Aug 20, 2013 at 03:40:32PM -0700, Greg KH wrote:
> > > Hi all,
> > > 
> > > Given that I had to just revert a patch in the recent stable releases
> > > that didn't get enough time to "bake" in Linus's tree (or in -next), I
> > > figured it was worth discussing some possible changes with how "fast" I
> > > pick up patches for stable releases.
> > > 
> > > So, how about this proposal:
> > > 
> > > - I will wait for a -rc to come out with the patch in it before putting
> > >   it into a stable release, unless:
> > >   - the maintainer ACKs it, or sends it directly (like DaveM does
> > > for networking patches)
> > >   - I have seen enough discussion about a patch to show that it
> > > really does fix something / is good / doesn't cause problems.
> > >   - obviously safe, i.e. "add a device id" type thing.
> > > 
> > > Given that we have -rc releases every week, except for the initial -rc1
> > > release, I don't think this will really cause any major delays.
> > 
> > In the last discussion you initiated on the subject, I proposed something
> > even more conservative which was the same as above except instead of
> > "wait for a -rc", it was "wait for rc1 after a full release containing
> > the patch", unless one of the conditions you proposed, or another one
> > which would be a tag "urgent" or something like this in the patch.
> 
> Waiting 3 months is too long, in my opinion, sorry.
> 
I definitely agree.

Guenter
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 07:40:49PM -0400, Josh Boyer wrote:
> On Tue, Aug 20, 2013 at 6:40 PM, Greg KH  wrote:
> > Hi all,
> >
> > Given that I had to just revert a patch in the recent stable releases
> > that didn't get enough time to "bake" in Linus's tree (or in -next), I
> > figured it was worth discussing some possible changes with how "fast" I
> > pick up patches for stable releases.
> >
> > So, how about this proposal:
> >
> > - I will wait for a -rc to come out with the patch in it before putting
> >   it into a stable release, unless:
> > - the maintainer ACKs it, or sends it directly (like DaveM does
> >   for networking patches)
> > - I have seen enough discussion about a patch to show that it
> >   really does fix something / is good / doesn't cause problems.
> > - obviously safe, i.e. "add a device id" type thing.
> >
> > Given that we have -rc releases every week, except for the initial -rc1
> > release, I don't think this will really cause any major delays.
> >
> > Also, now that we are about to head into my busy "travel season", odds
> > are, I'll be at least a week behind anyway, so this would probably start
> > happening without an "official" change.  It's been a boring summer, I've
> > been able to keep up with the stable stuff really easily, causing
> > problems like this :)
> >
> > Objections?  Comments?
> 
> I like this overall.  The only thing I might change is "wait for -rc2"
> for patches tagged with CC: stable that go in during the merge window.
>  It seems those are the ones that tend to bite us.

Maintainers can always tag their patches to have me hold off until -rc2
for that.

And, given the huge number of patches for stable that come in during
the -rc1 merge window, there's no way I can get to them all before -rc2
comes out, so this will probably end up being the case for most of them
anyway.

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 6:40 PM, Greg KH  wrote:
> Hi all,
>
> Given that I had to just revert a patch in the recent stable releases
> that didn't get enough time to "bake" in Linus's tree (or in -next), I
> figured it was worth discussing some possible changes with how "fast" I
> pick up patches for stable releases.
>
> So, how about this proposal:
>
> - I will wait for a -rc to come out with the patch in it before putting
>   it into a stable release, unless:
> - the maintainer ACKs it, or sends it directly (like DaveM does
>   for networking patches)
> - I have seen enough discussion about a patch to show that it
>   really does fix something / is good / doesn't cause problems.
> - obviously safe, i.e. "add a device id" type thing.
>
> Given that we have -rc releases every week, except for the initial -rc1
> release, I don't think this will really cause any major delays.
>
> Also, now that we are about to head into my busy "travel season", odds
> are, I'll be at least a week behind anyway, so this would probably start
> happening without an "official" change.  It's been a boring summer, I've
> been able to keep up with the stable stuff really easily, causing
> problems like this :)
>
> Objections?  Comments?

I like this overall.  The only thing I might change is "wait for -rc2"
for patches tagged with CC: stable that go in during the merge window.
 It seems those are the ones that tend to bite us.

Overall though, I think waiting for the next -rc is a good balance.

josh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 04:11:17PM -0700, Guenter Roeck wrote:
> It would be even better if you could find the time to push -rc releases
> into the stable repository before you accept patches into a stable branch.

What do you mean by this?

The git tree?  I could do that, but when I have to drop a patch, it
would cause a mess if I had to always go forwards.

I could do branches for -rc releases, that end up as the
"end-of-the-line", and I create the next .y release on top of the
previous one, not the -rc release.

That's kind of what I do "internally" when I create the -rc releases in
the first place, but I just delete those throw-away trees, and never
push them publicly anywhere.

Does it really help anyone to do this, except for some automated
testing?  Doesn't the -rc patch work good enough for that?

> I am now running my test suite on stable/master, so that would give it
> some time to catch new problems before they make their way into a stable
> branch.

I don't understand what you mean here.

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Wed, Aug 21, 2013 at 12:58:15AM +0200, Willy Tarreau wrote:
> Hi Greg,
> 
> On Tue, Aug 20, 2013 at 03:40:32PM -0700, Greg KH wrote:
> > Hi all,
> > 
> > Given that I had to just revert a patch in the recent stable releases
> > that didn't get enough time to "bake" in Linus's tree (or in -next), I
> > figured it was worth discussing some possible changes with how "fast" I
> > pick up patches for stable releases.
> > 
> > So, how about this proposal:
> > 
> > - I will wait for a -rc to come out with the patch in it before putting
> >   it into a stable release, unless:
> > - the maintainer ACKs it, or sends it directly (like DaveM does
> >   for networking patches)
> > - I have seen enough discussion about a patch to show that it
> >   really does fix something / is good / doesn't cause problems.
> > - obviously safe, i.e. "add a device id" type thing.
> > 
> > Given that we have -rc releases every week, except for the initial -rc1
> > release, I don't think this will really cause any major delays.
> 
> In the last discussion you initiated on the subject, I proposed something
> even more conservative which was the same as above except instead of
> "wait for a -rc", it was "wait for rc1 after a full release containing
> the patch", unless one of the conditions you proposed, or another one
> which would be a tag "urgent" or something like this in the patch.

Waiting 3 months is too long, in my opinion, sorry.

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 03:40:32PM -0700, Greg KH wrote:
> Hi all,
> 
> Given that I had to just revert a patch in the recent stable releases
> that didn't get enough time to "bake" in Linus's tree (or in -next), I
> figured it was worth discussing some possible changes with how "fast" I
> pick up patches for stable releases.
> 
> So, how about this proposal:
> 
> - I will wait for a -rc to come out with the patch in it before putting
>   it into a stable release, unless:
>   - the maintainer ACKs it, or sends it directly (like DaveM does
> for networking patches)
>   - I have seen enough discussion about a patch to show that it
> really does fix something / is good / doesn't cause problems.
>   - obviously safe, i.e. "add a device id" type thing.
> 
> Given that we have -rc releases every week, except for the initial -rc1
> release, I don't think this will really cause any major delays.
> 
> Also, now that we are about to head into my busy "travel season", odds
> are, I'll be at least a week behind anyway, so this would probably start
> happening without an "official" change.  It's been a boring summer, I've
> been able to keep up with the stable stuff really easily, causing
> problems like this :)
> 
> Objections?  Comments?
> 
Makes sense to me.

It would be even better if you could find the time to push -rc releases
into the stable repository before you accept patches into a stable branch.
I am now running my test suite on stable/master, so that would give it
some time to catch new problems before they make their way into a stable
branch.

Guenter
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 04:04:00PM -0700, Nicholas A. Bellinger wrote:
> On Tue, 2013-08-20 at 15:40 -0700, Greg KH wrote:
> > Hi all,
> > 
> > Given that I had to just revert a patch in the recent stable releases
> > that didn't get enough time to "bake" in Linus's tree (or in -next), I
> > figured it was worth discussing some possible changes with how "fast" I
> > pick up patches for stable releases.
> > 
> > So, how about this proposal:
> > 
> > - I will wait for a -rc to come out with the patch in it before putting
> >   it into a stable release, unless:
> > - the maintainer ACKs it, or sends it directly (like DaveM does
> >   for networking patches)
> > - I have seen enough discussion about a patch to show that it
> >   really does fix something / is good / doesn't cause problems.
> > - obviously safe, i.e. "add a device id" type thing.
> > 
> > Given that we have -rc releases every week, except for the initial -rc1
> > release, I don't think this will really cause any major delays.
> > 
> > Also, now that we are about to head into my busy "travel season", odds
> > are, I'll be at least a week behind anyway, so this would probably start
> > happening without an "official" change.  It's been a boring summer, I've
> > been able to keep up with the stable stuff really easily, causing
> > problems like this :)
> > 
> > Objections?  Comments?
> 
> Sounds like a reasonable tradeoff, as long as the 'please include this
> to stable ASAP' latency for critical fixes does not end up being too
> large..

Given the current number of those that I get are relativly small, it
shouldn't be that big of a deal.

All this really means is that I get a week vacation now, and then I'll
pick back up with stable releases :)

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

Hi Greg,

On Tue, Aug 20, 2013 at 03:40:32PM -0700, Greg KH wrote:
> Hi all,
> 
> Given that I had to just revert a patch in the recent stable releases
> that didn't get enough time to "bake" in Linus's tree (or in -next), I
> figured it was worth discussing some possible changes with how "fast" I
> pick up patches for stable releases.
> 
> So, how about this proposal:
> 
> - I will wait for a -rc to come out with the patch in it before putting
>   it into a stable release, unless:
>   - the maintainer ACKs it, or sends it directly (like DaveM does
> for networking patches)
>   - I have seen enough discussion about a patch to show that it
> really does fix something / is good / doesn't cause problems.
>   - obviously safe, i.e. "add a device id" type thing.
> 
> Given that we have -rc releases every week, except for the initial -rc1
> release, I don't think this will really cause any major delays.

In the last discussion you initiated on the subject, I proposed something
even more conservative which was the same as above except instead of
"wait for a -rc", it was "wait for rc1 after a full release containing
the patch", unless one of the conditions you proposed, or another one
which would be a tag "urgent" or something like this in the patch. I
tend to think it will be hard at the beginning but will quickly motivate
maintainers to care more about their fixes flow towards -stable depending
on their severity.

Best regards,
Willy

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

2013-08-20 Thread Nicholas A. Bellinger

On Tue, 2013-08-20 at 15:40 -0700, Greg KH wrote:
> Hi all,
> 
> Given that I had to just revert a patch in the recent stable releases
> that didn't get enough time to "bake" in Linus's tree (or in -next), I
> figured it was worth discussing some possible changes with how "fast" I
> pick up patches for stable releases.
> 
> So, how about this proposal:
> 
> - I will wait for a -rc to come out with the patch in it before putting
>   it into a stable release, unless:
>   - the maintainer ACKs it, or sends it directly (like DaveM does
> for networking patches)
>   - I have seen enough discussion about a patch to show that it
> really does fix something / is good / doesn't cause problems.
>   - obviously safe, i.e. "add a device id" type thing.
> 
> Given that we have -rc releases every week, except for the initial -rc1
> release, I don't think this will really cause any major delays.
> 
> Also, now that we are about to head into my busy "travel season", odds
> are, I'll be at least a week behind anyway, so this would probably start
> happening without an "official" change.  It's been a boring summer, I've
> been able to keep up with the stable stuff really easily, causing
> problems like this :)
> 
> Objections?  Comments?

Sounds like a reasonable tradeoff, as long as the 'please include this
to stable ASAP' latency for critical fixes does not end up being too
large..

--nab

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

2013-08-20 Thread Nicholas A. Bellinger

On Tue, 2013-08-20 at 15:40 -0700, Greg KH wrote:
 Hi all,
 
 Given that I had to just revert a patch in the recent stable releases
 that didn't get enough time to bake in Linus's tree (or in -next), I
 figured it was worth discussing some possible changes with how fast I
 pick up patches for stable releases.
 
 So, how about this proposal:
 
 - I will wait for a -rc to come out with the patch in it before putting
   it into a stable release, unless:
   - the maintainer ACKs it, or sends it directly (like DaveM does
 for networking patches)
   - I have seen enough discussion about a patch to show that it
 really does fix something / is good / doesn't cause problems.
   - obviously safe, i.e. add a device id type thing.
 
 Given that we have -rc releases every week, except for the initial -rc1
 release, I don't think this will really cause any major delays.
 
 Also, now that we are about to head into my busy travel season, odds
 are, I'll be at least a week behind anyway, so this would probably start
 happening without an official change.  It's been a boring summer, I've
 been able to keep up with the stable stuff really easily, causing
 problems like this :)
 
 Objections?  Comments?

Sounds like a reasonable tradeoff, as long as the 'please include this
to stable ASAP' latency for critical fixes does not end up being too
large..

--nab

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

Hi Greg,

On Tue, Aug 20, 2013 at 03:40:32PM -0700, Greg KH wrote:
 Hi all,
 
 Given that I had to just revert a patch in the recent stable releases
 that didn't get enough time to bake in Linus's tree (or in -next), I
 figured it was worth discussing some possible changes with how fast I
 pick up patches for stable releases.
 
 So, how about this proposal:
 
 - I will wait for a -rc to come out with the patch in it before putting
   it into a stable release, unless:
   - the maintainer ACKs it, or sends it directly (like DaveM does
 for networking patches)
   - I have seen enough discussion about a patch to show that it
 really does fix something / is good / doesn't cause problems.
   - obviously safe, i.e. add a device id type thing.
 
 Given that we have -rc releases every week, except for the initial -rc1
 release, I don't think this will really cause any major delays.

In the last discussion you initiated on the subject, I proposed something
even more conservative which was the same as above except instead of
wait for a -rc, it was wait for rc1 after a full release containing
the patch, unless one of the conditions you proposed, or another one
which would be a tag urgent or something like this in the patch. I
tend to think it will be hard at the beginning but will quickly motivate
maintainers to care more about their fixes flow towards -stable depending
on their severity.

Best regards,
Willy

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 04:04:00PM -0700, Nicholas A. Bellinger wrote:
 On Tue, 2013-08-20 at 15:40 -0700, Greg KH wrote:
  Hi all,
  
  Given that I had to just revert a patch in the recent stable releases
  that didn't get enough time to bake in Linus's tree (or in -next), I
  figured it was worth discussing some possible changes with how fast I
  pick up patches for stable releases.
  
  So, how about this proposal:
  
  - I will wait for a -rc to come out with the patch in it before putting
it into a stable release, unless:
  - the maintainer ACKs it, or sends it directly (like DaveM does
for networking patches)
  - I have seen enough discussion about a patch to show that it
really does fix something / is good / doesn't cause problems.
  - obviously safe, i.e. add a device id type thing.
  
  Given that we have -rc releases every week, except for the initial -rc1
  release, I don't think this will really cause any major delays.
  
  Also, now that we are about to head into my busy travel season, odds
  are, I'll be at least a week behind anyway, so this would probably start
  happening without an official change.  It's been a boring summer, I've
  been able to keep up with the stable stuff really easily, causing
  problems like this :)
  
  Objections?  Comments?
 
 Sounds like a reasonable tradeoff, as long as the 'please include this
 to stable ASAP' latency for critical fixes does not end up being too
 large..

Given the current number of those that I get are relativly small, it
shouldn't be that big of a deal.

All this really means is that I get a week vacation now, and then I'll
pick back up with stable releases :)

thanks,

greg k-h
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 03:40:32PM -0700, Greg KH wrote:
 Hi all,
 
 Given that I had to just revert a patch in the recent stable releases
 that didn't get enough time to bake in Linus's tree (or in -next), I
 figured it was worth discussing some possible changes with how fast I
 pick up patches for stable releases.
 
 So, how about this proposal:
 
 - I will wait for a -rc to come out with the patch in it before putting
   it into a stable release, unless:
   - the maintainer ACKs it, or sends it directly (like DaveM does
 for networking patches)
   - I have seen enough discussion about a patch to show that it
 really does fix something / is good / doesn't cause problems.
   - obviously safe, i.e. add a device id type thing.
 
 Given that we have -rc releases every week, except for the initial -rc1
 release, I don't think this will really cause any major delays.
 
 Also, now that we are about to head into my busy travel season, odds
 are, I'll be at least a week behind anyway, so this would probably start
 happening without an official change.  It's been a boring summer, I've
 been able to keep up with the stable stuff really easily, causing
 problems like this :)
 
 Objections?  Comments?
 
Makes sense to me.

It would be even better if you could find the time to push -rc releases
into the stable repository before you accept patches into a stable branch.
I am now running my test suite on stable/master, so that would give it
some time to catch new problems before they make their way into a stable
branch.

Guenter
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Wed, Aug 21, 2013 at 12:58:15AM +0200, Willy Tarreau wrote:
 Hi Greg,
 
 On Tue, Aug 20, 2013 at 03:40:32PM -0700, Greg KH wrote:
  Hi all,
  
  Given that I had to just revert a patch in the recent stable releases
  that didn't get enough time to bake in Linus's tree (or in -next), I
  figured it was worth discussing some possible changes with how fast I
  pick up patches for stable releases.
  
  So, how about this proposal:
  
  - I will wait for a -rc to come out with the patch in it before putting
it into a stable release, unless:
  - the maintainer ACKs it, or sends it directly (like DaveM does
for networking patches)
  - I have seen enough discussion about a patch to show that it
really does fix something / is good / doesn't cause problems.
  - obviously safe, i.e. add a device id type thing.
  
  Given that we have -rc releases every week, except for the initial -rc1
  release, I don't think this will really cause any major delays.
 
 In the last discussion you initiated on the subject, I proposed something
 even more conservative which was the same as above except instead of
 wait for a -rc, it was wait for rc1 after a full release containing
 the patch, unless one of the conditions you proposed, or another one
 which would be a tag urgent or something like this in the patch.

Waiting 3 months is too long, in my opinion, sorry.

thanks,

greg k-h
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 04:11:17PM -0700, Guenter Roeck wrote:
 It would be even better if you could find the time to push -rc releases
 into the stable repository before you accept patches into a stable branch.

What do you mean by this?

The git tree?  I could do that, but when I have to drop a patch, it
would cause a mess if I had to always go forwards.

I could do branches for -rc releases, that end up as the
end-of-the-line, and I create the next .y release on top of the
previous one, not the -rc release.

That's kind of what I do internally when I create the -rc releases in
the first place, but I just delete those throw-away trees, and never
push them publicly anywhere.

Does it really help anyone to do this, except for some automated
testing?  Doesn't the -rc patch work good enough for that?

 I am now running my test suite on stable/master, so that would give it
 some time to catch new problems before they make their way into a stable
 branch.

I don't understand what you mean here.

thanks,

greg k-h
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 6:40 PM, Greg KH gre...@linuxfoundation.org wrote:
 Hi all,

 Given that I had to just revert a patch in the recent stable releases
 that didn't get enough time to bake in Linus's tree (or in -next), I
 figured it was worth discussing some possible changes with how fast I
 pick up patches for stable releases.

 So, how about this proposal:

 - I will wait for a -rc to come out with the patch in it before putting
   it into a stable release, unless:
 - the maintainer ACKs it, or sends it directly (like DaveM does
   for networking patches)
 - I have seen enough discussion about a patch to show that it
   really does fix something / is good / doesn't cause problems.
 - obviously safe, i.e. add a device id type thing.

 Given that we have -rc releases every week, except for the initial -rc1
 release, I don't think this will really cause any major delays.

 Also, now that we are about to head into my busy travel season, odds
 are, I'll be at least a week behind anyway, so this would probably start
 happening without an official change.  It's been a boring summer, I've
 been able to keep up with the stable stuff really easily, causing
 problems like this :)

 Objections?  Comments?

I like this overall.  The only thing I might change is wait for -rc2
for patches tagged with CC: stable that go in during the merge window.
 It seems those are the ones that tend to bite us.

Overall though, I think waiting for the next -rc is a good balance.

josh
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 07:40:49PM -0400, Josh Boyer wrote:
 On Tue, Aug 20, 2013 at 6:40 PM, Greg KH gre...@linuxfoundation.org wrote:
  Hi all,
 
  Given that I had to just revert a patch in the recent stable releases
  that didn't get enough time to bake in Linus's tree (or in -next), I
  figured it was worth discussing some possible changes with how fast I
  pick up patches for stable releases.
 
  So, how about this proposal:
 
  - I will wait for a -rc to come out with the patch in it before putting
it into a stable release, unless:
  - the maintainer ACKs it, or sends it directly (like DaveM does
for networking patches)
  - I have seen enough discussion about a patch to show that it
really does fix something / is good / doesn't cause problems.
  - obviously safe, i.e. add a device id type thing.
 
  Given that we have -rc releases every week, except for the initial -rc1
  release, I don't think this will really cause any major delays.
 
  Also, now that we are about to head into my busy travel season, odds
  are, I'll be at least a week behind anyway, so this would probably start
  happening without an official change.  It's been a boring summer, I've
  been able to keep up with the stable stuff really easily, causing
  problems like this :)
 
  Objections?  Comments?
 
 I like this overall.  The only thing I might change is wait for -rc2
 for patches tagged with CC: stable that go in during the merge window.
  It seems those are the ones that tend to bite us.

Maintainers can always tag their patches to have me hold off until -rc2
for that.

And, given the huge number of patches for stable that come in during
the -rc1 merge window, there's no way I can get to them all before -rc2
comes out, so this will probably end up being the case for most of them
anyway.

thanks,

greg k-h
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 04:12:32PM -0700, Greg KH wrote:
 On Wed, Aug 21, 2013 at 12:58:15AM +0200, Willy Tarreau wrote:
  Hi Greg,
  
  On Tue, Aug 20, 2013 at 03:40:32PM -0700, Greg KH wrote:
   Hi all,
   
   Given that I had to just revert a patch in the recent stable releases
   that didn't get enough time to bake in Linus's tree (or in -next), I
   figured it was worth discussing some possible changes with how fast I
   pick up patches for stable releases.
   
   So, how about this proposal:
   
   - I will wait for a -rc to come out with the patch in it before putting
 it into a stable release, unless:
 - the maintainer ACKs it, or sends it directly (like DaveM does
   for networking patches)
 - I have seen enough discussion about a patch to show that it
   really does fix something / is good / doesn't cause problems.
 - obviously safe, i.e. add a device id type thing.
   
   Given that we have -rc releases every week, except for the initial -rc1
   release, I don't think this will really cause any major delays.
  
  In the last discussion you initiated on the subject, I proposed something
  even more conservative which was the same as above except instead of
  wait for a -rc, it was wait for rc1 after a full release containing
  the patch, unless one of the conditions you proposed, or another one
  which would be a tag urgent or something like this in the patch.
 
 Waiting 3 months is too long, in my opinion, sorry.
 
I definitely agree.

Guenter
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 04:17:43PM -0700, Greg KH wrote:
 On Tue, Aug 20, 2013 at 04:11:17PM -0700, Guenter Roeck wrote:
  It would be even better if you could find the time to push -rc releases
  into the stable repository before you accept patches into a stable branch.
 
 What do you mean by this?

 The git tree?  I could do that, but when I have to drop a patch, it
 would cause a mess if I had to always go forwards.
 
That is not what I mean. I referred to the master branch of the stable
repository, which you normally update to the most recent -rc when you
publish a new -stable release.

 I could do branches for -rc releases, that end up as the
 end-of-the-line, and I create the next .y release on top of the
 previous one, not the -rc release.
 
I would not ask for creating any new branches. The existing ones are good
enough.

 That's kind of what I do internally when I create the -rc releases in
 the first place, but I just delete those throw-away trees, and never
 push them publicly anywhere.
 
 Does it really help anyone to do this, except for some automated
 testing?  Doesn't the -rc patch work good enough for that?
 
  I am now running my test suite on stable/master, so that would give it
  some time to catch new problems before they make their way into a stable
  branch.
 
 I don't understand what you mean here.
 
The master branch of linux-stable.git on kernel.org currently points to
v3.11-rc5. All I asked for is to update it to the latest -rc prior to
accepting patches from this -rc into a stable release.

I am running my test suite on linux-stable:master to have a reference
and comparison against the various -stable branches. As soon as you update
the master branch of linux-stable.git (eg after you push v3.11-rc6 into it),
the test suite will automatically run on it.

Obviously I could track linux-kernel.git itself instead, but there is much more
activity on it and I am really only interested in the most recent tagged 
version.

Hope that describes it better.

Guenter
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 05:11:11PM -0700, Guenter Roeck wrote:
 On Tue, Aug 20, 2013 at 04:17:43PM -0700, Greg KH wrote:
  On Tue, Aug 20, 2013 at 04:11:17PM -0700, Guenter Roeck wrote:
   It would be even better if you could find the time to push -rc releases
   into the stable repository before you accept patches into a stable branch.
  
  What do you mean by this?
 
  The git tree?  I could do that, but when I have to drop a patch, it
  would cause a mess if I had to always go forwards.
  
 That is not what I mean. I referred to the master branch of the stable
 repository, which you normally update to the most recent -rc when you
 publish a new -stable release.

Ah, ok, that makes sense.  I've updated it to 3.11-rc6 now.  Sorry for
the confusion.

greg k-h
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 7:57 PM, Greg KH gre...@linuxfoundation.org wrote:
 On Tue, Aug 20, 2013 at 07:40:49PM -0400, Josh Boyer wrote:
 On Tue, Aug 20, 2013 at 6:40 PM, Greg KH gre...@linuxfoundation.org wrote:
  Hi all,
 
  Given that I had to just revert a patch in the recent stable releases
  that didn't get enough time to bake in Linus's tree (or in -next), I
  figured it was worth discussing some possible changes with how fast I
  pick up patches for stable releases.
 
  So, how about this proposal:
 
  - I will wait for a -rc to come out with the patch in it before putting
it into a stable release, unless:
  - the maintainer ACKs it, or sends it directly (like DaveM does
for networking patches)
  - I have seen enough discussion about a patch to show that it
really does fix something / is good / doesn't cause problems.
  - obviously safe, i.e. add a device id type thing.
 
  Given that we have -rc releases every week, except for the initial -rc1
  release, I don't think this will really cause any major delays.
 
  Also, now that we are about to head into my busy travel season, odds
  are, I'll be at least a week behind anyway, so this would probably start
  happening without an official change.  It's been a boring summer, I've
  been able to keep up with the stable stuff really easily, causing
  problems like this :)
 
  Objections?  Comments?

 I like this overall.  The only thing I might change is wait for -rc2
 for patches tagged with CC: stable that go in during the merge window.
  It seems those are the ones that tend to bite us.

 Maintainers can always tag their patches to have me hold off until -rc2
 for that.

They can (not immediately sure how though?), but they don't with the
exception of the few that don't tag at all and send you patches in
bundles.  I mean, that's what the huge thread about the stable trees
that hopefully leads to a conversation at KS is about, right?

 And, given the huge number of patches for stable that come in during
 the -rc1 merge window, there's no way I can get to them all before -rc2
 comes out, so this will probably end up being the case for most of them
 anyway.

Sheer numbers forcing some to be pushed out isn't really that great
when the problem is spread across the whole window.  I'm not saying
you aren't correct that a lot of them don't get pushed to stable
releases until post -rc2, but at that point you're playing catch up.
I was suggesting just sitting on them until -rc2 entirely.  Basically,
don't release a stable kernel until the next version is at -rc2.

Let me phrase this as a question instead.  Is there something we can
do to help catch the patches that get sucked into stable during the
merge window and then wind up causing issues and reverted/fixed after
things settle down in the -rc releases?

Reviewing the patches submitted for those stable kernels isn't
necessarily sufficient, since the issues I'm concerned about aren't
spotted until after the release anyway.  It's a two-fold problem.  The
first is one we're never going to fix in that people are human and
make mistakes and sometimes patches don't get tested well enough
before they're tagged for stable.  The second issue is a combination
of timing and volume.  I doubt we're going to fix the volume since we
keep touting the increasing change rate as a sign of success, but
could we fix the timing by forcing things to bake more in the -rc
releases?

I'm offering to help in whatever way you think is best.  It's your
workflow (and sanity) that are the most impacted.  However, I share
the pain whenever something breaks in stable through the wonderful
place that is Fedora bugzilla so I'm looking for ways to reduce that.

josh
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 08:41:23PM -0400, Josh Boyer wrote:
 On Tue, Aug 20, 2013 at 7:57 PM, Greg KH gre...@linuxfoundation.org wrote:
  I like this overall.  The only thing I might change is wait for -rc2
  for patches tagged with CC: stable that go in during the merge window.
   It seems those are the ones that tend to bite us.
 
  Maintainers can always tag their patches to have me hold off until -rc2
  for that.
 
 They can (not immediately sure how though?)

Some do:
Cc: stable sta...@vger.kernel.org # after -rc5 is out
or
Cc: stable sta...@vger.kernel.org # wait a -rc cycle
or
Cc: stable sta...@vger.kernel.org # wait a few weeks to bake

 , but they don't with the
 exception of the few that don't tag at all and send you patches in
 bundles.  I mean, that's what the huge thread about the stable trees
 that hopefully leads to a conversation at KS is about, right?

Hopefully, yes, but I don't know about that yet.

 Let me phrase this as a question instead.  Is there something we can
 do to help catch the patches that get sucked into stable during the
 merge window and then wind up causing issues and reverted/fixed after
 things settle down in the -rc releases?

Test linux-next and Linus's tree-of-the-day better.  If problems happen,
and a patch has a cc: stable@ on it, let stable@ know about it.

 I'm offering to help in whatever way you think is best.  It's your
 workflow (and sanity) that are the most impacted.  However, I share
 the pain whenever something breaks in stable through the wonderful
 place that is Fedora bugzilla so I'm looking for ways to reduce that.

Letting me know when something breaks is always good as well.  Right now
that doesn't seem to happen much, so either not much is breaking, or I'm
just not told about it, I don't know which.

thanks,

greg k-h
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 8:49 PM, Greg KH gre...@linuxfoundation.org wrote:
 On Tue, Aug 20, 2013 at 08:41:23PM -0400, Josh Boyer wrote:
 On Tue, Aug 20, 2013 at 7:57 PM, Greg KH gre...@linuxfoundation.org wrote:
  I like this overall.  The only thing I might change is wait for -rc2
  for patches tagged with CC: stable that go in during the merge window.
   It seems those are the ones that tend to bite us.
 
  Maintainers can always tag their patches to have me hold off until -rc2
  for that.

 They can (not immediately sure how though?)

 Some do:
 Cc: stable sta...@vger.kernel.org # after -rc5 is out
 or
 Cc: stable sta...@vger.kernel.org # wait a -rc cycle
 or
 Cc: stable sta...@vger.kernel.org # wait a few weeks to bake

 , but they don't with the
 exception of the few that don't tag at all and send you patches in
 bundles.  I mean, that's what the huge thread about the stable trees
 that hopefully leads to a conversation at KS is about, right?

 Hopefully, yes, but I don't know about that yet.

 Let me phrase this as a question instead.  Is there something we can
 do to help catch the patches that get sucked into stable during the
 merge window and then wind up causing issues and reverted/fixed after
 things settle down in the -rc releases?

 Test linux-next and Linus's tree-of-the-day better.  If problems happen,
 and a patch has a cc: stable@ on it, let stable@ know about it.

Ah, yes.  I forgot about the look through linux-next for CC: stable.
 That could probably be automated to a degree even.

We already build a Linus kernel in rawhide at least once a day.  Two
flavors even, once with debug options enabled and once without.  The
three of us run it and have an autotest setup going and being
improved, but there is no substitute for wide-spread user testing.
Unfortunately, we have problems getting that with rawhide for various
reasons.

 I'm offering to help in whatever way you think is best.  It's your
 workflow (and sanity) that are the most impacted.  However, I share
 the pain whenever something breaks in stable through the wonderful
 place that is Fedora bugzilla so I'm looking for ways to reduce that.

 Letting me know when something breaks is always good as well.  Right now
 that doesn't seem to happen much, so either not much is breaking, or I'm
 just not told about it, I don't know which.

There's no need to reply specifically to these, but off the top of my
head just for 3.10.y:

brcmsmac explodes in 3.10.{6,7,8,9}.  I think you know about that one
already and Felix and Arend are working on it.

Suspend/Resume is broken on a variety of Thinkpad T400 and T500
machines in 3.10.  This was true with 3.10.0 afaik.  Current thinking
is that it's related to the Intel mei/mei_me driver(s).  Blacklisting
those seems to fix things for a number of users.  There are patches in
3.11-rcX, but the fix highlighted doesn't fix it.

There were various i915 backlight issues reported with 3.10.3/4.  I
believe they were fixed with 3.10.5.

USB storage was broken because of the mishandled VPD stuff.  Fixed in 3.10.7.

I'm aware I'm reporting issues that you either already knew about or
were already fixed.  The problem we have is that we roll out a new
stable release and then we get bug reports for 2 weeks because not
everyone updates as frequently as stable releases, etc.  So something
that may seem to impact a small number of users at the time winds up
actually impacting lots of users once it rolls out in a distro.  As
far as I know, Fedora is possibly the only distro actually pushing
stable release kernels out on a normal basis.  I'd love to be wrong on
that point.

In the future, if we can get the information from the end user in
time, I'll be happy to forward issues that aren't already reported
onwards.  Or if you still want to hear about it, I can chime in on the
existing threads with bugzilla numbers.  I'm also willing to do a
monthly patches we're carrying not in stable report if people find
that helpful.  I'll likely be doing that within Fedora already and I'm
happy to send it to stable@, even if those patches aren't exactly
stable-rules matching.  We did that when kernel.org went down and it
helped then, just not sure how much it would help now or if people
care.

josh
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 09:03:12PM -0400, Josh Boyer wrote:
[ ... ]

 
 Suspend/Resume is broken on a variety of Thinkpad T400 and T500
 machines in 3.10.  This was true with 3.10.0 afaik.  Current thinking
 is that it's related to the Intel mei/mei_me driver(s).  Blacklisting

Reminds me ... I had to disable MEI on my servers at home because 
it caused some instability (random system hangs) and the watchdog
didn't work. I thought that was only me, but maybe not.

Guenter
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes

On Tue, Aug 20, 2013 at 04:12:32PM -0700, Greg KH wrote:
 On Wed, Aug 21, 2013 at 12:58:15AM +0200, Willy Tarreau wrote:
  Hi Greg,
  
  On Tue, Aug 20, 2013 at 03:40:32PM -0700, Greg KH wrote:
   Hi all,
   
   Given that I had to just revert a patch in the recent stable releases
   that didn't get enough time to bake in Linus's tree (or in -next), I
   figured it was worth discussing some possible changes with how fast I
   pick up patches for stable releases.
   
   So, how about this proposal:
   
   - I will wait for a -rc to come out with the patch in it before putting
 it into a stable release, unless:
 - the maintainer ACKs it, or sends it directly (like DaveM does
   for networking patches)
 - I have seen enough discussion about a patch to show that it
   really does fix something / is good / doesn't cause problems.
 - obviously safe, i.e. add a device id type thing.
   
   Given that we have -rc releases every week, except for the initial -rc1
   release, I don't think this will really cause any major delays.
  
  In the last discussion you initiated on the subject, I proposed something
  even more conservative which was the same as above except instead of
  wait for a -rc, it was wait for rc1 after a full release containing
  the patch, unless one of the conditions you proposed, or another one
  which would be a tag urgent or something like this in the patch.
 
 Waiting 3 months is too long, in my opinion, sorry.

I meant only for the non-important ones. Their authors will qualify the
ones that are important and must not wait. The same way as now many patches
are correctly tagged cc: stable, I suspect that we could end up with maybe
80% of patches tagged as must not wait, and the remaining 20% would indeed
wait up to 3 months, but if their authors think they should wait maybe we
should trust them.

Willy

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Proposed stable release changes