Re: [Ksummit-discuss] bug-introducing patches

[Willy Tarreau]

On Sat, Jul 14, 2018 at 11:09:13PM +0200, Pavel Machek wrote:
> > For anyone interested in making sure that obscure (whatever that means)
> > drivers are tested for stable releases, but does not want to spend time on 
> > it,
> > all I can recommend is to implement qemu support for it and let me know,
> > and I'll be happy to add a respective test to my test farm.
> 
> Umm. Yes, qemu support for every driver would be nice, but will not happen.

Well, I would argue that driver code changes much less than core code
between kernel versions, and that most of the changes in drivers are
mostly infrastructure changes. Drivers don't evolve much in general,
they are written, tested, merged, they receive fixes and then they
only receive infrastructure changes that touch all drivers in the same
category.

When you backport fixes to drivers, it is very common that the code
looks almost the same between even a very old kernel and mainline, and
when not, the adaptations generally look quite straightforward, and if
not it means the driver changed significantly and in this case we don't
backport the fix as we don't even know if it is relevant.

I've always had much more difficulties backporting fixes under the arch/
subdir where stuff changes all the time. Sometimes a patch applies but
doesn't even compile. I learned not to play black magic in this area
because some patches are subtle and if the code changed you often need
the author and/or maintainers to double-check. Some subsystems like KVM
improve a lot over time and are difficult to backport to as well, and
even if you manage to properly backport a fix you're uncertain how to
verify you backported it well. Similarly you don't want to improvise
yourself the backporter of the year in this area.

Drivers are often OK and are the ones harder to test, so in the end
you don't miss much by your limited ability to test a backport there.

What I can certainly say as a stable kernel user is that the regression
rate is so low compared to the fix rate that I never have any problem
upgrading to a more recent version in the same branch, because the
number of problems that will be fixed is much higher than the risk of
a single regression.

As Guenter says, we can always improve, but the most important is to
deliver fixes in a timely manner. When you see that any LTS branch
accumulates around 5000 fixes over time, you understand that any
single new kernel being released contains around 5000 bugs left to be
found. Fixing them quickly is much more important to me (as a user)
than ensuring that I will not reach 5001 by inheriting from a poorly
tested backport.

My hope is that thanks to all the automated testing in place we can
further accelerate the backport rate so that a stable kernel reaches
in 2 months the level of quality that we previously used to reach only
after one year. And I think we're already about there, as both 4.4.x
and 4.9.x in their early versions (x < 10) were already very good for
various use cases. 4.17.5 I'm using on this PC looks pretty slick as
well. Overall it means that we can provide a clean upgrade path for
users so that they don't stick to bogus or insecure kernels by fear
of upgrading. We can always argue that a bug may appear once in a
while but for me while technically this is true, stastistically this
is just FUD and is not relevant to end users' real usage.

Willy

Re: [Ksummit-discuss] bug-introducing patches

[Willy Tarreau]

On Sat, Jul 14, 2018 at 11:09:13PM +0200, Pavel Machek wrote:
> > For anyone interested in making sure that obscure (whatever that means)
> > drivers are tested for stable releases, but does not want to spend time on 
> > it,
> > all I can recommend is to implement qemu support for it and let me know,
> > and I'll be happy to add a respective test to my test farm.
> 
> Umm. Yes, qemu support for every driver would be nice, but will not happen.

Well, I would argue that driver code changes much less than core code
between kernel versions, and that most of the changes in drivers are
mostly infrastructure changes. Drivers don't evolve much in general,
they are written, tested, merged, they receive fixes and then they
only receive infrastructure changes that touch all drivers in the same
category.

When you backport fixes to drivers, it is very common that the code
looks almost the same between even a very old kernel and mainline, and
when not, the adaptations generally look quite straightforward, and if
not it means the driver changed significantly and in this case we don't
backport the fix as we don't even know if it is relevant.

I've always had much more difficulties backporting fixes under the arch/
subdir where stuff changes all the time. Sometimes a patch applies but
doesn't even compile. I learned not to play black magic in this area
because some patches are subtle and if the code changed you often need
the author and/or maintainers to double-check. Some subsystems like KVM
improve a lot over time and are difficult to backport to as well, and
even if you manage to properly backport a fix you're uncertain how to
verify you backported it well. Similarly you don't want to improvise
yourself the backporter of the year in this area.

Drivers are often OK and are the ones harder to test, so in the end
you don't miss much by your limited ability to test a backport there.

What I can certainly say as a stable kernel user is that the regression
rate is so low compared to the fix rate that I never have any problem
upgrading to a more recent version in the same branch, because the
number of problems that will be fixed is much higher than the risk of
a single regression.

As Guenter says, we can always improve, but the most important is to
deliver fixes in a timely manner. When you see that any LTS branch
accumulates around 5000 fixes over time, you understand that any
single new kernel being released contains around 5000 bugs left to be
found. Fixing them quickly is much more important to me (as a user)
than ensuring that I will not reach 5001 by inheriting from a poorly
tested backport.

My hope is that thanks to all the automated testing in place we can
further accelerate the backport rate so that a stable kernel reaches
in 2 months the level of quality that we previously used to reach only
after one year. And I think we're already about there, as both 4.4.x
and 4.9.x in their early versions (x < 10) were already very good for
various use cases. 4.17.5 I'm using on this PC looks pretty slick as
well. Overall it means that we can provide a clean upgrade path for
users so that they don't stick to bogus or insecure kernels by fear
of upgrading. We can always argue that a bug may appear once in a
while but for me while technically this is true, stastistically this
is just FUD and is not relevant to end users' real usage.

Willy

Re: [Ksummit-discuss] bug-introducing patches

[Pavel Machek]

Hi!

> >Well, 0day, kernelci etc... is nice... until the change is in the
> >driver. Most of the kernel are drivers, remember?
> >
> >I don't know. I'd say that if patch is important enough for -stable,
> >there should be someone testing it. For core kernel changes, that can
> >be 0day bot, but for drivers...
> >
> 
> For my part I am just glad that we were able to pick up a fix in xhci code
> just last week, tested or not, from -stable, instead of having to track it
> down ourselves. Similar for many other driver patches which _do_ affect us
> (like the flurry of ext4 patches this week). Granted, there are lots of
> patches which we don't use/need, but even there it is surprising how many
> problems are found with existing testing.
> 
> For anyone interested in making sure that obscure (whatever that means)
> drivers are tested for stable releases, but does not want to spend time on it,
> all I can recommend is to implement qemu support for it and let me know,
> and I'll be happy to add a respective test to my test farm.

Umm. Yes, qemu support for every driver would be nice, but will not happen.

> However, ultimately, stable release candidates are public. Everyone is
> invited to test them. Anyone interested in a specific release and
> driver

Yes, they are public. SubmittingPatches says every patch should be
tested, and that's clearly not happening for -stable. And I'd like
those patch marked such.

> >And problem exists on mainline, too.
> >
> >Hmm. Patch for obscure driver. Wow, nice, is WellKnownName actually
> >using that driver? Aha, no, he is not; he is doing global
> >search, and did not test the patch...
> >
> 
> Ah, like me with the strncpy(x, y, strlen(y)) -> memcpy() replacements
> I did a week or so ago ? You are right, I only compile tested those and
> otherwise trusted my ability to understand C code. If that caused any
> problems, please let me know, and hopefully I'll be able to learn something
> from it.

Yes, such stuff. No, I was not talking about you. I did not want to
give concrete example, but...

# > get_monotonic_boottime() is deprecated, so let's convert this to
# > the simpler ktime_get_boot_ns().
# >
# > Signed-off-by: 
# 
# Have you tested it?
#
...
#   > -curr_boot = timespec_to_ns(_time) * cpus;
# 
# Original code is pretty weird (notice the * cpus), so I'm
# double-checking.

Yes, often you can guess that patch was probably not tested, but it
would be nice to have

Tested: compile

annotation to take away the guesswork. It took me a while an some head
scratching in this concrete example, and it is not first time this
happened.

Best regards,
Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) 
http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html


signature.asc
Description: Digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Pavel Machek]

Hi!

> >Well, 0day, kernelci etc... is nice... until the change is in the
> >driver. Most of the kernel are drivers, remember?
> >
> >I don't know. I'd say that if patch is important enough for -stable,
> >there should be someone testing it. For core kernel changes, that can
> >be 0day bot, but for drivers...
> >
> 
> For my part I am just glad that we were able to pick up a fix in xhci code
> just last week, tested or not, from -stable, instead of having to track it
> down ourselves. Similar for many other driver patches which _do_ affect us
> (like the flurry of ext4 patches this week). Granted, there are lots of
> patches which we don't use/need, but even there it is surprising how many
> problems are found with existing testing.
> 
> For anyone interested in making sure that obscure (whatever that means)
> drivers are tested for stable releases, but does not want to spend time on it,
> all I can recommend is to implement qemu support for it and let me know,
> and I'll be happy to add a respective test to my test farm.

Umm. Yes, qemu support for every driver would be nice, but will not happen.

> However, ultimately, stable release candidates are public. Everyone is
> invited to test them. Anyone interested in a specific release and
> driver

Yes, they are public. SubmittingPatches says every patch should be
tested, and that's clearly not happening for -stable. And I'd like
those patch marked such.

> >And problem exists on mainline, too.
> >
> >Hmm. Patch for obscure driver. Wow, nice, is WellKnownName actually
> >using that driver? Aha, no, he is not; he is doing global
> >search, and did not test the patch...
> >
> 
> Ah, like me with the strncpy(x, y, strlen(y)) -> memcpy() replacements
> I did a week or so ago ? You are right, I only compile tested those and
> otherwise trusted my ability to understand C code. If that caused any
> problems, please let me know, and hopefully I'll be able to learn something
> from it.

Yes, such stuff. No, I was not talking about you. I did not want to
give concrete example, but...

# > get_monotonic_boottime() is deprecated, so let's convert this to
# > the simpler ktime_get_boot_ns().
# >
# > Signed-off-by: 
# 
# Have you tested it?
#
...
#   > -curr_boot = timespec_to_ns(_time) * cpus;
# 
# Original code is pretty weird (notice the * cpus), so I'm
# double-checking.

Yes, often you can guess that patch was probably not tested, but it
would be nice to have

Tested: compile

annotation to take away the guesswork. It took me a while an some head
scratching in this concrete example, and it is not first time this
happened.

Best regards,
Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) 
http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html


signature.asc
Description: Digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Guenter Roeck]

On 07/14/2018 12:47 PM, Pavel Machek wrote:

Hi!


The way I see it, if a commit can get one or two tested-by, it's a good
alternative to a week in -next.


Agreed. Even their own actually. And I'm not kidding. Those who run large
amounts of tests on certain patches could really mention is in tested-by,
as opposed to the most common cases where the code was just regularly
tested.


Actually, it would be cool to get "Tested: no" and "Tested: compile"
tags in the commit mesages. Sometimes it is clear from the context
that patch was not tested (treewide update of time to 64bit), but
sometime it is not.

This is especially problem for -stable, as it seems that lately
patches are backported from new version without any testing.



When I started my own testing some five years ago, most architectures
did not even build in stable releases. At that time, the only tests being
done on stable release candidates were a number of build tests, and most
of the results were ignored.

Today, we have 0day, kernelci, kerneltests, Linaro's LKFT, and more, plus
several merge and boot tests done by individuals. Stable release candidates
are build tested on all supported architectures, with hundreds of

...

Sure, testing is still far from perfect and needs to be improved. However,
requiring that every patch applied to stable releases be tested individually
(where ? on all affected architectures ?) would be the wrong
direction.


Well, 0day, kernelci etc... is nice... until the change is in the
driver. Most of the kernel are drivers, remember?

I don't know. I'd say that if patch is important enough for -stable,
there should be someone testing it. For core kernel changes, that can
be 0day bot, but for drivers...



For my part I am just glad that we were able to pick up a fix in xhci code
just last week, tested or not, from -stable, instead of having to track it
down ourselves. Similar for many other driver patches which _do_ affect us
(like the flurry of ext4 patches this week). Granted, there are lots of
patches which we don't use/need, but even there it is surprising how many
problems are found with existing testing.

For anyone interested in making sure that obscure (whatever that means)
drivers are tested for stable releases, but does not want to spend time on it,
all I can recommend is to implement qemu support for it and let me know,
and I'll be happy to add a respective test to my test farm.

However, ultimately, stable release candidates are public. Everyone is
invited to test them. Anyone interested in a specific release and driver
is invited take stable release candidates and do the necessary testing,
just like I and several others do.


And problem exists on mainline, too.

Hmm. Patch for obscure driver. Wow, nice, is WellKnownName actually
using that driver? Aha, no, he is not; he is doing global
search, and did not test the patch...



Ah, like me with the strncpy(x, y, strlen(y)) -> memcpy() replacements
I did a week or so ago ? You are right, I only compile tested those and
otherwise trusted my ability to understand C code. If that caused any
problems, please let me know, and hopefully I'll be able to learn something
from it.

Really, there are infrastructure changes all the time. Sometimes I am asked
to run a complete test sequence with those, but most of the time they are
applied to -next and people wait for the fallout. That may not be perfect but,
seriously, the only alternative would be to declare that in-kernel APIs
shall not be changed anymore. I don't think that would be feasible.

Thanks,
Guenter

Re: [Ksummit-discuss] bug-introducing patches

[Guenter Roeck]

On 07/14/2018 12:47 PM, Pavel Machek wrote:

Hi!


The way I see it, if a commit can get one or two tested-by, it's a good
alternative to a week in -next.


Agreed. Even their own actually. And I'm not kidding. Those who run large
amounts of tests on certain patches could really mention is in tested-by,
as opposed to the most common cases where the code was just regularly
tested.


Actually, it would be cool to get "Tested: no" and "Tested: compile"
tags in the commit mesages. Sometimes it is clear from the context
that patch was not tested (treewide update of time to 64bit), but
sometime it is not.

This is especially problem for -stable, as it seems that lately
patches are backported from new version without any testing.



When I started my own testing some five years ago, most architectures
did not even build in stable releases. At that time, the only tests being
done on stable release candidates were a number of build tests, and most
of the results were ignored.

Today, we have 0day, kernelci, kerneltests, Linaro's LKFT, and more, plus
several merge and boot tests done by individuals. Stable release candidates
are build tested on all supported architectures, with hundreds of

...

Sure, testing is still far from perfect and needs to be improved. However,
requiring that every patch applied to stable releases be tested individually
(where ? on all affected architectures ?) would be the wrong
direction.


Well, 0day, kernelci etc... is nice... until the change is in the
driver. Most of the kernel are drivers, remember?

I don't know. I'd say that if patch is important enough for -stable,
there should be someone testing it. For core kernel changes, that can
be 0day bot, but for drivers...



For my part I am just glad that we were able to pick up a fix in xhci code
just last week, tested or not, from -stable, instead of having to track it
down ourselves. Similar for many other driver patches which _do_ affect us
(like the flurry of ext4 patches this week). Granted, there are lots of
patches which we don't use/need, but even there it is surprising how many
problems are found with existing testing.

For anyone interested in making sure that obscure (whatever that means)
drivers are tested for stable releases, but does not want to spend time on it,
all I can recommend is to implement qemu support for it and let me know,
and I'll be happy to add a respective test to my test farm.

However, ultimately, stable release candidates are public. Everyone is
invited to test them. Anyone interested in a specific release and driver
is invited take stable release candidates and do the necessary testing,
just like I and several others do.


And problem exists on mainline, too.

Hmm. Patch for obscure driver. Wow, nice, is WellKnownName actually
using that driver? Aha, no, he is not; he is doing global
search, and did not test the patch...



Ah, like me with the strncpy(x, y, strlen(y)) -> memcpy() replacements
I did a week or so ago ? You are right, I only compile tested those and
otherwise trusted my ability to understand C code. If that caused any
problems, please let me know, and hopefully I'll be able to learn something
from it.

Really, there are infrastructure changes all the time. Sometimes I am asked
to run a complete test sequence with those, but most of the time they are
applied to -next and people wait for the fallout. That may not be perfect but,
seriously, the only alternative would be to declare that in-kernel APIs
shall not be changed anymore. I don't think that would be feasible.

Thanks,
Guenter

Re: [Ksummit-discuss] bug-introducing patches

[Pavel Machek]

Hi!

> >>>The way I see it, if a commit can get one or two tested-by, it's a good
> >>>alternative to a week in -next.
> >>
> >>Agreed. Even their own actually. And I'm not kidding. Those who run large
> >>amounts of tests on certain patches could really mention is in tested-by,
> >>as opposed to the most common cases where the code was just regularly
> >>tested.
> >
> >Actually, it would be cool to get "Tested: no" and "Tested: compile"
> >tags in the commit mesages. Sometimes it is clear from the context
> >that patch was not tested (treewide update of time to 64bit), but
> >sometime it is not.
> >
> >This is especially problem for -stable, as it seems that lately
> >patches are backported from new version without any testing.
> 
> 
> When I started my own testing some five years ago, most architectures
> did not even build in stable releases. At that time, the only tests being
> done on stable release candidates were a number of build tests, and most
> of the results were ignored.
> 
> Today, we have 0day, kernelci, kerneltests, Linaro's LKFT, and more, plus
> several merge and boot tests done by individuals. Stable release candidates
> are build tested on all supported architectures, with hundreds of
...
> Sure, testing is still far from perfect and needs to be improved. However,
> requiring that every patch applied to stable releases be tested individually
> (where ? on all affected architectures ?) would be the wrong
>direction.

Well, 0day, kernelci etc... is nice... until the change is in the
driver. Most of the kernel are drivers, remember?

I don't know. I'd say that if patch is important enough for -stable,
there should be someone testing it. For core kernel changes, that can
be 0day bot, but for drivers...

And problem exists on mainline, too.

Hmm. Patch for obscure driver. Wow, nice, is WellKnownName actually
using that driver? Aha, no, he is not; he is doing global
search, and did not test the patch...

Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) 
http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html


signature.asc
Description: Digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Pavel Machek]

Hi!

> >>>The way I see it, if a commit can get one or two tested-by, it's a good
> >>>alternative to a week in -next.
> >>
> >>Agreed. Even their own actually. And I'm not kidding. Those who run large
> >>amounts of tests on certain patches could really mention is in tested-by,
> >>as opposed to the most common cases where the code was just regularly
> >>tested.
> >
> >Actually, it would be cool to get "Tested: no" and "Tested: compile"
> >tags in the commit mesages. Sometimes it is clear from the context
> >that patch was not tested (treewide update of time to 64bit), but
> >sometime it is not.
> >
> >This is especially problem for -stable, as it seems that lately
> >patches are backported from new version without any testing.
> 
> 
> When I started my own testing some five years ago, most architectures
> did not even build in stable releases. At that time, the only tests being
> done on stable release candidates were a number of build tests, and most
> of the results were ignored.
> 
> Today, we have 0day, kernelci, kerneltests, Linaro's LKFT, and more, plus
> several merge and boot tests done by individuals. Stable release candidates
> are build tested on all supported architectures, with hundreds of
...
> Sure, testing is still far from perfect and needs to be improved. However,
> requiring that every patch applied to stable releases be tested individually
> (where ? on all affected architectures ?) would be the wrong
>direction.

Well, 0day, kernelci etc... is nice... until the change is in the
driver. Most of the kernel are drivers, remember?

I don't know. I'd say that if patch is important enough for -stable,
there should be someone testing it. For core kernel changes, that can
be 0day bot, but for drivers...

And problem exists on mainline, too.

Hmm. Patch for obscure driver. Wow, nice, is WellKnownName actually
using that driver? Aha, no, he is not; he is doing global
search, and did not test the patch...

Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) 
http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html


signature.asc
Description: Digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Guenter Roeck]

On 07/14/2018 10:38 AM, Pavel Machek wrote:

Hi!


The way I see it, if a commit can get one or two tested-by, it's a good
alternative to a week in -next.


Agreed. Even their own actually. And I'm not kidding. Those who run large
amounts of tests on certain patches could really mention is in tested-by,
as opposed to the most common cases where the code was just regularly
tested.


Actually, it would be cool to get "Tested: no" and "Tested: compile"
tags in the commit mesages. Sometimes it is clear from the context
that patch was not tested (treewide update of time to 64bit), but
sometime it is not.

This is especially problem for -stable, as it seems that lately
patches are backported from new version without any testing.



When I started my own testing some five years ago, most architectures
did not even build in stable releases. At that time, the only tests being
done on stable release candidates were a number of build tests, and most
of the results were ignored.

Today, we have 0day, kernelci, kerneltests, Linaro's LKFT, and more, plus
several merge and boot tests done by individuals. Stable release candidates
are build tested on all supported architectures, with hundreds of 
configurations.
Each stable release candidate is boot tested on qemu with more than 150
configurations on each architecture supported by qemu. A substantial amount of
boot tests run on real hardware. On key architectures, more sophisticated tests
such as kerneltests and LTP ensure that no new regressions are introduced.

What is new is that many more patches are being applied and backported
to stable releases, at least to degree due to Sasha's scripts, but also due
to tools like syzbot running on older kernels and finding problems which
have been fixed upstream, but the fix has not been backported.

At the same time, stable release test coverage has been improved substantially
over the last few years. I am _much_ more confident with stable releases
than I used to be a few of years ago.

Sure, there are regressions. However, the regression rate is very low (last
time I checked it was around 0.1% to 0.3% per stable release for us). Sure,
I would like to further reduce regression rate, to improve stability but also
because each and every regression is used by someone to argue that stable 
releases
would be unreliable. However, this is more a matter of perception than reality.
Reality is that more than 90% of all CVEs are fixed in stable releases by the 
time
they are published as affecting a stable release. Reality is that a substantial
percentage of problems reported by syzbot _are_ being fixed in stable releases.
Reality is that, by the time bugs are reported from the field, it is more and
more likely that we find out that the bug has already been fixed in a later
release due to a stable release merge.

Given all that, I think it is quite misleading to claim that the number of
patches applied to stable releases would create additional problems, or that
patches would be applied "without any testing". On the contrary, I would argue
that the additional testing now being performed _enabled_ us to apply more
patches (bug fixes) to stable releases.

Sure, testing is still far from perfect and needs to be improved. However,
requiring that every patch applied to stable releases be tested individually
(where ? on all affected architectures ?) would be the wrong direction.
What we need to do is to further improve test coverage. We should have no
regressions, but we need to get there by improving test coverage, not by
demanding explicit per-patch and per-release testing (which would be all
but impossible anyway - no one has the infrastructure necessary to test
a patch on all affected architectures).

I would encourage everyone interested in kernel testing to attend the
kernel test sessions at Linux Plumbers and ELC later this year to discuss
concerns and possible solutions.

Thanks,
Guenter

Re: [Ksummit-discuss] bug-introducing patches

[Guenter Roeck]

On 07/14/2018 10:38 AM, Pavel Machek wrote:

Hi!


The way I see it, if a commit can get one or two tested-by, it's a good
alternative to a week in -next.


Agreed. Even their own actually. And I'm not kidding. Those who run large
amounts of tests on certain patches could really mention is in tested-by,
as opposed to the most common cases where the code was just regularly
tested.


Actually, it would be cool to get "Tested: no" and "Tested: compile"
tags in the commit mesages. Sometimes it is clear from the context
that patch was not tested (treewide update of time to 64bit), but
sometime it is not.

This is especially problem for -stable, as it seems that lately
patches are backported from new version without any testing.



When I started my own testing some five years ago, most architectures
did not even build in stable releases. At that time, the only tests being
done on stable release candidates were a number of build tests, and most
of the results were ignored.

Today, we have 0day, kernelci, kerneltests, Linaro's LKFT, and more, plus
several merge and boot tests done by individuals. Stable release candidates
are build tested on all supported architectures, with hundreds of 
configurations.
Each stable release candidate is boot tested on qemu with more than 150
configurations on each architecture supported by qemu. A substantial amount of
boot tests run on real hardware. On key architectures, more sophisticated tests
such as kerneltests and LTP ensure that no new regressions are introduced.

What is new is that many more patches are being applied and backported
to stable releases, at least to degree due to Sasha's scripts, but also due
to tools like syzbot running on older kernels and finding problems which
have been fixed upstream, but the fix has not been backported.

At the same time, stable release test coverage has been improved substantially
over the last few years. I am _much_ more confident with stable releases
than I used to be a few of years ago.

Sure, there are regressions. However, the regression rate is very low (last
time I checked it was around 0.1% to 0.3% per stable release for us). Sure,
I would like to further reduce regression rate, to improve stability but also
because each and every regression is used by someone to argue that stable 
releases
would be unreliable. However, this is more a matter of perception than reality.
Reality is that more than 90% of all CVEs are fixed in stable releases by the 
time
they are published as affecting a stable release. Reality is that a substantial
percentage of problems reported by syzbot _are_ being fixed in stable releases.
Reality is that, by the time bugs are reported from the field, it is more and
more likely that we find out that the bug has already been fixed in a later
release due to a stable release merge.

Given all that, I think it is quite misleading to claim that the number of
patches applied to stable releases would create additional problems, or that
patches would be applied "without any testing". On the contrary, I would argue
that the additional testing now being performed _enabled_ us to apply more
patches (bug fixes) to stable releases.

Sure, testing is still far from perfect and needs to be improved. However,
requiring that every patch applied to stable releases be tested individually
(where ? on all affected architectures ?) would be the wrong direction.
What we need to do is to further improve test coverage. We should have no
regressions, but we need to get there by improving test coverage, not by
demanding explicit per-patch and per-release testing (which would be all
but impossible anyway - no one has the infrastructure necessary to test
a patch on all affected architectures).

I would encourage everyone interested in kernel testing to attend the
kernel test sessions at Linux Plumbers and ELC later this year to discuss
concerns and possible solutions.

Thanks,
Guenter

Re: [Ksummit-discuss] bug-introducing patches

[Mark Brown]

On Mon, May 14, 2018 at 10:36:04AM +0200, Ulf Hansson wrote:

> I will ping the kernelci folkz to request them to include your new
> fixes branch for daily builds.

No need, I already added it.


signature.asc
Description: PGP signature

Re: [Ksummit-discuss] bug-introducing patches

[Mark Brown]

On Mon, May 14, 2018 at 10:36:04AM +0200, Ulf Hansson wrote:

> I will ping the kernelci folkz to request them to include your new
> fixes branch for daily builds.

No need, I already added it.


signature.asc
Description: PGP signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Krzysztof,

On Tue, 15 May 2018 12:42:49 +0200 Krzysztof Kozlowski  wrote:
>
> Please merge following fixes branches from my trees:
> Tree: samsung-krzk
> git://git.kernel.org/pub/scm/linux/kernel/git/krzk/linux.git
> branch: fixes
> 
> Tree: pinctrl-samsung
> git://git.kernel.org/pub/scm/linux/kernel/git/pinctrl/samsung.git
> branch: pinctrl-fixes
> 
> Although both are currently empty... but I guess you are collecting
> this also for future.

Yes, indeed.

Added from tomorrow.

Thanks for adding your subsystem tree as a participant of linux-next.  As
you may know, this is not a judgement of your code.  The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window. 

You will need to ensure that the patches/commits in your tree/series have
been:
 * submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
 * posted to the relevant mailing list,
 * reviewed by you (or another maintainer of your subsystem tree),
 * successfully unit tested, and 
 * destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch).  It is allowed to be rebased if you deem it necessary.

-- 
Cheers,
Stephen Rothwell 
s...@canb.auug.org.au


pgpzU0Aj7Seze.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Krzysztof,

On Tue, 15 May 2018 12:42:49 +0200 Krzysztof Kozlowski  wrote:
>
> Please merge following fixes branches from my trees:
> Tree: samsung-krzk
> git://git.kernel.org/pub/scm/linux/kernel/git/krzk/linux.git
> branch: fixes
> 
> Tree: pinctrl-samsung
> git://git.kernel.org/pub/scm/linux/kernel/git/pinctrl/samsung.git
> branch: pinctrl-fixes
> 
> Although both are currently empty... but I guess you are collecting
> this also for future.

Yes, indeed.

Added from tomorrow.

Thanks for adding your subsystem tree as a participant of linux-next.  As
you may know, this is not a judgement of your code.  The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window. 

You will need to ensure that the patches/commits in your tree/series have
been:
 * submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
 * posted to the relevant mailing list,
 * reviewed by you (or another maintainer of your subsystem tree),
 * successfully unit tested, and 
 * destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch).  It is allowed to be rebased if you deem it necessary.

-- 
Cheers,
Stephen Rothwell 
s...@canb.auug.org.au


pgpzU0Aj7Seze.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Krzysztof Kozlowski]

On Wed, May 9, 2018 at 2:43 PM, Stephen Rothwell  wrote:
> Hi Vinod,
>
> On Wed, 9 May 2018 16:25:34 +0530 Vinod Koul  wrote:
>> >
>> > I currently have 44 such fixes branches.  More welcome!
>>
>> Great so do you want us to send fixes branch or scan the existing trees and 
>> add
>> them.
>
> The former.

Please merge following fixes branches from my trees:
Tree: samsung-krzk
git://git.kernel.org/pub/scm/linux/kernel/git/krzk/linux.git
branch: fixes

Tree: pinctrl-samsung
git://git.kernel.org/pub/scm/linux/kernel/git/pinctrl/samsung.git
branch: pinctrl-fixes

Although both are currently empty... but I guess you are collecting
this also for future.

Best regards,
Krzysztof

Re: [Ksummit-discuss] bug-introducing patches

[Krzysztof Kozlowski]

On Wed, May 9, 2018 at 2:43 PM, Stephen Rothwell  wrote:
> Hi Vinod,
>
> On Wed, 9 May 2018 16:25:34 +0530 Vinod Koul  wrote:
>> >
>> > I currently have 44 such fixes branches.  More welcome!
>>
>> Great so do you want us to send fixes branch or scan the existing trees and 
>> add
>> them.
>
> The former.

Please merge following fixes branches from my trees:
Tree: samsung-krzk
git://git.kernel.org/pub/scm/linux/kernel/git/krzk/linux.git
branch: fixes

Tree: pinctrl-samsung
git://git.kernel.org/pub/scm/linux/kernel/git/pinctrl/samsung.git
branch: pinctrl-fixes

Although both are currently empty... but I guess you are collecting
this also for future.

Best regards,
Krzysztof

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Ulf,

On Mon, 14 May 2018 10:36:04 +0200 Ulf Hansson  wrote:
>
> I will ping the kernelci folkz to request them to include your new
> fixes branch for daily builds.

Excellent, thanks.

> For mmc, please add my fixes branch according to below.
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc.git fixes

Added from today.

Thanks for adding your subsystem tree as a participant of linux-next.  As
you may know, this is not a judgement of your code.  The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window. 

You will need to ensure that the patches/commits in your tree/series have
been:
 * submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
 * posted to the relevant mailing list,
 * reviewed by you (or another maintainer of your subsystem tree),
 * successfully unit tested, and 
 * destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch).  It is allowed to be rebased if you deem it necessary.

-- 
Cheers,
Stephen Rothwell 
s...@canb.auug.org.au


pgp44chxMztcw.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Ulf,

On Mon, 14 May 2018 10:36:04 +0200 Ulf Hansson  wrote:
>
> I will ping the kernelci folkz to request them to include your new
> fixes branch for daily builds.

Excellent, thanks.

> For mmc, please add my fixes branch according to below.
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc.git fixes

Added from today.

Thanks for adding your subsystem tree as a participant of linux-next.  As
you may know, this is not a judgement of your code.  The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window. 

You will need to ensure that the patches/commits in your tree/series have
been:
 * submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
 * posted to the relevant mailing list,
 * reviewed by you (or another maintainer of your subsystem tree),
 * successfully unit tested, and 
 * destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch).  It is allowed to be rebased if you deem it necessary.

-- 
Cheers,
Stephen Rothwell 
s...@canb.auug.org.au


pgp44chxMztcw.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Fengguang Wu]

On Mon, May 14, 2018 at 10:48:03AM +0200, Boris Brezillon wrote:

+Fengguang

On Mon, 14 May 2018 10:40:10 +0200
Geert Uytterhoeven  wrote:


Hi Boris,

On Mon, May 14, 2018 at 10:34 AM, Boris Brezillon
 wrote:
> On Mon, 14 May 2018 10:29:04 +0200
> Geert Uytterhoeven  wrote:
>> On Mon, May 14, 2018 at 10:12 AM, Boris Brezillon
>>  wrote:
>> > On Mon, 14 May 2018 10:00:30 +0200
>> > Geert Uytterhoeven  wrote:
>> >> On Tue, May 1, 2018 at 10:00 PM, Sasha Levin
>> >>  wrote:
>> >> > On Tue, May 01, 2018 at 03:44:50PM -0400, Theodore Y. Ts'o wrote:
>> >> >>On Tue, May 01, 2018 at 04:38:21PM +, Sasha Levin wrote:
>> >> > What's worse is that that commit is tagged for stable, which means
>> >> > that (given Greg's schedule) it may find it's way to -stable users
>> >> > even before some -next users/bots had a chance to test it out.
>> >>
>> >> I just noticed a case where a commit was picked up for stable, while a
>> >> bot had flagged it as a build regression 18 hours earlier (with a CC to
>> >> lkml).
>> >
>> > Also, this patch has been on a tree that I know is tested by Fengguang's
>> > robots for more than a week (and in linux-next for 2 days, which, I
>> > agree, is probably not enough), and still, I only received the bug
>> > report when the patch reached mainline. Are there tests that are only
>> > run on Linus' tree?
>>
>> Have your received a success report from Fengguang's bot, listing all
>> configs tested (the broken one should be included; it is included in the
>> configs tested on my branches)?
>
> Yes I did (see below).
>
> -->8--
> From: kbuild test robot 
> To: Boris Brezillon 
> Subject: [bbrezillon-0day:mtd/fixes] BUILD SUCCESS  
fc3a9e15b492eef707afd56b7478001fdecfe53f
> Date: Mon, 07 May 2018 20:05:52 +0800
> User-Agent: Heirloom mailx 12.5 6/20/10
>
> tree/branch: https://github.com/bbrezillon/linux-0day  mtd/fixes
> branch HEAD: fc3a9e15b492eef707afd56b7478001fdecfe53f  mtd: rawnand: Make 
sure we wait tWB before polling the STATUS reg
>
> elapsed time: 49m
>
> configs tested: 142

But the failed config (m68k/allmodconfig) is not listed?


Yes, that's my point. It seems that some configs are only rarely
(never?) tested on my linux-0day tree (probably because they take longer
to build), and I should only take kbuild robot results as an indication
not a guarantee.


Yeah sorry, there is no 100% guarantee. There are 2 main aspects to
this problem.

- Response time vs coverage. Most build errors can be caught within 1
 day. The build success notification email is typically sent within
 half day (a reasonable feedback time). At this time, it can only be
 a rough indication not a guarantee. After sending the 0day build
 success notification, the build tests will actually continue for
 about 1 week to increase test coverage. 


- Merge-test-bisect based workflow. If one branch is hard to merge
 with others, especially if it's based on old kernel, it'll receive
 much less test coverage. Branches with known build/boot errors will
 be excluded from further merges, too.

Thanks,
Fengguang

Re: [Ksummit-discuss] bug-introducing patches

[Fengguang Wu]

On Mon, May 14, 2018 at 10:48:03AM +0200, Boris Brezillon wrote:

+Fengguang

On Mon, 14 May 2018 10:40:10 +0200
Geert Uytterhoeven  wrote:


Hi Boris,

On Mon, May 14, 2018 at 10:34 AM, Boris Brezillon
 wrote:
> On Mon, 14 May 2018 10:29:04 +0200
> Geert Uytterhoeven  wrote:
>> On Mon, May 14, 2018 at 10:12 AM, Boris Brezillon
>>  wrote:
>> > On Mon, 14 May 2018 10:00:30 +0200
>> > Geert Uytterhoeven  wrote:
>> >> On Tue, May 1, 2018 at 10:00 PM, Sasha Levin
>> >>  wrote:
>> >> > On Tue, May 01, 2018 at 03:44:50PM -0400, Theodore Y. Ts'o wrote:
>> >> >>On Tue, May 01, 2018 at 04:38:21PM +, Sasha Levin wrote:
>> >> > What's worse is that that commit is tagged for stable, which means
>> >> > that (given Greg's schedule) it may find it's way to -stable users
>> >> > even before some -next users/bots had a chance to test it out.
>> >>
>> >> I just noticed a case where a commit was picked up for stable, while a
>> >> bot had flagged it as a build regression 18 hours earlier (with a CC to
>> >> lkml).
>> >
>> > Also, this patch has been on a tree that I know is tested by Fengguang's
>> > robots for more than a week (and in linux-next for 2 days, which, I
>> > agree, is probably not enough), and still, I only received the bug
>> > report when the patch reached mainline. Are there tests that are only
>> > run on Linus' tree?
>>
>> Have your received a success report from Fengguang's bot, listing all
>> configs tested (the broken one should be included; it is included in the
>> configs tested on my branches)?
>
> Yes I did (see below).
>
> -->8--
> From: kbuild test robot 
> To: Boris Brezillon 
> Subject: [bbrezillon-0day:mtd/fixes] BUILD SUCCESS  
fc3a9e15b492eef707afd56b7478001fdecfe53f
> Date: Mon, 07 May 2018 20:05:52 +0800
> User-Agent: Heirloom mailx 12.5 6/20/10
>
> tree/branch: https://github.com/bbrezillon/linux-0day  mtd/fixes
> branch HEAD: fc3a9e15b492eef707afd56b7478001fdecfe53f  mtd: rawnand: Make 
sure we wait tWB before polling the STATUS reg
>
> elapsed time: 49m
>
> configs tested: 142

But the failed config (m68k/allmodconfig) is not listed?


Yes, that's my point. It seems that some configs are only rarely
(never?) tested on my linux-0day tree (probably because they take longer
to build), and I should only take kbuild robot results as an indication
not a guarantee.


Yeah sorry, there is no 100% guarantee. There are 2 main aspects to
this problem.

- Response time vs coverage. Most build errors can be caught within 1
 day. The build success notification email is typically sent within
 half day (a reasonable feedback time). At this time, it can only be
 a rough indication not a guarantee. After sending the 0day build
 success notification, the build tests will actually continue for
 about 1 week to increase test coverage. 


- Merge-test-bisect based workflow. If one branch is hard to merge
 with others, especially if it's based on old kernel, it'll receive
 much less test coverage. Branches with known build/boot errors will
 be excluded from further merges, too.

Thanks,
Fengguang

Re: [Ksummit-discuss] bug-introducing patches

[Boris Brezillon]

+Fengguang

On Mon, 14 May 2018 10:40:10 +0200
Geert Uytterhoeven  wrote:

> Hi Boris,
> 
> On Mon, May 14, 2018 at 10:34 AM, Boris Brezillon
>  wrote:
> > On Mon, 14 May 2018 10:29:04 +0200
> > Geert Uytterhoeven  wrote:  
> >> On Mon, May 14, 2018 at 10:12 AM, Boris Brezillon
> >>  wrote:  
> >> > On Mon, 14 May 2018 10:00:30 +0200
> >> > Geert Uytterhoeven  wrote:  
> >> >> On Tue, May 1, 2018 at 10:00 PM, Sasha Levin
> >> >>  wrote:  
> >> >> > On Tue, May 01, 2018 at 03:44:50PM -0400, Theodore Y. Ts'o wrote:  
> >> >> >>On Tue, May 01, 2018 at 04:38:21PM +, Sasha Levin wrote:  
> >> >> > What's worse is that that commit is tagged for stable, which means
> >> >> > that (given Greg's schedule) it may find it's way to -stable users
> >> >> > even before some -next users/bots had a chance to test it out.  
> >> >>
> >> >> I just noticed a case where a commit was picked up for stable, while a
> >> >> bot had flagged it as a build regression 18 hours earlier (with a CC to
> >> >> lkml).  
> >> >
> >> > Also, this patch has been on a tree that I know is tested by Fengguang's
> >> > robots for more than a week (and in linux-next for 2 days, which, I
> >> > agree, is probably not enough), and still, I only received the bug
> >> > report when the patch reached mainline. Are there tests that are only
> >> > run on Linus' tree?  
> >>
> >> Have your received a success report from Fengguang's bot, listing all
> >> configs tested (the broken one should be included; it is included in the
> >> configs tested on my branches)?  
> >
> > Yes I did (see below).
> >  
> > -->8--  
> > From: kbuild test robot 
> > To: Boris Brezillon 
> > Subject: [bbrezillon-0day:mtd/fixes] BUILD SUCCESS  
> > fc3a9e15b492eef707afd56b7478001fdecfe53f
> > Date: Mon, 07 May 2018 20:05:52 +0800
> > User-Agent: Heirloom mailx 12.5 6/20/10
> >
> > tree/branch: https://github.com/bbrezillon/linux-0day  mtd/fixes
> > branch HEAD: fc3a9e15b492eef707afd56b7478001fdecfe53f  mtd: rawnand: Make 
> > sure we wait tWB before polling the STATUS reg
> >
> > elapsed time: 49m
> >
> > configs tested: 142  
> 
> But the failed config (m68k/allmodconfig) is not listed?

Yes, that's my point. It seems that some configs are only rarely
(never?) tested on my linux-0day tree (probably because they take longer
to build), and I should only take kbuild robot results as an indication
not a guarantee.

Re: [Ksummit-discuss] bug-introducing patches

[Boris Brezillon]

+Fengguang

On Mon, 14 May 2018 10:40:10 +0200
Geert Uytterhoeven  wrote:

> Hi Boris,
> 
> On Mon, May 14, 2018 at 10:34 AM, Boris Brezillon
>  wrote:
> > On Mon, 14 May 2018 10:29:04 +0200
> > Geert Uytterhoeven  wrote:  
> >> On Mon, May 14, 2018 at 10:12 AM, Boris Brezillon
> >>  wrote:  
> >> > On Mon, 14 May 2018 10:00:30 +0200
> >> > Geert Uytterhoeven  wrote:  
> >> >> On Tue, May 1, 2018 at 10:00 PM, Sasha Levin
> >> >>  wrote:  
> >> >> > On Tue, May 01, 2018 at 03:44:50PM -0400, Theodore Y. Ts'o wrote:  
> >> >> >>On Tue, May 01, 2018 at 04:38:21PM +, Sasha Levin wrote:  
> >> >> > What's worse is that that commit is tagged for stable, which means
> >> >> > that (given Greg's schedule) it may find it's way to -stable users
> >> >> > even before some -next users/bots had a chance to test it out.  
> >> >>
> >> >> I just noticed a case where a commit was picked up for stable, while a
> >> >> bot had flagged it as a build regression 18 hours earlier (with a CC to
> >> >> lkml).  
> >> >
> >> > Also, this patch has been on a tree that I know is tested by Fengguang's
> >> > robots for more than a week (and in linux-next for 2 days, which, I
> >> > agree, is probably not enough), and still, I only received the bug
> >> > report when the patch reached mainline. Are there tests that are only
> >> > run on Linus' tree?  
> >>
> >> Have your received a success report from Fengguang's bot, listing all
> >> configs tested (the broken one should be included; it is included in the
> >> configs tested on my branches)?  
> >
> > Yes I did (see below).
> >  
> > -->8--  
> > From: kbuild test robot 
> > To: Boris Brezillon 
> > Subject: [bbrezillon-0day:mtd/fixes] BUILD SUCCESS  
> > fc3a9e15b492eef707afd56b7478001fdecfe53f
> > Date: Mon, 07 May 2018 20:05:52 +0800
> > User-Agent: Heirloom mailx 12.5 6/20/10
> >
> > tree/branch: https://github.com/bbrezillon/linux-0day  mtd/fixes
> > branch HEAD: fc3a9e15b492eef707afd56b7478001fdecfe53f  mtd: rawnand: Make 
> > sure we wait tWB before polling the STATUS reg
> >
> > elapsed time: 49m
> >
> > configs tested: 142  
> 
> But the failed config (m68k/allmodconfig) is not listed?

Yes, that's my point. It seems that some configs are only rarely
(never?) tested on my linux-0day tree (probably because they take longer
to build), and I should only take kbuild robot results as an indication
not a guarantee.

Re: [Ksummit-discuss] bug-introducing patches

[Geert Uytterhoeven]

Hi Boris,

On Mon, May 14, 2018 at 10:34 AM, Boris Brezillon
 wrote:
> On Mon, 14 May 2018 10:29:04 +0200
> Geert Uytterhoeven  wrote:
>> On Mon, May 14, 2018 at 10:12 AM, Boris Brezillon
>>  wrote:
>> > On Mon, 14 May 2018 10:00:30 +0200
>> > Geert Uytterhoeven  wrote:
>> >> On Tue, May 1, 2018 at 10:00 PM, Sasha Levin
>> >>  wrote:
>> >> > On Tue, May 01, 2018 at 03:44:50PM -0400, Theodore Y. Ts'o wrote:
>> >> >>On Tue, May 01, 2018 at 04:38:21PM +, Sasha Levin wrote:
>> >> > What's worse is that that commit is tagged for stable, which means
>> >> > that (given Greg's schedule) it may find it's way to -stable users
>> >> > even before some -next users/bots had a chance to test it out.
>> >>
>> >> I just noticed a case where a commit was picked up for stable, while a
>> >> bot had flagged it as a build regression 18 hours earlier (with a CC to
>> >> lkml).
>> >
>> > Also, this patch has been on a tree that I know is tested by Fengguang's
>> > robots for more than a week (and in linux-next for 2 days, which, I
>> > agree, is probably not enough), and still, I only received the bug
>> > report when the patch reached mainline. Are there tests that are only
>> > run on Linus' tree?
>>
>> Have your received a success report from Fengguang's bot, listing all
>> configs tested (the broken one should be included; it is included in the
>> configs tested on my branches)?
>
> Yes I did (see below).
>
> -->8--
> From: kbuild test robot 
> To: Boris Brezillon 
> Subject: [bbrezillon-0day:mtd/fixes] BUILD SUCCESS  
> fc3a9e15b492eef707afd56b7478001fdecfe53f
> Date: Mon, 07 May 2018 20:05:52 +0800
> User-Agent: Heirloom mailx 12.5 6/20/10
>
> tree/branch: https://github.com/bbrezillon/linux-0day  mtd/fixes
> branch HEAD: fc3a9e15b492eef707afd56b7478001fdecfe53f  mtd: rawnand: Make 
> sure we wait tWB before polling the STATUS reg
>
> elapsed time: 49m
>
> configs tested: 142

But the failed config (m68k/allmodconfig) is not listed?

BTW, my last report had:

configs tested: 178

Gr{oetje,eeting}s,

Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: [Ksummit-discuss] bug-introducing patches

[Geert Uytterhoeven]

Hi Boris,

On Mon, May 14, 2018 at 10:34 AM, Boris Brezillon
 wrote:
> On Mon, 14 May 2018 10:29:04 +0200
> Geert Uytterhoeven  wrote:
>> On Mon, May 14, 2018 at 10:12 AM, Boris Brezillon
>>  wrote:
>> > On Mon, 14 May 2018 10:00:30 +0200
>> > Geert Uytterhoeven  wrote:
>> >> On Tue, May 1, 2018 at 10:00 PM, Sasha Levin
>> >>  wrote:
>> >> > On Tue, May 01, 2018 at 03:44:50PM -0400, Theodore Y. Ts'o wrote:
>> >> >>On Tue, May 01, 2018 at 04:38:21PM +, Sasha Levin wrote:
>> >> > What's worse is that that commit is tagged for stable, which means
>> >> > that (given Greg's schedule) it may find it's way to -stable users
>> >> > even before some -next users/bots had a chance to test it out.
>> >>
>> >> I just noticed a case where a commit was picked up for stable, while a
>> >> bot had flagged it as a build regression 18 hours earlier (with a CC to
>> >> lkml).
>> >
>> > Also, this patch has been on a tree that I know is tested by Fengguang's
>> > robots for more than a week (and in linux-next for 2 days, which, I
>> > agree, is probably not enough), and still, I only received the bug
>> > report when the patch reached mainline. Are there tests that are only
>> > run on Linus' tree?
>>
>> Have your received a success report from Fengguang's bot, listing all
>> configs tested (the broken one should be included; it is included in the
>> configs tested on my branches)?
>
> Yes I did (see below).
>
> -->8--
> From: kbuild test robot 
> To: Boris Brezillon 
> Subject: [bbrezillon-0day:mtd/fixes] BUILD SUCCESS  
> fc3a9e15b492eef707afd56b7478001fdecfe53f
> Date: Mon, 07 May 2018 20:05:52 +0800
> User-Agent: Heirloom mailx 12.5 6/20/10
>
> tree/branch: https://github.com/bbrezillon/linux-0day  mtd/fixes
> branch HEAD: fc3a9e15b492eef707afd56b7478001fdecfe53f  mtd: rawnand: Make 
> sure we wait tWB before polling the STATUS reg
>
> elapsed time: 49m
>
> configs tested: 142

But the failed config (m68k/allmodconfig) is not listed?

BTW, my last report had:

configs tested: 178

Gr{oetje,eeting}s,

Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: [Ksummit-discuss] bug-introducing patches

[Ulf Hansson]

On 9 May 2018 at 12:47, Stephen Rothwell  wrote:
> On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
>>
>> On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
>> > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:
>>
>> > > I think this is an excellent idea, copying in Stephen for his input.
>> > > I'm currently on holiday but unless someone convinces me it's a terrible
>> > > idea I'm willing to at least give it a go on a trial basis once I'm back
>> > > home.
>>
>> > Since Stephen merges all -fixes branches first, before merging all the
>> > -next branches, he already generates that as part of linux-next. All
>> > he'd need to do is push that intermediate state out to some
>> > linux-fixes branch for consumption by test bots.
>
> Good idea ... I will see what I can do.
>
>> True.  It's currently only those -fixes branches that people have asked
>> him to merge separately which isn't as big a proportion of trees as have
>> them (perhaps fortunately given people's enthusiasm for fixes branches
>> that don't merge cleanly with their development branches) so we'd also
>> need to encourage people to add them separately.
>
> I currently have 44 such fixes branches.  More welcome!

Great!

Stephen, thanks for picking up the idea so quickly!

I will ping the kernelci folkz to request them to include your new
fixes branch for daily builds.

For mmc, please add my fixes branch according to below.

git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc.git fixes

Kind regards
Uffe

Re: [Ksummit-discuss] bug-introducing patches

[Ulf Hansson]

On 9 May 2018 at 12:47, Stephen Rothwell  wrote:
> On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
>>
>> On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
>> > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:
>>
>> > > I think this is an excellent idea, copying in Stephen for his input.
>> > > I'm currently on holiday but unless someone convinces me it's a terrible
>> > > idea I'm willing to at least give it a go on a trial basis once I'm back
>> > > home.
>>
>> > Since Stephen merges all -fixes branches first, before merging all the
>> > -next branches, he already generates that as part of linux-next. All
>> > he'd need to do is push that intermediate state out to some
>> > linux-fixes branch for consumption by test bots.
>
> Good idea ... I will see what I can do.
>
>> True.  It's currently only those -fixes branches that people have asked
>> him to merge separately which isn't as big a proportion of trees as have
>> them (perhaps fortunately given people's enthusiasm for fixes branches
>> that don't merge cleanly with their development branches) so we'd also
>> need to encourage people to add them separately.
>
> I currently have 44 such fixes branches.  More welcome!

Great!

Stephen, thanks for picking up the idea so quickly!

I will ping the kernelci folkz to request them to include your new
fixes branch for daily builds.

For mmc, please add my fixes branch according to below.

git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc.git fixes

Kind regards
Uffe

Re: [Ksummit-discuss] bug-introducing patches

[Boris Brezillon]

On Mon, 14 May 2018 10:29:04 +0200
Geert Uytterhoeven  wrote:

> Hi Boris,
> 
> On Mon, May 14, 2018 at 10:12 AM, Boris Brezillon
>  wrote:
> > On Mon, 14 May 2018 10:00:30 +0200
> > Geert Uytterhoeven  wrote:  
> >> On Tue, May 1, 2018 at 10:00 PM, Sasha Levin
> >>  wrote:  
> >> > On Tue, May 01, 2018 at 03:44:50PM -0400, Theodore Y. Ts'o wrote:  
> >> >>On Tue, May 01, 2018 at 04:38:21PM +, Sasha Levin wrote:  
> >> > What's worse is that that commit is tagged for stable, which means
> >> > that (given Greg's schedule) it may find it's way to -stable users
> >> > even before some -next users/bots had a chance to test it out.  
> >>
> >> I just noticed a case where a commit was picked up for stable, while a
> >> bot had flagged it as a build regression 18 hours earlier (with a CC to
> >> lkml).  
> >
> > Also, this patch has been on a tree that I know is tested by Fengguang's
> > robots for more than a week (and in linux-next for 2 days, which, I
> > agree, is probably not enough), and still, I only received the bug
> > report when the patch reached mainline. Are there tests that are only
> > run on Linus' tree?  
> 
> Have your received a success report from Fengguang's bot, listing all
> configs tested (the broken one should be included; it is included in the
> configs tested on my branches)?

Yes I did (see below).

-->8--
From: kbuild test robot 
To: Boris Brezillon 
Subject: [bbrezillon-0day:mtd/fixes] BUILD SUCCESS  
fc3a9e15b492eef707afd56b7478001fdecfe53f
Date: Mon, 07 May 2018 20:05:52 +0800
User-Agent: Heirloom mailx 12.5 6/20/10

tree/branch: https://github.com/bbrezillon/linux-0day  mtd/fixes
branch HEAD: fc3a9e15b492eef707afd56b7478001fdecfe53f  mtd: rawnand: Make sure 
we wait tWB before polling the STATUS reg

elapsed time: 49m

configs tested: 142

The following configs have been built successfully.
More configs may be tested in the coming days.

powerpc skiroot_defconfig
sh  kfr2r09_defconfig
x86_64 acpi-redef
x86_64   allyesdebian
x86_64nfsroot
m68k   bvme6000_defconfig
powerpc ppa8548_defconfig
shallnoconfig
sh  rsk7269_defconfig
sh  sh7785lcr_32bit_defconfig
shtitan_defconfig
i386   randconfig-c0-05071338
i386   tinyconfig
i386 randconfig-n0-201818
x86_64 randconfig-x002-201818
x86_64 randconfig-x006-201818
x86_64 randconfig-x005-201818
x86_64 randconfig-x001-201818
x86_64 randconfig-x009-201818
x86_64 randconfig-x004-201818
x86_64 randconfig-x003-201818
x86_64 randconfig-x007-201818
x86_64 randconfig-x000-201818
x86_64 randconfig-x008-201818
i386 randconfig-i1-201818
i386 randconfig-i0-201818
alpha   defconfig
pariscallnoconfig
parisc b180_defconfig
pariscc3000_defconfig
parisc  defconfig
ia64defconfig
mipsdefconfig
powerpc   allnoconfig
powerpc defconfig
powerpc   ppc64_defconfig
s390default_defconfig
x86_64 randconfig-g0-05071702
openriscor1ksim_defconfig
um i386_defconfig
um   x86_64_defconfig
i386 allmodconfig
ia64 alldefconfig
ia64  allnoconfig
i386 randconfig-a0-201818
i386 randconfig-a1-201818
x86_64 randconfig-s0-05071833
x86_64 randconfig-s1-05071833
x86_64 randconfig-s2-05071833
x86_64 randconfig-s0-05071933
x86_64 randconfig-s1-05071933
x86_64 randconfig-s2-05071933
c6xevmc6678_defconfig
h8300h8300h-sim_defconfig
nios2 10m50_defconfig
xtensa   common_defconfig
xtensa  iss_defconfig
i386 alldefconfig
i386  allnoconfig
i386defconfig
x86_64  federa-25
x86_64   rhel
x86_64   rhel-7.2
m68k   m5475evb_defconfig
m68k 

Re: [Ksummit-discuss] bug-introducing patches

[Boris Brezillon]

On Mon, 14 May 2018 10:29:04 +0200
Geert Uytterhoeven  wrote:

> Hi Boris,
> 
> On Mon, May 14, 2018 at 10:12 AM, Boris Brezillon
>  wrote:
> > On Mon, 14 May 2018 10:00:30 +0200
> > Geert Uytterhoeven  wrote:  
> >> On Tue, May 1, 2018 at 10:00 PM, Sasha Levin
> >>  wrote:  
> >> > On Tue, May 01, 2018 at 03:44:50PM -0400, Theodore Y. Ts'o wrote:  
> >> >>On Tue, May 01, 2018 at 04:38:21PM +, Sasha Levin wrote:  
> >> > What's worse is that that commit is tagged for stable, which means
> >> > that (given Greg's schedule) it may find it's way to -stable users
> >> > even before some -next users/bots had a chance to test it out.  
> >>
> >> I just noticed a case where a commit was picked up for stable, while a
> >> bot had flagged it as a build regression 18 hours earlier (with a CC to
> >> lkml).  
> >
> > Also, this patch has been on a tree that I know is tested by Fengguang's
> > robots for more than a week (and in linux-next for 2 days, which, I
> > agree, is probably not enough), and still, I only received the bug
> > report when the patch reached mainline. Are there tests that are only
> > run on Linus' tree?  
> 
> Have your received a success report from Fengguang's bot, listing all
> configs tested (the broken one should be included; it is included in the
> configs tested on my branches)?

Yes I did (see below).

-->8--
From: kbuild test robot 
To: Boris Brezillon 
Subject: [bbrezillon-0day:mtd/fixes] BUILD SUCCESS  
fc3a9e15b492eef707afd56b7478001fdecfe53f
Date: Mon, 07 May 2018 20:05:52 +0800
User-Agent: Heirloom mailx 12.5 6/20/10

tree/branch: https://github.com/bbrezillon/linux-0day  mtd/fixes
branch HEAD: fc3a9e15b492eef707afd56b7478001fdecfe53f  mtd: rawnand: Make sure 
we wait tWB before polling the STATUS reg

elapsed time: 49m

configs tested: 142

The following configs have been built successfully.
More configs may be tested in the coming days.

powerpc skiroot_defconfig
sh  kfr2r09_defconfig
x86_64 acpi-redef
x86_64   allyesdebian
x86_64nfsroot
m68k   bvme6000_defconfig
powerpc ppa8548_defconfig
shallnoconfig
sh  rsk7269_defconfig
sh  sh7785lcr_32bit_defconfig
shtitan_defconfig
i386   randconfig-c0-05071338
i386   tinyconfig
i386 randconfig-n0-201818
x86_64 randconfig-x002-201818
x86_64 randconfig-x006-201818
x86_64 randconfig-x005-201818
x86_64 randconfig-x001-201818
x86_64 randconfig-x009-201818
x86_64 randconfig-x004-201818
x86_64 randconfig-x003-201818
x86_64 randconfig-x007-201818
x86_64 randconfig-x000-201818
x86_64 randconfig-x008-201818
i386 randconfig-i1-201818
i386 randconfig-i0-201818
alpha   defconfig
pariscallnoconfig
parisc b180_defconfig
pariscc3000_defconfig
parisc  defconfig
ia64defconfig
mipsdefconfig
powerpc   allnoconfig
powerpc defconfig
powerpc   ppc64_defconfig
s390default_defconfig
x86_64 randconfig-g0-05071702
openriscor1ksim_defconfig
um i386_defconfig
um   x86_64_defconfig
i386 allmodconfig
ia64 alldefconfig
ia64  allnoconfig
i386 randconfig-a0-201818
i386 randconfig-a1-201818
x86_64 randconfig-s0-05071833
x86_64 randconfig-s1-05071833
x86_64 randconfig-s2-05071833
x86_64 randconfig-s0-05071933
x86_64 randconfig-s1-05071933
x86_64 randconfig-s2-05071933
c6xevmc6678_defconfig
h8300h8300h-sim_defconfig
nios2 10m50_defconfig
xtensa   common_defconfig
xtensa  iss_defconfig
i386 alldefconfig
i386  allnoconfig
i386defconfig
x86_64  federa-25
x86_64   rhel
x86_64   rhel-7.2
m68k   m5475evb_defconfig
m68k  multi_defconfig
m68k   sun3_defconfig
i386 randconfig-s0-201818
i386

Re: [Ksummit-discuss] bug-introducing patches

[Geert Uytterhoeven]

Hi Boris,

On Mon, May 14, 2018 at 10:12 AM, Boris Brezillon
 wrote:
> On Mon, 14 May 2018 10:00:30 +0200
> Geert Uytterhoeven  wrote:
>> On Tue, May 1, 2018 at 10:00 PM, Sasha Levin
>>  wrote:
>> > On Tue, May 01, 2018 at 03:44:50PM -0400, Theodore Y. Ts'o wrote:
>> >>On Tue, May 01, 2018 at 04:38:21PM +, Sasha Levin wrote:
>> > What's worse is that that commit is tagged for stable, which means
>> > that (given Greg's schedule) it may find it's way to -stable users
>> > even before some -next users/bots had a chance to test it out.
>>
>> I just noticed a case where a commit was picked up for stable, while a
>> bot had flagged it as a build regression 18 hours earlier (with a CC to
>> lkml).
>
> Also, this patch has been on a tree that I know is tested by Fengguang's
> robots for more than a week (and in linux-next for 2 days, which, I
> agree, is probably not enough), and still, I only received the bug
> report when the patch reached mainline. Are there tests that are only
> run on Linus' tree?

Have your received a success report from Fengguang's bot, listing all
configs tested (the broken one should be included; it is included in the
configs tested on my branches)?

Gr{oetje,eeting}s,

Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: [Ksummit-discuss] bug-introducing patches

[Geert Uytterhoeven]

Hi Boris,

On Mon, May 14, 2018 at 10:12 AM, Boris Brezillon
 wrote:
> On Mon, 14 May 2018 10:00:30 +0200
> Geert Uytterhoeven  wrote:
>> On Tue, May 1, 2018 at 10:00 PM, Sasha Levin
>>  wrote:
>> > On Tue, May 01, 2018 at 03:44:50PM -0400, Theodore Y. Ts'o wrote:
>> >>On Tue, May 01, 2018 at 04:38:21PM +, Sasha Levin wrote:
>> > What's worse is that that commit is tagged for stable, which means
>> > that (given Greg's schedule) it may find it's way to -stable users
>> > even before some -next users/bots had a chance to test it out.
>>
>> I just noticed a case where a commit was picked up for stable, while a
>> bot had flagged it as a build regression 18 hours earlier (with a CC to
>> lkml).
>
> Also, this patch has been on a tree that I know is tested by Fengguang's
> robots for more than a week (and in linux-next for 2 days, which, I
> agree, is probably not enough), and still, I only received the bug
> report when the patch reached mainline. Are there tests that are only
> run on Linus' tree?

Have your received a success report from Fengguang's bot, listing all
configs tested (the broken one should be included; it is included in the
configs tested on my branches)?

Gr{oetje,eeting}s,

Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: [Ksummit-discuss] bug-introducing patches

[Boris Brezillon]

On Mon, 14 May 2018 10:00:30 +0200
Geert Uytterhoeven  wrote:

> On Tue, May 1, 2018 at 10:00 PM, Sasha Levin
>  wrote:
> > On Tue, May 01, 2018 at 03:44:50PM -0400, Theodore Y. Ts'o wrote:  
> >>On Tue, May 01, 2018 at 04:38:21PM +, Sasha Levin wrote:  
> >>>  - A merge window commit spent 50% more days, on average, in -next than a 
> >>> -rc
> >>>commit.  
> >>
> >>So it *used* to be the case that after the merge window, I would queue
> >>up bug fixes for the next merge window.  Greg K-H pushed for me to
> >>send them to Linus sooner, instead of waiting for the next merge
> >>window.  TBH, it's actually easier for me to just wait until the next
> >>merge window, but please understand that there are multiple pressures
> >>on maintainers going on here, and the latest efforts to try to use
> >>AUTOSEL is just the most recent pressure placed on maintainers.
> >>
> >>The other thing is that when there is a regression users who are
> >>testing linux-next want it fixed *fast*.  That's considered more
> >>important to them than waiting for one, perfect patch, just to keep
> >>AUTOSEL happy.
> >>
> >>So please understand that when you say that maintainers *need* to do X
> >>or Y, that there you are not the only one standing in line putting
> >>pressures on maintainers saying they *need* to do something.  And
> >>quite frankly, I consider keeping people who are nice enough to test
> >>linux-next happy to be **far** more important than AUTOSEL.  
> >
> > Ted,
> >
> > I'm not at all asking to wait before adding the patches to your tree,
> > or to -next. I'm asking to hold on to them a bit longer before you
> > push them to Linus because I can show that patches that don't spend
> > enough time in -next are more likely to introduce bugs.
> >
> > Yes, linux-next users want it fixed *now* and I completely agree it
> > should be done that way, but the fix should not be immediately pushed to
> > Linus as well.
> >
> > I've just finished reading an interesting article on LWN about the
> > PostgreSQL fsync issues (https://lwn.net/Articles/752952/). If you
> > look at Willy's commit, he wrote the final version of it about 5 days
> > ago, Jeff merged it in 3 days ago, and Linus merged it in the tree
> > today. Did it spend any time getting -next testing? nope.
> >
> > What's worse is that that commit is tagged for stable, which means
> > that (given Greg's schedule) it may find it's way to -stable users
> > even before some -next users/bots had a chance to test it out.  
> 
> I just noticed a case where a commit was picked up for stable, while a
> bot had flagged it as a build regression 18 hours earlier (with a CC to
> lkml).

Also, this patch has been on a tree that I know is tested by Fengguang's
robots for more than a week (and in linux-next for 2 days, which, I
agree, is probably not enough), and still, I only received the bug
report when the patch reached mainline. Are there tests that are only
run on Linus' tree?

> 
> So it looks like the script for backporting commits should be enhanced to
> check for this (searching for the commit ID in my email archive found the
> bot report).
> 
> Thanks!
> 
> Gr{oetje,eeting}s,
> 
> Geert
> 

Re: [Ksummit-discuss] bug-introducing patches

[Boris Brezillon]

On Mon, 14 May 2018 10:00:30 +0200
Geert Uytterhoeven  wrote:

> On Tue, May 1, 2018 at 10:00 PM, Sasha Levin
>  wrote:
> > On Tue, May 01, 2018 at 03:44:50PM -0400, Theodore Y. Ts'o wrote:  
> >>On Tue, May 01, 2018 at 04:38:21PM +, Sasha Levin wrote:  
> >>>  - A merge window commit spent 50% more days, on average, in -next than a 
> >>> -rc
> >>>commit.  
> >>
> >>So it *used* to be the case that after the merge window, I would queue
> >>up bug fixes for the next merge window.  Greg K-H pushed for me to
> >>send them to Linus sooner, instead of waiting for the next merge
> >>window.  TBH, it's actually easier for me to just wait until the next
> >>merge window, but please understand that there are multiple pressures
> >>on maintainers going on here, and the latest efforts to try to use
> >>AUTOSEL is just the most recent pressure placed on maintainers.
> >>
> >>The other thing is that when there is a regression users who are
> >>testing linux-next want it fixed *fast*.  That's considered more
> >>important to them than waiting for one, perfect patch, just to keep
> >>AUTOSEL happy.
> >>
> >>So please understand that when you say that maintainers *need* to do X
> >>or Y, that there you are not the only one standing in line putting
> >>pressures on maintainers saying they *need* to do something.  And
> >>quite frankly, I consider keeping people who are nice enough to test
> >>linux-next happy to be **far** more important than AUTOSEL.  
> >
> > Ted,
> >
> > I'm not at all asking to wait before adding the patches to your tree,
> > or to -next. I'm asking to hold on to them a bit longer before you
> > push them to Linus because I can show that patches that don't spend
> > enough time in -next are more likely to introduce bugs.
> >
> > Yes, linux-next users want it fixed *now* and I completely agree it
> > should be done that way, but the fix should not be immediately pushed to
> > Linus as well.
> >
> > I've just finished reading an interesting article on LWN about the
> > PostgreSQL fsync issues (https://lwn.net/Articles/752952/). If you
> > look at Willy's commit, he wrote the final version of it about 5 days
> > ago, Jeff merged it in 3 days ago, and Linus merged it in the tree
> > today. Did it spend any time getting -next testing? nope.
> >
> > What's worse is that that commit is tagged for stable, which means
> > that (given Greg's schedule) it may find it's way to -stable users
> > even before some -next users/bots had a chance to test it out.  
> 
> I just noticed a case where a commit was picked up for stable, while a
> bot had flagged it as a build regression 18 hours earlier (with a CC to
> lkml).

Also, this patch has been on a tree that I know is tested by Fengguang's
robots for more than a week (and in linux-next for 2 days, which, I
agree, is probably not enough), and still, I only received the bug
report when the patch reached mainline. Are there tests that are only
run on Linus' tree?

> 
> So it looks like the script for backporting commits should be enhanced to
> check for this (searching for the commit ID in my email archive found the
> bot report).
> 
> Thanks!
> 
> Gr{oetje,eeting}s,
> 
> Geert
> 

Re: [Ksummit-discuss] bug-introducing patches

[Geert Uytterhoeven]

On Thu, May 10, 2018 at 6:47 PM, Sasha Levin via Ksummit-discuss
 wrote:
> On Thu, May 10, 2018 at 06:03:22PM +0200, Jiri Kosina wrote:
>>On Wed, 9 May 2018, Daniel Vetter wrote:
>>> >> Then, why don't we have a pre-integration tree for fixes? That would
>>> >> at least simply automated testing of fixes separately from new
>>> >> material.
>>> >
>>> >> Perhaps this has already been discussed, and concluded and it's not
>>> >> worth it, then apologize for my ignorance.
>>> >
>>> > I think this is an excellent idea, copying in Stephen for his input.
>>> > I'm currently on holiday but unless someone convinces me it's a terrible
>>> > idea I'm willing to at least give it a go on a trial basis once I'm back
>>> > home.
>>>
>>> Since Stephen merges all -fixes branches first, before merging all the
>>> -next branches, he already generates that as part of linux-next. All
>>> he'd need to do is push that intermediate state out to some
>>> linux-fixes branch for consumption by test bots.
>>
>>What I do for my trees is that I actually merge the '-fixes' branch (that
>>is scheduled to go to Linus in the 'current' cycle) into my for-next
>>branch as well.
>>
>>This has the advantage of (a) getting all the coverage linux-next does (b)
>>seeing any potential merge conflicts early
>>
>>Is this not feasible for other trees?
>
> When Linus tags -rc1, -next will start filling up with commits destined
> for the next merge window. The resulting -next tree becomes very
> unstable, and very difficult to test.
>
> The idea behind next-fixes is to provide a tree that will contain fixes
> for the current merge window, which will generate a much more stable
> tree that users/bots could actually run and validate the fixes that will
> be merged in the upcoming weeks.
>
> Right now, with the method you've described, there is no easy way to
> test your '-fixes' branch even though the commits in there will be
> pulled in by Linus much sooner than your 'for-next' branch.
>
> You'll still get the same coverage from -next, but if you provide your
> -fixes branch seperately you'll also get more coverage for the fixes
> you're about to send to Linus.

I think you missed the "as well" in Jiri's response.

When I create the bi-weekly renesas-drivers release (see e.g.
https://www.spinics.net/lists/linux-renesas-soc/msg27350.html), there are
some subsystems that manage to have several conflicts between their
for-next branch and their fixes in Linus' tree almost every single release.
Hence I strongly support merging your own fixes branches into your own
for-next branch, and resolve the conflicts yourself, to keep your for-next
branch conflict free.

(Note that the last release linked above was very atypical: it was one of
the very few (first one ever?) that didn't have any conflicts).

Thanks!

Gr{oetje,eeting}s,

Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: [Ksummit-discuss] bug-introducing patches

[Geert Uytterhoeven]

On Thu, May 10, 2018 at 6:47 PM, Sasha Levin via Ksummit-discuss
 wrote:
> On Thu, May 10, 2018 at 06:03:22PM +0200, Jiri Kosina wrote:
>>On Wed, 9 May 2018, Daniel Vetter wrote:
>>> >> Then, why don't we have a pre-integration tree for fixes? That would
>>> >> at least simply automated testing of fixes separately from new
>>> >> material.
>>> >
>>> >> Perhaps this has already been discussed, and concluded and it's not
>>> >> worth it, then apologize for my ignorance.
>>> >
>>> > I think this is an excellent idea, copying in Stephen for his input.
>>> > I'm currently on holiday but unless someone convinces me it's a terrible
>>> > idea I'm willing to at least give it a go on a trial basis once I'm back
>>> > home.
>>>
>>> Since Stephen merges all -fixes branches first, before merging all the
>>> -next branches, he already generates that as part of linux-next. All
>>> he'd need to do is push that intermediate state out to some
>>> linux-fixes branch for consumption by test bots.
>>
>>What I do for my trees is that I actually merge the '-fixes' branch (that
>>is scheduled to go to Linus in the 'current' cycle) into my for-next
>>branch as well.
>>
>>This has the advantage of (a) getting all the coverage linux-next does (b)
>>seeing any potential merge conflicts early
>>
>>Is this not feasible for other trees?
>
> When Linus tags -rc1, -next will start filling up with commits destined
> for the next merge window. The resulting -next tree becomes very
> unstable, and very difficult to test.
>
> The idea behind next-fixes is to provide a tree that will contain fixes
> for the current merge window, which will generate a much more stable
> tree that users/bots could actually run and validate the fixes that will
> be merged in the upcoming weeks.
>
> Right now, with the method you've described, there is no easy way to
> test your '-fixes' branch even though the commits in there will be
> pulled in by Linus much sooner than your 'for-next' branch.
>
> You'll still get the same coverage from -next, but if you provide your
> -fixes branch seperately you'll also get more coverage for the fixes
> you're about to send to Linus.

I think you missed the "as well" in Jiri's response.

When I create the bi-weekly renesas-drivers release (see e.g.
https://www.spinics.net/lists/linux-renesas-soc/msg27350.html), there are
some subsystems that manage to have several conflicts between their
for-next branch and their fixes in Linus' tree almost every single release.
Hence I strongly support merging your own fixes branches into your own
for-next branch, and resolve the conflicts yourself, to keep your for-next
branch conflict free.

(Note that the last release linked above was very atypical: it was one of
the very few (first one ever?) that didn't have any conflicts).

Thanks!

Gr{oetje,eeting}s,

Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: [Ksummit-discuss] bug-introducing patches

[Andy Shevchenko]

On Sat, May 12, 2018 at 7:38 AM, Stephen Rothwell  wrote:
> Hi all,
>
> On Wed, 9 May 2018 20:47:27 +1000 Stephen Rothwell  
> wrote:
>>
>> On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
>> >
>> > On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
>> > > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:
>> >
>> > > > I think this is an excellent idea, copying in Stephen for his input.
>> > > > I'm currently on holiday but unless someone convinces me it's a 
>> > > > terrible
>> > > > idea I'm willing to at least give it a go on a trial basis once I'm 
>> > > > back
>> > > > home.
>> >
>> > > Since Stephen merges all -fixes branches first, before merging all the
>> > > -next branches, he already generates that as part of linux-next. All
>> > > he'd need to do is push that intermediate state out to some
>> > > linux-fixes branch for consumption by test bots.
>>
>> Good idea ... I will see what I can do.
>
> See my announcement of a pending-fixes branch in linux-next (on LKML
> and others)
>
>> I currently have 44 such fixes branches.  More welcome!
>
> We are up to 55.

For PDx86 we have

git://git.infradead.org/linux-platform-drivers-x86.git fixes

branch

-- 
With Best Regards,
Andy Shevchenko

Re: [Ksummit-discuss] bug-introducing patches

[Andy Shevchenko]

On Sat, May 12, 2018 at 7:38 AM, Stephen Rothwell  wrote:
> Hi all,
>
> On Wed, 9 May 2018 20:47:27 +1000 Stephen Rothwell  
> wrote:
>>
>> On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
>> >
>> > On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
>> > > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:
>> >
>> > > > I think this is an excellent idea, copying in Stephen for his input.
>> > > > I'm currently on holiday but unless someone convinces me it's a 
>> > > > terrible
>> > > > idea I'm willing to at least give it a go on a trial basis once I'm 
>> > > > back
>> > > > home.
>> >
>> > > Since Stephen merges all -fixes branches first, before merging all the
>> > > -next branches, he already generates that as part of linux-next. All
>> > > he'd need to do is push that intermediate state out to some
>> > > linux-fixes branch for consumption by test bots.
>>
>> Good idea ... I will see what I can do.
>
> See my announcement of a pending-fixes branch in linux-next (on LKML
> and others)
>
>> I currently have 44 such fixes branches.  More welcome!
>
> We are up to 55.

For PDx86 we have

git://git.infradead.org/linux-platform-drivers-x86.git fixes

branch

-- 
With Best Regards,
Andy Shevchenko

Re: [Ksummit-discuss] bug-introducing patches

[Guenter Roeck]

On 05/11/2018 09:38 PM, Stephen Rothwell wrote:

Hi all,

On Wed, 9 May 2018 20:47:27 +1000 Stephen Rothwell  
wrote:


On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:


On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:

On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:
   

I think this is an excellent idea, copying in Stephen for his input.
I'm currently on holiday but unless someone convinces me it's a terrible
idea I'm willing to at least give it a go on a trial basis once I'm back
home.
   

Since Stephen merges all -fixes branches first, before merging all the
-next branches, he already generates that as part of linux-next. All
he'd need to do is push that intermediate state out to some
linux-fixes branch for consumption by test bots.


Good idea ... I will see what I can do.


See my announcement of a pending-fixes branch in linux-next (on LKML
and others)


Excellent.

Build/test results match mainline.

For v4.17-rc4-241-ga1b6c55:

Build results:
total: 132 pass: 130 fail: 2
Failed builds:
m68k:allmodconfig
xtensa:allmodconfig
Qemu test results:
total: 138 pass: 138 fail: 0

Guenter

Re: [Ksummit-discuss] bug-introducing patches

[Guenter Roeck]

On 05/11/2018 09:38 PM, Stephen Rothwell wrote:

Hi all,

On Wed, 9 May 2018 20:47:27 +1000 Stephen Rothwell  
wrote:


On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:


On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:

On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:
   

I think this is an excellent idea, copying in Stephen for his input.
I'm currently on holiday but unless someone convinces me it's a terrible
idea I'm willing to at least give it a go on a trial basis once I'm back
home.
   

Since Stephen merges all -fixes branches first, before merging all the
-next branches, he already generates that as part of linux-next. All
he'd need to do is push that intermediate state out to some
linux-fixes branch for consumption by test bots.


Good idea ... I will see what I can do.


See my announcement of a pending-fixes branch in linux-next (on LKML
and others)


Excellent.

Build/test results match mainline.

For v4.17-rc4-241-ga1b6c55:

Build results:
total: 132 pass: 130 fail: 2
Failed builds:
m68k:allmodconfig
xtensa:allmodconfig
Qemu test results:
total: 138 pass: 138 fail: 0

Guenter

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi all,

On Wed, 9 May 2018 20:47:27 +1000 Stephen Rothwell  
wrote:
>
> On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
> >
> > On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:  
> > > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:   
> > >  
> >   
> > > > I think this is an excellent idea, copying in Stephen for his input.
> > > > I'm currently on holiday but unless someone convinces me it's a terrible
> > > > idea I'm willing to at least give it a go on a trial basis once I'm back
> > > > home.
> >   
> > > Since Stephen merges all -fixes branches first, before merging all the
> > > -next branches, he already generates that as part of linux-next. All
> > > he'd need to do is push that intermediate state out to some
> > > linux-fixes branch for consumption by test bots.  
> 
> Good idea ... I will see what I can do.

See my announcement of a pending-fixes branch in linux-next (on LKML
and others)

> I currently have 44 such fixes branches.  More welcome!

We are up to 55.

-- 
Cheers,
Stephen Rothwell


pgpjVG8LUNEnK.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi all,

On Wed, 9 May 2018 20:47:27 +1000 Stephen Rothwell  
wrote:
>
> On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
> >
> > On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:  
> > > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:   
> > >  
> >   
> > > > I think this is an excellent idea, copying in Stephen for his input.
> > > > I'm currently on holiday but unless someone convinces me it's a terrible
> > > > idea I'm willing to at least give it a go on a trial basis once I'm back
> > > > home.
> >   
> > > Since Stephen merges all -fixes branches first, before merging all the
> > > -next branches, he already generates that as part of linux-next. All
> > > he'd need to do is push that intermediate state out to some
> > > linux-fixes branch for consumption by test bots.  
> 
> Good idea ... I will see what I can do.

See my announcement of a pending-fixes branch in linux-next (on LKML
and others)

> I currently have 44 such fixes branches.  More welcome!

We are up to 55.

-- 
Cheers,
Stephen Rothwell


pgpjVG8LUNEnK.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi David,

On Fri, 11 May 2018 10:47:01 +0200 David Sterba  wrote:
>
> Please add
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git next-fixes

Added from Monday (as btrfs-fixes).

Thanks for adding your subsystem tree as a participant of linux-next.  As
you may know, this is not a judgement of your code.  The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window. 

You will need to ensure that the patches/commits in your tree/series have
been:
 * submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
 * posted to the relevant mailing list,
 * reviewed by you (or another maintainer of your subsystem tree),
 * successfully unit tested, and 
 * destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch).  It is allowed to be rebased if you deem it necessary.

-- 
Cheers,
Stephen Rothwell 
s...@canb.auug.org.au


pgpRAzEmZx0E1.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi David,

On Fri, 11 May 2018 10:47:01 +0200 David Sterba  wrote:
>
> Please add
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git next-fixes

Added from Monday (as btrfs-fixes).

Thanks for adding your subsystem tree as a participant of linux-next.  As
you may know, this is not a judgement of your code.  The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window. 

You will need to ensure that the patches/commits in your tree/series have
been:
 * submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
 * posted to the relevant mailing list,
 * reviewed by you (or another maintainer of your subsystem tree),
 * successfully unit tested, and 
 * destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch).  It is allowed to be rebased if you deem it necessary.

-- 
Cheers,
Stephen Rothwell 
s...@canb.auug.org.au


pgpRAzEmZx0E1.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[David Sterba]

On Wed, May 09, 2018 at 08:47:27PM +1000, Stephen Rothwell wrote:
> > True.  It's currently only those -fixes branches that people have asked
> > him to merge separately which isn't as big a proportion of trees as have
> > them (perhaps fortunately given people's enthusiasm for fixes branches
> > that don't merge cleanly with their development branches) so we'd also
> > need to encourage people to add them separately.
> 
> I currently have 44 such fixes branches.  More welcome!

Please add

  git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git next-fixes

Thanks.

Re: [Ksummit-discuss] bug-introducing patches

[David Sterba]

On Wed, May 09, 2018 at 08:47:27PM +1000, Stephen Rothwell wrote:
> > True.  It's currently only those -fixes branches that people have asked
> > him to merge separately which isn't as big a proportion of trees as have
> > them (perhaps fortunately given people's enthusiasm for fixes branches
> > that don't merge cleanly with their development branches) so we'd also
> > need to encourage people to add them separately.
> 
> I currently have 44 such fixes branches.  More welcome!

Please add

  git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git next-fixes

Thanks.

Re: [Ksummit-discuss] bug-introducing patches

[Mark Brown]

On Thu, May 10, 2018 at 06:03:22PM +0200, Jiri Kosina wrote:
> On Wed, 9 May 2018, Daniel Vetter wrote:

> > Since Stephen merges all -fixes branches first, before merging all the
> > -next branches, he already generates that as part of linux-next. All
> > he'd need to do is push that intermediate state out to some
> > linux-fixes branch for consumption by test bots.

> What I do for my trees is that I actually merge the '-fixes' branch (that 
> is scheduled to go to Linus in the 'current' cycle) into my for-next 
> branch as well.

> This has the advantage of (a) getting all the coverage linux-next does (b) 
> seeing any potential merge conflicts early

> Is this not feasible for other trees?

That's obviously best practice which I hope everyone who doesn't have a
separate fix branch in -next is doing but it means that the fixes branch
is not getting tested without the changes in your -next branch, and also
reduces the coverage separate to other people's -next branches.  This
means that there's room for implicit dependencies to slip through.


signature.asc
Description: PGP signature

Re: [Ksummit-discuss] bug-introducing patches

[Mark Brown]

On Thu, May 10, 2018 at 06:03:22PM +0200, Jiri Kosina wrote:
> On Wed, 9 May 2018, Daniel Vetter wrote:

> > Since Stephen merges all -fixes branches first, before merging all the
> > -next branches, he already generates that as part of linux-next. All
> > he'd need to do is push that intermediate state out to some
> > linux-fixes branch for consumption by test bots.

> What I do for my trees is that I actually merge the '-fixes' branch (that 
> is scheduled to go to Linus in the 'current' cycle) into my for-next 
> branch as well.

> This has the advantage of (a) getting all the coverage linux-next does (b) 
> seeing any potential merge conflicts early

> Is this not feasible for other trees?

That's obviously best practice which I hope everyone who doesn't have a
separate fix branch in -next is doing but it means that the fixes branch
is not getting tested without the changes in your -next branch, and also
reduces the coverage separate to other people's -next branches.  This
means that there's room for implicit dependencies to slip through.


signature.asc
Description: PGP signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Tony,

On Thu, 10 May 2018 08:57:55 -0700 Tony Lindgren  wrote:
>
> * Stephen Rothwell  [180509 10:49]:
> > I currently have 44 such fixes branches.  More welcome!  
> 
> Can you please also add mine:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/tmlind/linux-omap.git fixes

Added from today.

Thanks for adding your subsystem tree as a participant of linux-next.  As
you may know, this is not a judgement of your code.  The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window. 

You will need to ensure that the patches/commits in your tree/series have
been:
 * submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
 * posted to the relevant mailing list,
 * reviewed by you (or another maintainer of your subsystem tree),
 * successfully unit tested, and 
 * destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch).  It is allowed to be rebased if you deem it necessary.

-- 
Cheers,
Stephen Rothwell 
s...@canb.auug.org.au


pgp81XWVcjcJn.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Tony,

On Thu, 10 May 2018 08:57:55 -0700 Tony Lindgren  wrote:
>
> * Stephen Rothwell  [180509 10:49]:
> > I currently have 44 such fixes branches.  More welcome!  
> 
> Can you please also add mine:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/tmlind/linux-omap.git fixes

Added from today.

Thanks for adding your subsystem tree as a participant of linux-next.  As
you may know, this is not a judgement of your code.  The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window. 

You will need to ensure that the patches/commits in your tree/series have
been:
 * submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
 * posted to the relevant mailing list,
 * reviewed by you (or another maintainer of your subsystem tree),
 * successfully unit tested, and 
 * destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch).  It is allowed to be rebased if you deem it necessary.

-- 
Cheers,
Stephen Rothwell 
s...@canb.auug.org.au


pgp81XWVcjcJn.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Mark,

On Thu, 10 May 2018 22:36:28 +0900 Mark Brown  wrote:
>
> On Thu, May 10, 2018 at 08:09:09AM +1000, Stephen Rothwell wrote:
> > On Wed, 9 May 2018 23:05:32 +0900 Mark Brown  wrote:  
> 
> > > Well, all my trees have a for-linus branch to go with the for-next
> > > branch for a start.  
> 
> > The regmap and regulator trees have no for-linus branch (currently).
> > Added sound-asoc-fixes and spi-fixes from today.  
> 
> That's not what git claims when I try to push...  no idea what's going
> on there, I just deleted and repushed - hopefully that helps.

I have added regmap-fixes and regulator-fixes now.

-- 
Cheers,
Stephen Rothwell


pgpCgZxrp_zrr.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Mark,

On Thu, 10 May 2018 22:36:28 +0900 Mark Brown  wrote:
>
> On Thu, May 10, 2018 at 08:09:09AM +1000, Stephen Rothwell wrote:
> > On Wed, 9 May 2018 23:05:32 +0900 Mark Brown  wrote:  
> 
> > > Well, all my trees have a for-linus branch to go with the for-next
> > > branch for a start.  
> 
> > The regmap and regulator trees have no for-linus branch (currently).
> > Added sound-asoc-fixes and spi-fixes from today.  
> 
> That's not what git claims when I try to push...  no idea what's going
> on there, I just deleted and repushed - hopefully that helps.

I have added regmap-fixes and regulator-fixes now.

-- 
Cheers,
Stephen Rothwell


pgpCgZxrp_zrr.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Sasha Levin]

On Thu, May 10, 2018 at 06:03:22PM +0200, Jiri Kosina wrote:
>On Wed, 9 May 2018, Daniel Vetter wrote:
>
>> >> Then, why don't we have a pre-integration tree for fixes? That would
>> >> at least simply automated testing of fixes separately from new
>> >> material.
>> >
>> >> Perhaps this has already been discussed, and concluded and it's not
>> >> worth it, then apologize for my ignorance.
>> >
>> > I think this is an excellent idea, copying in Stephen for his input.
>> > I'm currently on holiday but unless someone convinces me it's a terrible
>> > idea I'm willing to at least give it a go on a trial basis once I'm back
>> > home.
>>
>> Since Stephen merges all -fixes branches first, before merging all the
>> -next branches, he already generates that as part of linux-next. All
>> he'd need to do is push that intermediate state out to some
>> linux-fixes branch for consumption by test bots.
>
>What I do for my trees is that I actually merge the '-fixes' branch (that
>is scheduled to go to Linus in the 'current' cycle) into my for-next
>branch as well.
>
>This has the advantage of (a) getting all the coverage linux-next does (b)
>seeing any potential merge conflicts early
>
>Is this not feasible for other trees?

When Linus tags -rc1, -next will start filling up with commits destined
for the next merge window. The resulting -next tree becomes very
unstable, and very difficult to test.

The idea behind next-fixes is to provide a tree that will contain fixes
for the current merge window, which will generate a much more stable
tree that users/bots could actually run and validate the fixes that will
be merged in the upcoming weeks.

Right now, with the method you've described, there is no easy way to
test your '-fixes' branch even though the commits in there will be
pulled in by Linus much sooner than your 'for-next' branch.

You'll still get the same coverage from -next, but if you provide your
-fixes branch seperately you'll also get more coverage for the fixes
you're about to send to Linus.

Re: [Ksummit-discuss] bug-introducing patches

[Sasha Levin]

On Thu, May 10, 2018 at 06:03:22PM +0200, Jiri Kosina wrote:
>On Wed, 9 May 2018, Daniel Vetter wrote:
>
>> >> Then, why don't we have a pre-integration tree for fixes? That would
>> >> at least simply automated testing of fixes separately from new
>> >> material.
>> >
>> >> Perhaps this has already been discussed, and concluded and it's not
>> >> worth it, then apologize for my ignorance.
>> >
>> > I think this is an excellent idea, copying in Stephen for his input.
>> > I'm currently on holiday but unless someone convinces me it's a terrible
>> > idea I'm willing to at least give it a go on a trial basis once I'm back
>> > home.
>>
>> Since Stephen merges all -fixes branches first, before merging all the
>> -next branches, he already generates that as part of linux-next. All
>> he'd need to do is push that intermediate state out to some
>> linux-fixes branch for consumption by test bots.
>
>What I do for my trees is that I actually merge the '-fixes' branch (that
>is scheduled to go to Linus in the 'current' cycle) into my for-next
>branch as well.
>
>This has the advantage of (a) getting all the coverage linux-next does (b)
>seeing any potential merge conflicts early
>
>Is this not feasible for other trees?

When Linus tags -rc1, -next will start filling up with commits destined
for the next merge window. The resulting -next tree becomes very
unstable, and very difficult to test.

The idea behind next-fixes is to provide a tree that will contain fixes
for the current merge window, which will generate a much more stable
tree that users/bots could actually run and validate the fixes that will
be merged in the upcoming weeks.

Right now, with the method you've described, there is no easy way to
test your '-fixes' branch even though the commits in there will be
pulled in by Linus much sooner than your 'for-next' branch.

You'll still get the same coverage from -next, but if you provide your
-fixes branch seperately you'll also get more coverage for the fixes
you're about to send to Linus.

Re: [Ksummit-discuss] bug-introducing patches

[Sasha Levin]

On Tue, May 08, 2018 at 06:15:34PM -0400, Theodore Y. Ts'o wrote:
>On Tue, May 08, 2018 at 08:29:14PM +, Sasha Levin wrote:
>>
>> This is interesting. We have a group of power users who are testing out
>> -rc releases, who are usually happy to test out a fast moving target and
>> provide helpful reports back. We also have a group who run a -stable
>> kernel (-stable build/distro/android/etc) who want to avoid having to
>> report bugs to us.
>>
>> What we don't have is a group of people who use Linus's actual releases
>> (not the -rc stuff, but the actual point releases). Power users will
>> move on to the next kernel, and -stable folks won't touch that release
>> until there's a corresponding -stable.
>
>Linus doesn't release the point releases.  Those are done by the Greg
>K-H and use the same process as does the stable kernels.  The only
>difference is that the life point release doesn't last very long; just
>until the next kernel release from Linus.
>
>There are probably fewer people who use the point releases compared to
>the stable kernels.  But I'd hesitate to call it zero.  We once
>assumed that companies were all using Distro kernels, and very few
>people used the stable kernels except for distribution channels
>(enterprise kernels, BSP kernels, etc.).  Then we discovered that
>there are people who use the stable kernel and don't go through the
>enterprise distro vendors at all.  It wouldn't surprise me if there
>are also a silent (and perhaps large) set of users who take Linus's
>releases, and then follow along on the dot releases until the next
>release from Linus.

I was referring to Linus's non-rc releases (4.15, 4.16, etc). While many
users start with, for example, 4.16, most users will either switch to
Greg's releases which start about a week after Linus's release, or
they'll move on to test the 4.17 -rc releases.

There are pretty much no users who pick 4.16, stay with it for 3 months,
switch to 4.17, stay with that for 3 months, and so on.

>> What if, instead, Linus doesn't actually ever release a point
>> release?  We can make the merge window open more often, and since
>> there's no actual release, people won't rush to push fixes in later
>> -rc cycles.
>
>I dont' understand your proposal.  Linus doesn't actually release
>point releases.  Those happen during the -rcX cycle for those people
>who are too chicken to follow Linus's tree, and just want the bug
>fixes.

What I'm suggesting is that "4.16" never gets released. When 4.16-rc8 is
closing and Linus would have tagged 4.16, he'd just open the merge
window and start merging in new features.

At this point Greg could either release 4.16.0 or wait a week and do
4.16.1. This effectively puts the kernel on a weekly release schedule.

>Getting rid of the point releases isn't going to change how frequently
>the merge window opens.  What would do that is being much more strict
>about when we only allow regression fixes only into the tree, so
>hopefully the tree stablizes itself by -rc5 or -rc6.
>
>> Merge window will happen more often, so there's no real reason to
>> rush things in a particular window, and since -stable releases every
>> week there's no rush to push a fix in since the next release is just
>> a week away.
>
>Huh?
>
>I can see shortening the release cycle to a six weeks, instead of our
>current 8-9 week cycle.  But after the each cycle where we introduce
>new features, during the merge window / integration phase, we do need
>to have a time when are fixing regression bugs.

What I'm suggesting is that most of the commits in -rc6/7/8 actually fix
bugs introduced in older kernels rather than the current merge window.
Thus, they don't have much value in "stabilizing" the release.

On the other hand, for some odd reason, folks will try squeezing poorly
tested commits into these late -rc cycles because "Linus is about to
release and we must make it in time for the release", even though in
practice there's no big rush to make it to a particular release since
most folks will just keep updating via -stable or distro kernels.

So since there's not much value in -rc6/7/8, just cancel them.

Re: [Ksummit-discuss] bug-introducing patches

[Sasha Levin]

On Tue, May 08, 2018 at 06:15:34PM -0400, Theodore Y. Ts'o wrote:
>On Tue, May 08, 2018 at 08:29:14PM +, Sasha Levin wrote:
>>
>> This is interesting. We have a group of power users who are testing out
>> -rc releases, who are usually happy to test out a fast moving target and
>> provide helpful reports back. We also have a group who run a -stable
>> kernel (-stable build/distro/android/etc) who want to avoid having to
>> report bugs to us.
>>
>> What we don't have is a group of people who use Linus's actual releases
>> (not the -rc stuff, but the actual point releases). Power users will
>> move on to the next kernel, and -stable folks won't touch that release
>> until there's a corresponding -stable.
>
>Linus doesn't release the point releases.  Those are done by the Greg
>K-H and use the same process as does the stable kernels.  The only
>difference is that the life point release doesn't last very long; just
>until the next kernel release from Linus.
>
>There are probably fewer people who use the point releases compared to
>the stable kernels.  But I'd hesitate to call it zero.  We once
>assumed that companies were all using Distro kernels, and very few
>people used the stable kernels except for distribution channels
>(enterprise kernels, BSP kernels, etc.).  Then we discovered that
>there are people who use the stable kernel and don't go through the
>enterprise distro vendors at all.  It wouldn't surprise me if there
>are also a silent (and perhaps large) set of users who take Linus's
>releases, and then follow along on the dot releases until the next
>release from Linus.

I was referring to Linus's non-rc releases (4.15, 4.16, etc). While many
users start with, for example, 4.16, most users will either switch to
Greg's releases which start about a week after Linus's release, or
they'll move on to test the 4.17 -rc releases.

There are pretty much no users who pick 4.16, stay with it for 3 months,
switch to 4.17, stay with that for 3 months, and so on.

>> What if, instead, Linus doesn't actually ever release a point
>> release?  We can make the merge window open more often, and since
>> there's no actual release, people won't rush to push fixes in later
>> -rc cycles.
>
>I dont' understand your proposal.  Linus doesn't actually release
>point releases.  Those happen during the -rcX cycle for those people
>who are too chicken to follow Linus's tree, and just want the bug
>fixes.

What I'm suggesting is that "4.16" never gets released. When 4.16-rc8 is
closing and Linus would have tagged 4.16, he'd just open the merge
window and start merging in new features.

At this point Greg could either release 4.16.0 or wait a week and do
4.16.1. This effectively puts the kernel on a weekly release schedule.

>Getting rid of the point releases isn't going to change how frequently
>the merge window opens.  What would do that is being much more strict
>about when we only allow regression fixes only into the tree, so
>hopefully the tree stablizes itself by -rc5 or -rc6.
>
>> Merge window will happen more often, so there's no real reason to
>> rush things in a particular window, and since -stable releases every
>> week there's no rush to push a fix in since the next release is just
>> a week away.
>
>Huh?
>
>I can see shortening the release cycle to a six weeks, instead of our
>current 8-9 week cycle.  But after the each cycle where we introduce
>new features, during the merge window / integration phase, we do need
>to have a time when are fixing regression bugs.

What I'm suggesting is that most of the commits in -rc6/7/8 actually fix
bugs introduced in older kernels rather than the current merge window.
Thus, they don't have much value in "stabilizing" the release.

On the other hand, for some odd reason, folks will try squeezing poorly
tested commits into these late -rc cycles because "Linus is about to
release and we must make it in time for the release", even though in
practice there's no big rush to make it to a particular release since
most folks will just keep updating via -stable or distro kernels.

So since there's not much value in -rc6/7/8, just cancel them.

Re: [Ksummit-discuss] bug-introducing patches

[Jiri Kosina]

On Wed, 9 May 2018, Daniel Vetter wrote:

> >> Then, why don't we have a pre-integration tree for fixes? That would
> >> at least simply automated testing of fixes separately from new
> >> material.
> >
> >> Perhaps this has already been discussed, and concluded and it's not
> >> worth it, then apologize for my ignorance.
> >
> > I think this is an excellent idea, copying in Stephen for his input.
> > I'm currently on holiday but unless someone convinces me it's a terrible
> > idea I'm willing to at least give it a go on a trial basis once I'm back
> > home.
> 
> Since Stephen merges all -fixes branches first, before merging all the
> -next branches, he already generates that as part of linux-next. All
> he'd need to do is push that intermediate state out to some
> linux-fixes branch for consumption by test bots.

What I do for my trees is that I actually merge the '-fixes' branch (that 
is scheduled to go to Linus in the 'current' cycle) into my for-next 
branch as well.

This has the advantage of (a) getting all the coverage linux-next does (b) 
seeing any potential merge conflicts early

Is this not feasible for other trees?

-- 
Jiri Kosina
SUSE Labs

Re: [Ksummit-discuss] bug-introducing patches

[Jiri Kosina]

On Wed, 9 May 2018, Daniel Vetter wrote:

> >> Then, why don't we have a pre-integration tree for fixes? That would
> >> at least simply automated testing of fixes separately from new
> >> material.
> >
> >> Perhaps this has already been discussed, and concluded and it's not
> >> worth it, then apologize for my ignorance.
> >
> > I think this is an excellent idea, copying in Stephen for his input.
> > I'm currently on holiday but unless someone convinces me it's a terrible
> > idea I'm willing to at least give it a go on a trial basis once I'm back
> > home.
> 
> Since Stephen merges all -fixes branches first, before merging all the
> -next branches, he already generates that as part of linux-next. All
> he'd need to do is push that intermediate state out to some
> linux-fixes branch for consumption by test bots.

What I do for my trees is that I actually merge the '-fixes' branch (that 
is scheduled to go to Linus in the 'current' cycle) into my for-next 
branch as well.

This has the advantage of (a) getting all the coverage linux-next does (b) 
seeing any potential merge conflicts early

Is this not feasible for other trees?

-- 
Jiri Kosina
SUSE Labs

Re: [Ksummit-discuss] bug-introducing patches

[Tony Lindgren]

* Stephen Rothwell  [180509 10:49]:
> I currently have 44 such fixes branches.  More welcome!

Can you please also add mine:

git://git.kernel.org/pub/scm/linux/kernel/git/tmlind/linux-omap.git fixes

Thanks,

Tony




signature.asc
Description: PGP signature

Re: [Ksummit-discuss] bug-introducing patches

[Tony Lindgren]

* Stephen Rothwell  [180509 10:49]:
> I currently have 44 such fixes branches.  More welcome!

Can you please also add mine:

git://git.kernel.org/pub/scm/linux/kernel/git/tmlind/linux-omap.git fixes

Thanks,

Tony




signature.asc
Description: PGP signature

Re: [Ksummit-discuss] bug-introducing patches

[Tony Lindgren]

* Tony Lindgren  [180508 14:52]:
> * Theodore Y. Ts'o  [180508 03:50]:
> > Would I pull down linux-next, and fire up a VM running gce-xfstests?
> > Sure.  But that's not a real-life use case; that's just running canned
> > test cases.  And more often than not, linux-next will be broken while
> > Linus's -rcX tree is just fine; which is why I do most of my ext4
> > testing using patches based on top of -rcX, not based on top of
> > linux-next.
> 
> Ideally we would somehow always end up with an -rc1 that people dare
> to use though for the "prosumer" testing :)

BTW, the reason why I think we all should test Linux next on regular
basis is that it's often "some other people's branches(tm)" that cause
the regressions :) Maybe because their own test cases did not show
any regressions, or because they were unable to test the patches.
Or it's because of some "clean-up" work that's completely untested
on some systems.

And that's how we end up with regressions getting merged into -rc1.

Regards,

Tony

Re: [Ksummit-discuss] bug-introducing patches

[Tony Lindgren]

* Tony Lindgren  [180508 14:52]:
> * Theodore Y. Ts'o  [180508 03:50]:
> > Would I pull down linux-next, and fire up a VM running gce-xfstests?
> > Sure.  But that's not a real-life use case; that's just running canned
> > test cases.  And more often than not, linux-next will be broken while
> > Linus's -rcX tree is just fine; which is why I do most of my ext4
> > testing using patches based on top of -rcX, not based on top of
> > linux-next.
> 
> Ideally we would somehow always end up with an -rc1 that people dare
> to use though for the "prosumer" testing :)

BTW, the reason why I think we all should test Linux next on regular
basis is that it's often "some other people's branches(tm)" that cause
the regressions :) Maybe because their own test cases did not show
any regressions, or because they were unable to test the patches.
Or it's because of some "clean-up" work that's completely untested
on some systems.

And that's how we end up with regressions getting merged into -rc1.

Regards,

Tony

Re: [Ksummit-discuss] bug-introducing patches

[Mark Brown]

On Thu, May 10, 2018 at 08:09:09AM +1000, Stephen Rothwell wrote:
> On Wed, 9 May 2018 23:05:32 +0900 Mark Brown  wrote:

> > Well, all my trees have a for-linus branch to go with the for-next
> > branch for a start.

> The regmap and regulator trees have no for-linus branch (currently).
> Added sound-asoc-fixes and spi-fixes from today.

That's not what git claims when I try to push...  no idea what's going
on there, I just deleted and repushed - hopefully that helps.


signature.asc
Description: PGP signature

Re: [Ksummit-discuss] bug-introducing patches

[Mark Brown]

On Thu, May 10, 2018 at 08:09:09AM +1000, Stephen Rothwell wrote:
> On Wed, 9 May 2018 23:05:32 +0900 Mark Brown  wrote:

> > Well, all my trees have a for-linus branch to go with the for-next
> > branch for a start.

> The regmap and regulator trees have no for-linus branch (currently).
> Added sound-asoc-fixes and spi-fixes from today.

That's not what git claims when I try to push...  no idea what's going
on there, I just deleted and repushed - hopefully that helps.


signature.asc
Description: PGP signature

Re: [Ksummit-discuss] bug-introducing patches

[Sasha Levin]

On Wed, May 09, 2018 at 08:47:27PM +1000, Stephen Rothwell wrote:
>On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
>>
>> On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
>> > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:
>>
>> > > I think this is an excellent idea, copying in Stephen for his input.
>> > > I'm currently on holiday but unless someone convinces me it's a terrible
>> > > idea I'm willing to at least give it a go on a trial basis once I'm back
>> > > home.
>>
>> > Since Stephen merges all -fixes branches first, before merging all the
>> > -next branches, he already generates that as part of linux-next. All
>> > he'd need to do is push that intermediate state out to some
>> > linux-fixes branch for consumption by test bots.
>
>Good idea ... I will see what I can do.

This is very interesting! I'm curious how the statistics will look when
we'll compare patches that didn't go through this tree, patches that
spent minimal time in this tree, and patches that has spent some time
in the tree before being merged in Linus's tree.

Would this be something we would want to point actual users to, rather
than just bots? If every commit in next-fixes is essentially queued up
for Linus at some point, users might as well test out next-fixes instead
of -rc.

>> True.  It's currently only those -fixes branches that people have asked
>> him to merge separately which isn't as big a proportion of trees as have
>> them (perhaps fortunately given people's enthusiasm for fixes branches
>> that don't merge cleanly with their development branches) so we'd also
>> need to encourage people to add them separately.
>
>I currently have 44 such fixes branches.  More welcome!

I've tried looking at git for bigger subsystems, and a few branches that
would fit this description, and are not in -next are:

git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git perf/urgent
git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git sched/urgent
git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git x86/urgent
git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers/urgent
git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/urgent
git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git core/urgent
git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git efi/urgent
git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git locking/urgent
git.kernel.dk/linux-block.git for-linus
git.kernel.org/pub/scm/linux/kernel/git/s390/linux.git fixes
git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git for-linux
git.kernel.org/pub/scm/linux/kernel/git/iwlwifi/iwlwifi-fixes.git master
git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi.git fixes
git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git queue-rc
git.kernel.org/pub/scm/linux/kernel/git/jikos/hid.git for-linus
git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git ftrace/urgent

Would it make sense to ping the respective maintainers to ack the
inclusion of these branches?

There are a few other trees where the fixes branch has a name that
depends on the release, we can ask them to also create a simple fixes
branch so that next-fixes could merge it in?

git.kernel.org/pub/scm/fs/xfs/xfs-linux.git xfs-4.17-fixes
git.kernel.org/pub/scm/linux/kernel/git/rw/uml.git for-linux-4.17-rc1
git.kernel.org/pub/scm/linux/kernel/git/khilman/linux-amlogic.git v4.17/fixes

Re: [Ksummit-discuss] bug-introducing patches

[Sasha Levin]

On Wed, May 09, 2018 at 08:47:27PM +1000, Stephen Rothwell wrote:
>On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
>>
>> On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
>> > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:
>>
>> > > I think this is an excellent idea, copying in Stephen for his input.
>> > > I'm currently on holiday but unless someone convinces me it's a terrible
>> > > idea I'm willing to at least give it a go on a trial basis once I'm back
>> > > home.
>>
>> > Since Stephen merges all -fixes branches first, before merging all the
>> > -next branches, he already generates that as part of linux-next. All
>> > he'd need to do is push that intermediate state out to some
>> > linux-fixes branch for consumption by test bots.
>
>Good idea ... I will see what I can do.

This is very interesting! I'm curious how the statistics will look when
we'll compare patches that didn't go through this tree, patches that
spent minimal time in this tree, and patches that has spent some time
in the tree before being merged in Linus's tree.

Would this be something we would want to point actual users to, rather
than just bots? If every commit in next-fixes is essentially queued up
for Linus at some point, users might as well test out next-fixes instead
of -rc.

>> True.  It's currently only those -fixes branches that people have asked
>> him to merge separately which isn't as big a proportion of trees as have
>> them (perhaps fortunately given people's enthusiasm for fixes branches
>> that don't merge cleanly with their development branches) so we'd also
>> need to encourage people to add them separately.
>
>I currently have 44 such fixes branches.  More welcome!

I've tried looking at git for bigger subsystems, and a few branches that
would fit this description, and are not in -next are:

git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git perf/urgent
git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git sched/urgent
git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git x86/urgent
git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers/urgent
git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git irq/urgent
git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git core/urgent
git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git efi/urgent
git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git locking/urgent
git.kernel.dk/linux-block.git for-linus
git.kernel.org/pub/scm/linux/kernel/git/s390/linux.git fixes
git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse.git for-linux
git.kernel.org/pub/scm/linux/kernel/git/iwlwifi/iwlwifi-fixes.git master
git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi.git fixes
git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git queue-rc
git.kernel.org/pub/scm/linux/kernel/git/jikos/hid.git for-linus
git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace.git ftrace/urgent

Would it make sense to ping the respective maintainers to ack the
inclusion of these branches?

There are a few other trees where the fixes branch has a name that
depends on the release, we can ask them to also create a simple fixes
branch so that next-fixes could merge it in?

git.kernel.org/pub/scm/fs/xfs/xfs-linux.git xfs-4.17-fixes
git.kernel.org/pub/scm/linux/kernel/git/rw/uml.git for-linux-4.17-rc1
git.kernel.org/pub/scm/linux/kernel/git/khilman/linux-amlogic.git v4.17/fixes

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Mark,

On Wed, 9 May 2018 23:05:32 +0900 Mark Brown  wrote:
>
> Well, all my trees have a for-linus branch to go with the for-next
> branch for a start.

The regmap and regulator trees have no for-linus branch (currently).
Added sound-asoc-fixes and spi-fixes from today.

Thanks for adding your subsystem tree as a participant of linux-next.  As
you may know, this is not a judgement of your code.  The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window. 

You will need to ensure that the patches/commits in your tree/series have
been:
 * submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
 * posted to the relevant mailing list,
 * reviewed by you (or another maintainer of your subsystem tree),
 * successfully unit tested, and 
 * destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch).  It is allowed to be rebased if you deem it necessary.

-- 
Cheers,
Stephen Rothwell 
s...@canb.auug.org.au


pgpPlz6HmaT24.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Mark,

On Wed, 9 May 2018 23:05:32 +0900 Mark Brown  wrote:
>
> Well, all my trees have a for-linus branch to go with the for-next
> branch for a start.

The regmap and regulator trees have no for-linus branch (currently).
Added sound-asoc-fixes and spi-fixes from today.

Thanks for adding your subsystem tree as a participant of linux-next.  As
you may know, this is not a judgement of your code.  The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window. 

You will need to ensure that the patches/commits in your tree/series have
been:
 * submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
 * posted to the relevant mailing list,
 * reviewed by you (or another maintainer of your subsystem tree),
 * successfully unit tested, and 
 * destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch).  It is allowed to be rebased if you deem it necessary.

-- 
Cheers,
Stephen Rothwell 
s...@canb.auug.org.au


pgpPlz6HmaT24.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Boris,

On Wed, 9 May 2018 21:35:28 +0200 Boris Brezillon  
wrote:
>
> I see that the nand/fixes and spi-nor/fixes branch are already there [1].
> You can add:
> 
> mtd-fixes git git://git.infradead.org/linux-mtd.git#master

Added from today.

> You can also remove the mtd entry [2], since mtd-2.6.git is just a sym
> link to linux-mtd.git, so it will just be a duplicate of the mtd-fixes
> entry. You can also rename the l2-mtd entry [3] into mtd.

All done.

Thanks for adding your subsystem tree as a participant of linux-next.  As
you may know, this is not a judgement of your code.  The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window. 

You will need to ensure that the patches/commits in your tree/series have
been:
 * submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
 * posted to the relevant mailing list,
 * reviewed by you (or another maintainer of your subsystem tree),
 * successfully unit tested, and 
 * destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch).  It is allowed to be rebased if you deem it necessary.

-- 
Cheers,
Stephen Rothwell 
s...@canb.auug.org.au


pgppjLBapvudz.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Boris,

On Wed, 9 May 2018 21:35:28 +0200 Boris Brezillon  
wrote:
>
> I see that the nand/fixes and spi-nor/fixes branch are already there [1].
> You can add:
> 
> mtd-fixes git git://git.infradead.org/linux-mtd.git#master

Added from today.

> You can also remove the mtd entry [2], since mtd-2.6.git is just a sym
> link to linux-mtd.git, so it will just be a duplicate of the mtd-fixes
> entry. You can also rename the l2-mtd entry [3] into mtd.

All done.

Thanks for adding your subsystem tree as a participant of linux-next.  As
you may know, this is not a judgement of your code.  The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window. 

You will need to ensure that the patches/commits in your tree/series have
been:
 * submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
 * posted to the relevant mailing list,
 * reviewed by you (or another maintainer of your subsystem tree),
 * successfully unit tested, and 
 * destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch).  It is allowed to be rebased if you deem it necessary.

-- 
Cheers,
Stephen Rothwell 
s...@canb.auug.org.au


pgppjLBapvudz.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Dan,

On Wed, 9 May 2018 09:04:31 -0700 Dan Williams  wrote:
>
> Please add:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm.git
> libnvdimm-fixes
> 
> We currently merge this into libnvdimm-for-next for -next coverage,
> and resolve any conflicts vs new development. Do you want to see those
> conflicts? Otherwise I would recommend only pulling libnvdimm-for-next
> for -next and libnvdimm-fixes for this new -next-fixes effort.

The conflicts are usually fine (but if you do the merges, I won't see
them - which is even better :-)).  I like to have the fixes branches in
linux-next so that noone has to worry about probelsm in Linus' tree
that have pending fixes already.

Added from today.

Thanks for adding your subsystem tree as a participant of linux-next.  As
you may know, this is not a judgement of your code.  The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window. 

You will need to ensure that the patches/commits in your tree/series have
been:
 * submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
 * posted to the relevant mailing list,
 * reviewed by you (or another maintainer of your subsystem tree),
 * successfully unit tested, and 
 * destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch).  It is allowed to be rebased if you deem it necessary.

-- 
Cheers,
Stephen Rothwell 
s...@canb.auug.org.au


pgpRYyFeeZ1J6.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Dan,

On Wed, 9 May 2018 09:04:31 -0700 Dan Williams  wrote:
>
> Please add:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm.git
> libnvdimm-fixes
> 
> We currently merge this into libnvdimm-for-next for -next coverage,
> and resolve any conflicts vs new development. Do you want to see those
> conflicts? Otherwise I would recommend only pulling libnvdimm-for-next
> for -next and libnvdimm-fixes for this new -next-fixes effort.

The conflicts are usually fine (but if you do the merges, I won't see
them - which is even better :-)).  I like to have the fixes branches in
linux-next so that noone has to worry about probelsm in Linus' tree
that have pending fixes already.

Added from today.

Thanks for adding your subsystem tree as a participant of linux-next.  As
you may know, this is not a judgement of your code.  The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window. 

You will need to ensure that the patches/commits in your tree/series have
been:
 * submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
 * posted to the relevant mailing list,
 * reviewed by you (or another maintainer of your subsystem tree),
 * successfully unit tested, and 
 * destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch).  It is allowed to be rebased if you deem it necessary.

-- 
Cheers,
Stephen Rothwell 
s...@canb.auug.org.au


pgpRYyFeeZ1J6.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Guenter,

On Wed, 9 May 2018 08:57:33 -0700 Guenter Roeck  wrote:
>
> Please add
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/groeck/linux-staging.git hwmon
> 
> as fixes branch.

Added from today.

Thanks for adding your subsystem tree as a participant of linux-next.  As
you may know, this is not a judgement of your code.  The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window. 

You will need to ensure that the patches/commits in your tree/series have
been:
 * submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
 * posted to the relevant mailing list,
 * reviewed by you (or another maintainer of your subsystem tree),
 * successfully unit tested, and 
 * destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch).  It is allowed to be rebased if you deem it necessary.

-- 
Cheers,
Stephen Rothwell 
s...@canb.auug.org.au


pgpuzQtRDCBsQ.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Guenter,

On Wed, 9 May 2018 08:57:33 -0700 Guenter Roeck  wrote:
>
> Please add
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/groeck/linux-staging.git hwmon
> 
> as fixes branch.

Added from today.

Thanks for adding your subsystem tree as a participant of linux-next.  As
you may know, this is not a judgement of your code.  The purpose of
linux-next is for integration testing and to lower the impact of
conflicts between subsystems in the next merge window. 

You will need to ensure that the patches/commits in your tree/series have
been:
 * submitted under GPL v2 (or later) and include the Contributor's
Signed-off-by,
 * posted to the relevant mailing list,
 * reviewed by you (or another maintainer of your subsystem tree),
 * successfully unit tested, and 
 * destined for the current or next Linux merge window.

Basically, this should be just what you would send to Linus (or ask him
to fetch).  It is allowed to be rebased if you deem it necessary.

-- 
Cheers,
Stephen Rothwell 
s...@canb.auug.org.au


pgpuzQtRDCBsQ.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Boris Brezillon]

Hi Stephen,

On Wed, 9 May 2018 20:47:27 +1000
Stephen Rothwell  wrote:

> On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
> >
> > On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:  
> > > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:   
> > >  
> >   
> > > > I think this is an excellent idea, copying in Stephen for his input.
> > > > I'm currently on holiday but unless someone convinces me it's a terrible
> > > > idea I'm willing to at least give it a go on a trial basis once I'm back
> > > > home.
> >   
> > > Since Stephen merges all -fixes branches first, before merging all the
> > > -next branches, he already generates that as part of linux-next. All
> > > he'd need to do is push that intermediate state out to some
> > > linux-fixes branch for consumption by test bots.  
> 
> Good idea ... I will see what I can do.
>   
> > True.  It's currently only those -fixes branches that people have asked
> > him to merge separately which isn't as big a proportion of trees as have
> > them (perhaps fortunately given people's enthusiasm for fixes branches
> > that don't merge cleanly with their development branches) so we'd also
> > need to encourage people to add them separately.  
> 
> I currently have 44 such fixes branches.  More welcome!

I see that the nand/fixes and spi-nor/fixes branch are already there [1].
You can add:

mtd-fixes   git git://git.infradead.org/linux-mtd.git#master

You can also remove the mtd entry [2], since mtd-2.6.git is just a sym
link to linux-mtd.git, so it will just be a duplicate of the mtd-fixes
entry. You can also rename the l2-mtd entry [3] into mtd.

Regards,

Boris

[1]https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next/+/43cd1f4979998ba0ef1c0b8e1c5d23d2de5ab172/Next/Trees#41
[2]https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next/+/43cd1f4979998ba0ef1c0b8e1c5d23d2de5ab172/Next/Trees#155
[3]https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next/+/43cd1f4979998ba0ef1c0b8e1c5d23d2de5ab172/Next/Trees#156

Re: [Ksummit-discuss] bug-introducing patches

[Boris Brezillon]

Hi Stephen,

On Wed, 9 May 2018 20:47:27 +1000
Stephen Rothwell  wrote:

> On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
> >
> > On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:  
> > > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:   
> > >  
> >   
> > > > I think this is an excellent idea, copying in Stephen for his input.
> > > > I'm currently on holiday but unless someone convinces me it's a terrible
> > > > idea I'm willing to at least give it a go on a trial basis once I'm back
> > > > home.
> >   
> > > Since Stephen merges all -fixes branches first, before merging all the
> > > -next branches, he already generates that as part of linux-next. All
> > > he'd need to do is push that intermediate state out to some
> > > linux-fixes branch for consumption by test bots.  
> 
> Good idea ... I will see what I can do.
>   
> > True.  It's currently only those -fixes branches that people have asked
> > him to merge separately which isn't as big a proportion of trees as have
> > them (perhaps fortunately given people's enthusiasm for fixes branches
> > that don't merge cleanly with their development branches) so we'd also
> > need to encourage people to add them separately.  
> 
> I currently have 44 such fixes branches.  More welcome!

I see that the nand/fixes and spi-nor/fixes branch are already there [1].
You can add:

mtd-fixes   git git://git.infradead.org/linux-mtd.git#master

You can also remove the mtd entry [2], since mtd-2.6.git is just a sym
link to linux-mtd.git, so it will just be a duplicate of the mtd-fixes
entry. You can also rename the l2-mtd entry [3] into mtd.

Regards,

Boris

[1]https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next/+/43cd1f4979998ba0ef1c0b8e1c5d23d2de5ab172/Next/Trees#41
[2]https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next/+/43cd1f4979998ba0ef1c0b8e1c5d23d2de5ab172/Next/Trees#155
[3]https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next/+/43cd1f4979998ba0ef1c0b8e1c5d23d2de5ab172/Next/Trees#156

Re: [Ksummit-discuss] bug-introducing patches

[Dan Williams]

On Wed, May 9, 2018 at 3:47 AM, Stephen Rothwell  wrote:
> On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
>>
>> On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
>> > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:
>>
>> > > I think this is an excellent idea, copying in Stephen for his input.
>> > > I'm currently on holiday but unless someone convinces me it's a terrible
>> > > idea I'm willing to at least give it a go on a trial basis once I'm back
>> > > home.
>>
>> > Since Stephen merges all -fixes branches first, before merging all the
>> > -next branches, he already generates that as part of linux-next. All
>> > he'd need to do is push that intermediate state out to some
>> > linux-fixes branch for consumption by test bots.
>
> Good idea ... I will see what I can do.
>
>> True.  It's currently only those -fixes branches that people have asked
>> him to merge separately which isn't as big a proportion of trees as have
>> them (perhaps fortunately given people's enthusiasm for fixes branches
>> that don't merge cleanly with their development branches) so we'd also
>> need to encourage people to add them separately.
>
> I currently have 44 such fixes branches.  More welcome!

Please add:

git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm.git
libnvdimm-fixes

We currently merge this into libnvdimm-for-next for -next coverage,
and resolve any conflicts vs new development. Do you want to see those
conflicts? Otherwise I would recommend only pulling libnvdimm-for-next
for -next and libnvdimm-fixes for this new -next-fixes effort.

Re: [Ksummit-discuss] bug-introducing patches

[Dan Williams]

On Wed, May 9, 2018 at 3:47 AM, Stephen Rothwell  wrote:
> On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
>>
>> On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
>> > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:
>>
>> > > I think this is an excellent idea, copying in Stephen for his input.
>> > > I'm currently on holiday but unless someone convinces me it's a terrible
>> > > idea I'm willing to at least give it a go on a trial basis once I'm back
>> > > home.
>>
>> > Since Stephen merges all -fixes branches first, before merging all the
>> > -next branches, he already generates that as part of linux-next. All
>> > he'd need to do is push that intermediate state out to some
>> > linux-fixes branch for consumption by test bots.
>
> Good idea ... I will see what I can do.
>
>> True.  It's currently only those -fixes branches that people have asked
>> him to merge separately which isn't as big a proportion of trees as have
>> them (perhaps fortunately given people's enthusiasm for fixes branches
>> that don't merge cleanly with their development branches) so we'd also
>> need to encourage people to add them separately.
>
> I currently have 44 such fixes branches.  More welcome!

Please add:

git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm.git
libnvdimm-fixes

We currently merge this into libnvdimm-for-next for -next coverage,
and resolve any conflicts vs new development. Do you want to see those
conflicts? Otherwise I would recommend only pulling libnvdimm-for-next
for -next and libnvdimm-fixes for this new -next-fixes effort.

Re: [Ksummit-discuss] bug-introducing patches

[Guenter Roeck]

On Wed, May 09, 2018 at 08:47:27PM +1000, Stephen Rothwell wrote:
> On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
> >
> > On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
> > > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:  
> > 
> > > > I think this is an excellent idea, copying in Stephen for his input.
> > > > I'm currently on holiday but unless someone convinces me it's a terrible
> > > > idea I'm willing to at least give it a go on a trial basis once I'm back
> > > > home.  
> > 
> > > Since Stephen merges all -fixes branches first, before merging all the
> > > -next branches, he already generates that as part of linux-next. All
> > > he'd need to do is push that intermediate state out to some
> > > linux-fixes branch for consumption by test bots.
> 
> Good idea ... I will see what I can do.
>   
> > True.  It's currently only those -fixes branches that people have asked
> > him to merge separately which isn't as big a proportion of trees as have
> > them (perhaps fortunately given people's enthusiasm for fixes branches
> > that don't merge cleanly with their development branches) so we'd also
> > need to encourage people to add them separately.
> 
> I currently have 44 such fixes branches.  More welcome!
> 
Please add

git://git.kernel.org/pub/scm/linux/kernel/git/groeck/linux-staging.git hwmon

as fixes branch.

Thanks,
Guenter

> -- 
> Cheers,
> Stephen Rothwell



> ___
> Ksummit-discuss mailing list
> ksummit-disc...@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] bug-introducing patches

[Guenter Roeck]

On Wed, May 09, 2018 at 08:47:27PM +1000, Stephen Rothwell wrote:
> On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
> >
> > On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
> > > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:  
> > 
> > > > I think this is an excellent idea, copying in Stephen for his input.
> > > > I'm currently on holiday but unless someone convinces me it's a terrible
> > > > idea I'm willing to at least give it a go on a trial basis once I'm back
> > > > home.  
> > 
> > > Since Stephen merges all -fixes branches first, before merging all the
> > > -next branches, he already generates that as part of linux-next. All
> > > he'd need to do is push that intermediate state out to some
> > > linux-fixes branch for consumption by test bots.
> 
> Good idea ... I will see what I can do.
>   
> > True.  It's currently only those -fixes branches that people have asked
> > him to merge separately which isn't as big a proportion of trees as have
> > them (perhaps fortunately given people's enthusiasm for fixes branches
> > that don't merge cleanly with their development branches) so we'd also
> > need to encourage people to add them separately.
> 
> I currently have 44 such fixes branches.  More welcome!
> 
Please add

git://git.kernel.org/pub/scm/linux/kernel/git/groeck/linux-staging.git hwmon

as fixes branch.

Thanks,
Guenter

> -- 
> Cheers,
> Stephen Rothwell



> ___
> Ksummit-discuss mailing list
> ksummit-disc...@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/ksummit-discuss

Re: [Ksummit-discuss] bug-introducing patches

[Mark Brown]

On Wed, May 09, 2018 at 08:47:27PM +1000, Stephen Rothwell wrote:
> On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
> > On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
> > > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:  

> > True.  It's currently only those -fixes branches that people have asked
> > him to merge separately which isn't as big a proportion of trees as have
> > them (perhaps fortunately given people's enthusiasm for fixes branches
> > that don't merge cleanly with their development branches) so we'd also
> > need to encourage people to add them separately.

> I currently have 44 such fixes branches.  More welcome!

Well, all my trees have a for-linus branch to go with the for-next
branch for a start.


signature.asc
Description: PGP signature

Re: [Ksummit-discuss] bug-introducing patches

[Mark Brown]

On Wed, May 09, 2018 at 08:47:27PM +1000, Stephen Rothwell wrote:
> On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
> > On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
> > > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:  

> > True.  It's currently only those -fixes branches that people have asked
> > him to merge separately which isn't as big a proportion of trees as have
> > them (perhaps fortunately given people's enthusiasm for fixes branches
> > that don't merge cleanly with their development branches) so we'd also
> > need to encourage people to add them separately.

> I currently have 44 such fixes branches.  More welcome!

Well, all my trees have a for-linus branch to go with the for-next
branch for a start.


signature.asc
Description: PGP signature

Re: [Ksummit-discuss] bug-introducing patches

[Vinod Koul]

On 09-05-18, 22:43, Stephen Rothwell wrote:
> Hi Vinod,
> 
> On Wed, 9 May 2018 16:25:34 +0530 Vinod Koul  wrote:
> > > 
> > > I currently have 44 such fixes branches.  More welcome!  
> > 
> > Great so do you want us to send fixes branch or scan the existing trees and 
> > add
> > them.
> 
> The former.
> 
> > In case of former please do add slave-dma/fixes as well
> 
> Done.  Should I switch your contact address to your kernel.org one
> (from your Intel one)?

Yes please, that is no longer valid.

-- 
~Vinod

Re: [Ksummit-discuss] bug-introducing patches

[Vinod Koul]

On 09-05-18, 22:43, Stephen Rothwell wrote:
> Hi Vinod,
> 
> On Wed, 9 May 2018 16:25:34 +0530 Vinod Koul  wrote:
> > > 
> > > I currently have 44 such fixes branches.  More welcome!  
> > 
> > Great so do you want us to send fixes branch or scan the existing trees and 
> > add
> > them.
> 
> The former.
> 
> > In case of former please do add slave-dma/fixes as well
> 
> Done.  Should I switch your contact address to your kernel.org one
> (from your Intel one)?

Yes please, that is no longer valid.

-- 
~Vinod

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Vinod,

On Wed, 9 May 2018 16:25:34 +0530 Vinod Koul  wrote:
> > 
> > I currently have 44 such fixes branches.  More welcome!  
> 
> Great so do you want us to send fixes branch or scan the existing trees and 
> add
> them.

The former.

> In case of former please do add slave-dma/fixes as well

Done.  Should I switch your contact address to your kernel.org one
(from your Intel one)?

-- 
Cheers,
Stephen Rothwell


pgpIRIciw4SrD.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

Hi Vinod,

On Wed, 9 May 2018 16:25:34 +0530 Vinod Koul  wrote:
> > 
> > I currently have 44 such fixes branches.  More welcome!  
> 
> Great so do you want us to send fixes branch or scan the existing trees and 
> add
> them.

The former.

> In case of former please do add slave-dma/fixes as well

Done.  Should I switch your contact address to your kernel.org one
(from your Intel one)?

-- 
Cheers,
Stephen Rothwell


pgpIRIciw4SrD.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Vinod Koul]

On 09-05-18, 20:47, Stephen Rothwell wrote:
> On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
> >
> > On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
> > > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:  
> > 
> > > > I think this is an excellent idea, copying in Stephen for his input.
> > > > I'm currently on holiday but unless someone convinces me it's a terrible
> > > > idea I'm willing to at least give it a go on a trial basis once I'm back
> > > > home.  
> > 
> > > Since Stephen merges all -fixes branches first, before merging all the
> > > -next branches, he already generates that as part of linux-next. All
> > > he'd need to do is push that intermediate state out to some
> > > linux-fixes branch for consumption by test bots.
> 
> Good idea ... I will see what I can do.
>   
> > True.  It's currently only those -fixes branches that people have asked
> > him to merge separately which isn't as big a proportion of trees as have
> > them (perhaps fortunately given people's enthusiasm for fixes branches
> > that don't merge cleanly with their development branches) so we'd also
> > need to encourage people to add them separately.
> 
> I currently have 44 such fixes branches.  More welcome!

Great so do you want us to send fixes branch or scan the existing trees and add
them.

In case of former please do add slave-dma/fixes as well

~Vinod

Re: [Ksummit-discuss] bug-introducing patches

[Vinod Koul]

On 09-05-18, 20:47, Stephen Rothwell wrote:
> On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
> >
> > On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
> > > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:  
> > 
> > > > I think this is an excellent idea, copying in Stephen for his input.
> > > > I'm currently on holiday but unless someone convinces me it's a terrible
> > > > idea I'm willing to at least give it a go on a trial basis once I'm back
> > > > home.  
> > 
> > > Since Stephen merges all -fixes branches first, before merging all the
> > > -next branches, he already generates that as part of linux-next. All
> > > he'd need to do is push that intermediate state out to some
> > > linux-fixes branch for consumption by test bots.
> 
> Good idea ... I will see what I can do.
>   
> > True.  It's currently only those -fixes branches that people have asked
> > him to merge separately which isn't as big a proportion of trees as have
> > them (perhaps fortunately given people's enthusiasm for fixes branches
> > that don't merge cleanly with their development branches) so we'd also
> > need to encourage people to add them separately.
> 
> I currently have 44 such fixes branches.  More welcome!

Great so do you want us to send fixes branch or scan the existing trees and add
them.

In case of former please do add slave-dma/fixes as well

~Vinod

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
>
> On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
> > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:  
> 
> > > I think this is an excellent idea, copying in Stephen for his input.
> > > I'm currently on holiday but unless someone convinces me it's a terrible
> > > idea I'm willing to at least give it a go on a trial basis once I'm back
> > > home.  
> 
> > Since Stephen merges all -fixes branches first, before merging all the
> > -next branches, he already generates that as part of linux-next. All
> > he'd need to do is push that intermediate state out to some
> > linux-fixes branch for consumption by test bots.

Good idea ... I will see what I can do.
  
> True.  It's currently only those -fixes branches that people have asked
> him to merge separately which isn't as big a proportion of trees as have
> them (perhaps fortunately given people's enthusiasm for fixes branches
> that don't merge cleanly with their development branches) so we'd also
> need to encourage people to add them separately.

I currently have 44 such fixes branches.  More welcome!

-- 
Cheers,
Stephen Rothwell


pgplKNJQXLB0o.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Stephen Rothwell]

On Wed, 9 May 2018 18:03:46 +0900 Mark Brown  wrote:
>
> On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
> > On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:  
> 
> > > I think this is an excellent idea, copying in Stephen for his input.
> > > I'm currently on holiday but unless someone convinces me it's a terrible
> > > idea I'm willing to at least give it a go on a trial basis once I'm back
> > > home.  
> 
> > Since Stephen merges all -fixes branches first, before merging all the
> > -next branches, he already generates that as part of linux-next. All
> > he'd need to do is push that intermediate state out to some
> > linux-fixes branch for consumption by test bots.

Good idea ... I will see what I can do.
  
> True.  It's currently only those -fixes branches that people have asked
> him to merge separately which isn't as big a proportion of trees as have
> them (perhaps fortunately given people's enthusiasm for fixes branches
> that don't merge cleanly with their development branches) so we'd also
> need to encourage people to add them separately.

I currently have 44 such fixes branches.  More welcome!

-- 
Cheers,
Stephen Rothwell


pgplKNJQXLB0o.pgp
Description: OpenPGP digital signature

Re: [Ksummit-discuss] bug-introducing patches

[Mark Brown]

On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
> On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:

> > I think this is an excellent idea, copying in Stephen for his input.
> > I'm currently on holiday but unless someone convinces me it's a terrible
> > idea I'm willing to at least give it a go on a trial basis once I'm back
> > home.

> Since Stephen merges all -fixes branches first, before merging all the
> -next branches, he already generates that as part of linux-next. All
> he'd need to do is push that intermediate state out to some
> linux-fixes branch for consumption by test bots.

True.  It's currently only those -fixes branches that people have asked
him to merge separately which isn't as big a proportion of trees as have
them (perhaps fortunately given people's enthusiasm for fixes branches
that don't merge cleanly with their development branches) so we'd also
need to encourage people to add them separately.


signature.asc
Description: PGP signature

Re: [Ksummit-discuss] bug-introducing patches

[Mark Brown]

On Wed, May 09, 2018 at 10:47:57AM +0200, Daniel Vetter wrote:
> On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:

> > I think this is an excellent idea, copying in Stephen for his input.
> > I'm currently on holiday but unless someone convinces me it's a terrible
> > idea I'm willing to at least give it a go on a trial basis once I'm back
> > home.

> Since Stephen merges all -fixes branches first, before merging all the
> -next branches, he already generates that as part of linux-next. All
> he'd need to do is push that intermediate state out to some
> linux-fixes branch for consumption by test bots.

True.  It's currently only those -fixes branches that people have asked
him to merge separately which isn't as big a proportion of trees as have
them (perhaps fortunately given people's enthusiasm for fixes branches
that don't merge cleanly with their development branches) so we'd also
need to encourage people to add them separately.


signature.asc
Description: PGP signature

Re: [Ksummit-discuss] bug-introducing patches

[Geert Uytterhoeven]

On Wed, May 9, 2018 at 10:47 AM, Daniel Vetter  wrote:
> On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:
>> On Fri, May 04, 2018 at 04:21:26PM +0200, Ulf Hansson wrote:
>>> Then, why don't we have a pre-integration tree for fixes? That would
>>> at least simply automated testing of fixes separately from new
>>> material.
>>
>>> Perhaps this has already been discussed, and concluded and it's not
>>> worth it, then apologize for my ignorance.
>>
>> I think this is an excellent idea, copying in Stephen for his input.
>> I'm currently on holiday but unless someone convinces me it's a terrible
>> idea I'm willing to at least give it a go on a trial basis once I'm back
>> home.
>
> Since Stephen merges all -fixes branches first, before merging all the
> -next branches, he already generates that as part of linux-next. All
> he'd need to do is push that intermediate state out to some
> linux-fixes branch for consumption by test bots.

+1

Gr{oetje,eeting}s,

Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: [Ksummit-discuss] bug-introducing patches

[Geert Uytterhoeven]

On Wed, May 9, 2018 at 10:47 AM, Daniel Vetter  wrote:
> On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:
>> On Fri, May 04, 2018 at 04:21:26PM +0200, Ulf Hansson wrote:
>>> Then, why don't we have a pre-integration tree for fixes? That would
>>> at least simply automated testing of fixes separately from new
>>> material.
>>
>>> Perhaps this has already been discussed, and concluded and it's not
>>> worth it, then apologize for my ignorance.
>>
>> I think this is an excellent idea, copying in Stephen for his input.
>> I'm currently on holiday but unless someone convinces me it's a terrible
>> idea I'm willing to at least give it a go on a trial basis once I'm back
>> home.
>
> Since Stephen merges all -fixes branches first, before merging all the
> -next branches, he already generates that as part of linux-next. All
> he'd need to do is push that intermediate state out to some
> linux-fixes branch for consumption by test bots.

+1

Gr{oetje,eeting}s,

Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: [Ksummit-discuss] bug-introducing patches

[Daniel Vetter]

On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:
> On Fri, May 04, 2018 at 04:21:26PM +0200, Ulf Hansson wrote:
>
>> Then, why don't we have a pre-integration tree for fixes? That would
>> at least simply automated testing of fixes separately from new
>> material.
>
>> Perhaps this has already been discussed, and concluded and it's not
>> worth it, then apologize for my ignorance.
>
> I think this is an excellent idea, copying in Stephen for his input.
> I'm currently on holiday but unless someone convinces me it's a terrible
> idea I'm willing to at least give it a go on a trial basis once I'm back
> home.

Since Stephen merges all -fixes branches first, before merging all the
-next branches, he already generates that as part of linux-next. All
he'd need to do is push that intermediate state out to some
linux-fixes branch for consumption by test bots.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: [Ksummit-discuss] bug-introducing patches

[Daniel Vetter]

On Wed, May 9, 2018 at 10:44 AM, Mark Brown  wrote:
> On Fri, May 04, 2018 at 04:21:26PM +0200, Ulf Hansson wrote:
>
>> Then, why don't we have a pre-integration tree for fixes? That would
>> at least simply automated testing of fixes separately from new
>> material.
>
>> Perhaps this has already been discussed, and concluded and it's not
>> worth it, then apologize for my ignorance.
>
> I think this is an excellent idea, copying in Stephen for his input.
> I'm currently on holiday but unless someone convinces me it's a terrible
> idea I'm willing to at least give it a go on a trial basis once I'm back
> home.

Since Stephen merges all -fixes branches first, before merging all the
-next branches, he already generates that as part of linux-next. All
he'd need to do is push that intermediate state out to some
linux-fixes branch for consumption by test bots.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: [Ksummit-discuss] bug-introducing patches

[Mark Brown]

On Fri, May 04, 2018 at 04:21:26PM +0200, Ulf Hansson wrote:

> Then, why don't we have a pre-integration tree for fixes? That would
> at least simply automated testing of fixes separately from new
> material.

> Perhaps this has already been discussed, and concluded and it's not
> worth it, then apologize for my ignorance.

I think this is an excellent idea, copying in Stephen for his input.
I'm currently on holiday but unless someone convinces me it's a terrible
idea I'm willing to at least give it a go on a trial basis once I'm back
home.


signature.asc
Description: PGP signature

Re: [Ksummit-discuss] bug-introducing patches

[Mark Brown]

On Fri, May 04, 2018 at 04:21:26PM +0200, Ulf Hansson wrote:

> Then, why don't we have a pre-integration tree for fixes? That would
> at least simply automated testing of fixes separately from new
> material.

> Perhaps this has already been discussed, and concluded and it's not
> worth it, then apologize for my ignorance.

I think this is an excellent idea, copying in Stephen for his input.
I'm currently on holiday but unless someone convinces me it's a terrible
idea I'm willing to at least give it a go on a trial basis once I'm back
home.


signature.asc
Description: PGP signature

Re: [Ksummit-discuss] bug-introducing patches

[Mark Brown]

On Tue, May 08, 2018 at 07:49:59AM -0700, Tony Lindgren wrote:
> * Theodore Y. Ts'o  [180508 03:50]:

> > The people who run Linus's tree and test -rc kernels tend to be kernel
> > developers and individual users who want to run bleeding edge kernels
> > and who generally are technically clueful.  If you were talking about
> > SLR cameras, you'd call them the "prosumers" segment of the market.

> Yup that's the category. People tinkering with their devices and
> using bleeding edge kernels because of some new device driver only
> being in thr -rc series for example.

You also get some people who are intending to ship on stable kernels but
are tracking upstream during product development so that they can be on
the most current stable release when they go to production.


signature.asc
Description: PGP signature

Re: [Ksummit-discuss] bug-introducing patches

[Mark Brown]

On Tue, May 08, 2018 at 07:49:59AM -0700, Tony Lindgren wrote:
> * Theodore Y. Ts'o  [180508 03:50]:

> > The people who run Linus's tree and test -rc kernels tend to be kernel
> > developers and individual users who want to run bleeding edge kernels
> > and who generally are technically clueful.  If you were talking about
> > SLR cameras, you'd call them the "prosumers" segment of the market.

> Yup that's the category. People tinkering with their devices and
> using bleeding edge kernels because of some new device driver only
> being in thr -rc series for example.

You also get some people who are intending to ship on stable kernels but
are tracking upstream during product development so that they can be on
the most current stable release when they go to production.


signature.asc
Description: PGP signature

Re: [Ksummit-discuss] bug-introducing patches

[Willy Tarreau]

On Tue, May 08, 2018 at 08:29:14PM +, Sasha Levin wrote:
> What if, instead, Linus doesn't actually ever release a point release?
> We can make the merge window open more often, and since there's no
> actual release, people won't rush to push fixes in later -rc cycles.

And then what's the purpose of these later -rc cycles if you remove one
release ? You're just removing one step and shifting everything down by
one -rc but the issues are the same.

> We take away the incentive to push poorly tested code. Maintainers still
> free to commit anything they'd like, but there's no reason to commit
> code they're not confident of just to make it to a random release no one
> will use.
> 
> Merge window will happen more often, so there's no real reason to rush
> things in a particular window, and since -stable releases every week
> there's no rush to push a fix in since the next release is just a week
> away.

I'm not sure what model you're having in mind but the description above
reminds me of 2.5 which was constantly had something broken and which
used to be unusable for many developers. Many of us even bought some
SCSI cards and disks by then because for a long time IDE was broken.

The primary purpose of Linus' releases and -rc is to synchronise everyone
on the same goal at the same time. The merge window is "send me your crap,
it must be OK but we know problems happen and you'll be allowed to fix it
later". The -rc ones are there so that everyone fixes their crap in
parallel so that we converge towards something acceptable for everyone.

Your argument that the .0 release is useless is wrong in my opinion. It
is as wrong as saying "statistics show that less people use .3 than .7".
And comparing "stable kernels" to ".0" is wrong because there are roughly
10 times more stable kernels than releases so statistically you'll find
10 times more of them in field. The reality is that deploying .0 always
takes a bit more time for end users so statistically it should be a bit
less common in field :
  - you're never certain when the new version is going to be released
(will rc8/rc9 exist?)
  - when it's released, you have to update your config and it takes
some time.
  - by the time you find a quiet moment to do all this, it's not
unlikely that the end of the week is reached with .1 appearing.

And so what ? The .0 release is a stable release like any other one.
It doesn't deserve to be deployed more than any other specific stable
release. It serves as a reference. Before .0 the code experiences some
possibly breaking changes, even some reverts. After .0 it experiences
only small fixes according to the stable rules.

Overall I think the current model is not that bad, and that what is the
most needed is some education regarding how -stable works to encourage
developers to rush their fixes less (after more tests), and to ensure
that those who generally push good quality fixes can submit them at any
moment in the cycle so that we get them as fast as possible in -stable.

Willy

Re: [Ksummit-discuss] bug-introducing patches

[Willy Tarreau]

On Tue, May 08, 2018 at 08:29:14PM +, Sasha Levin wrote:
> What if, instead, Linus doesn't actually ever release a point release?
> We can make the merge window open more often, and since there's no
> actual release, people won't rush to push fixes in later -rc cycles.

And then what's the purpose of these later -rc cycles if you remove one
release ? You're just removing one step and shifting everything down by
one -rc but the issues are the same.

> We take away the incentive to push poorly tested code. Maintainers still
> free to commit anything they'd like, but there's no reason to commit
> code they're not confident of just to make it to a random release no one
> will use.
> 
> Merge window will happen more often, so there's no real reason to rush
> things in a particular window, and since -stable releases every week
> there's no rush to push a fix in since the next release is just a week
> away.

I'm not sure what model you're having in mind but the description above
reminds me of 2.5 which was constantly had something broken and which
used to be unusable for many developers. Many of us even bought some
SCSI cards and disks by then because for a long time IDE was broken.

The primary purpose of Linus' releases and -rc is to synchronise everyone
on the same goal at the same time. The merge window is "send me your crap,
it must be OK but we know problems happen and you'll be allowed to fix it
later". The -rc ones are there so that everyone fixes their crap in
parallel so that we converge towards something acceptable for everyone.

Your argument that the .0 release is useless is wrong in my opinion. It
is as wrong as saying "statistics show that less people use .3 than .7".
And comparing "stable kernels" to ".0" is wrong because there are roughly
10 times more stable kernels than releases so statistically you'll find
10 times more of them in field. The reality is that deploying .0 always
takes a bit more time for end users so statistically it should be a bit
less common in field :
  - you're never certain when the new version is going to be released
(will rc8/rc9 exist?)
  - when it's released, you have to update your config and it takes
some time.
  - by the time you find a quiet moment to do all this, it's not
unlikely that the end of the week is reached with .1 appearing.

And so what ? The .0 release is a stable release like any other one.
It doesn't deserve to be deployed more than any other specific stable
release. It serves as a reference. Before .0 the code experiences some
possibly breaking changes, even some reverts. After .0 it experiences
only small fixes according to the stable rules.

Overall I think the current model is not that bad, and that what is the
most needed is some education regarding how -stable works to encourage
developers to rush their fixes less (after more tests), and to ensure
that those who generally push good quality fixes can submit them at any
moment in the cycle so that we get them as fast as possible in -stable.

Willy