Re: [PATCH 4.12 00/84] 4.12.3-stable review

[Jan Lübbe]

On Sat, 2017-07-22 at 08:53 -0700, Kevin Hilman wrote:
> > Boot Failures Detected:
> > 
> > arm:
> > 
> >  multi_v7_defconfig
> >  imx6ul-pico-hobbit_rootfs:nfs: 1 failed lab
> > 
> >  tegra_defconfig
> >  tegra124-jetson-tk1_rootfs:nfs: 1 failed lab
> 
> @Jan looks like a couple boards in your lab are having NFS issues
> with stable kernels.  Could you have a closer look?

NFS was not configured correctly in our lab. We've fixed it and they
now boot correctly via NFS.

Regards,
Jan
-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

Re: [PATCH 4.12 00/84] 4.12.3-stable review

[Jan Lübbe]

On Sat, 2017-07-22 at 08:53 -0700, Kevin Hilman wrote:
> > Boot Failures Detected:
> > 
> > arm:
> > 
> >  multi_v7_defconfig
> >  imx6ul-pico-hobbit_rootfs:nfs: 1 failed lab
> > 
> >  tegra_defconfig
> >  tegra124-jetson-tk1_rootfs:nfs: 1 failed lab
> 
> @Jan looks like a couple boards in your lab are having NFS issues
> with stable kernels.  Could you have a closer look?

NFS was not configured correctly in our lab. We've fixed it and they
now boot correctly via NFS.

Regards,
Jan
-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

Re: [PATCH 4.12 00/84] 4.12.3-stable review

[Sjoerd Simons]

On Sat, 2017-07-22 at 08:47 -0700, Kevin Hilman wrote:
> + Sjoerd @ Collabora
> 
> Shuah Khan  writes:
> 
> > On 07/19/2017 10:29 AM, kernelci.org bot wrote:
> > > stable-rc/linux-4.12.y boot: 166 boots: 5 failed, 161 passed
> > > (v4.12.2-85-g908a8d27d1c5)
> > > 
> > > Full Boot Summary: https://kernelci.org/boot/all/job/stable-rc/br
> > > anch/linux-4.12.y/kernel/v4.12.2-85-g908a8d27d1c5/
> > > Full Build Summary: https://kernelci.org/build/stable-rc/branch/l
> > > inux-4.12.y/kernel/v4.12.2-85-g908a8d27d1c5/
> > > 
> > > Tree: stable-rc
> > > Branch: linux-4.12.y
> > > Git Describe: v4.12.2-85-g908a8d27d1c5
> > > Git Commit: 908a8d27d1c52aaafab80d7267e8e61f9a174ecc
> > > Git URL: http://git.kernel.org/pub/scm/linux/kernel/git/stable/li
> > > nux-stable-rc.git
> > > Tested: 44 unique boards, 14 SoC families, 24 builds out of 204
> > > 
> > > Boot Regressions Detected:
> > > 
> > > arm:
> > > 
> > > multi_v7_defconfig+CONFIG_PROVE_LOCKING=y:
> > > exynos5422-odroidxu3:
> > > lab-collabora: new failure (last pass: v4.12.2)
> > 
> > I am seeing boot failure on odroid-xu4 with 4.13-rc1 with
> > exynos_defconfig.
> > 4.12 is fine.
> > 
> > I am debugging this and based on this report it sounds like it
> > might be
> > easier for me to start with 4.12 and try to isolate the change.
> > Will keep
> > you posted.
> 
> I suspect that these CONFIG_PROVE_LOCKING=y issues are actually not
> kernel issues directly, but kernel size limitations based on the load
> addresses used in lab-collabora LAVA.
> 
> Comparing against my lab, I'm using load address: 0x4200 where as
> lab-collabora is using 0x4100.  I'm wondering if that is not
> enough
> room to ucompress down to the default boot address without
> overwriting itself?

Adjusting the load address for XU3 resolved the boot failure (changing
from memory start + 16M to memory start + 32M which is the default for
that board in u-boot/lava as well these days)

> I suspect the same problem in the lab-collabora panda boots since the
> load address differes from mine in the same way.

Correct, tuning the load addresses there results in a happy boot.

The kernel does relocate itself on boot if it detects decompression
will overwrite itself. On the panda though that seemingly meant the dtb
got overwritten, just bumping that out of the way made it boot as well.
I didn't check if the same happened on XU3 (which has/had far more
space between the kernel and the dtb).. That said, it's far safer (and
faster) to avoid running into that corner case :) 

> @Sjoerd: can someone in lab-collabora maybe have a closer look?
> 
> Kevin

-- 
Sjoerd Simons
Collabora Ltd.

Re: [PATCH 4.12 00/84] 4.12.3-stable review

[Sjoerd Simons]

On Sat, 2017-07-22 at 08:47 -0700, Kevin Hilman wrote:
> + Sjoerd @ Collabora
> 
> Shuah Khan  writes:
> 
> > On 07/19/2017 10:29 AM, kernelci.org bot wrote:
> > > stable-rc/linux-4.12.y boot: 166 boots: 5 failed, 161 passed
> > > (v4.12.2-85-g908a8d27d1c5)
> > > 
> > > Full Boot Summary: https://kernelci.org/boot/all/job/stable-rc/br
> > > anch/linux-4.12.y/kernel/v4.12.2-85-g908a8d27d1c5/
> > > Full Build Summary: https://kernelci.org/build/stable-rc/branch/l
> > > inux-4.12.y/kernel/v4.12.2-85-g908a8d27d1c5/
> > > 
> > > Tree: stable-rc
> > > Branch: linux-4.12.y
> > > Git Describe: v4.12.2-85-g908a8d27d1c5
> > > Git Commit: 908a8d27d1c52aaafab80d7267e8e61f9a174ecc
> > > Git URL: http://git.kernel.org/pub/scm/linux/kernel/git/stable/li
> > > nux-stable-rc.git
> > > Tested: 44 unique boards, 14 SoC families, 24 builds out of 204
> > > 
> > > Boot Regressions Detected:
> > > 
> > > arm:
> > > 
> > > multi_v7_defconfig+CONFIG_PROVE_LOCKING=y:
> > > exynos5422-odroidxu3:
> > > lab-collabora: new failure (last pass: v4.12.2)
> > 
> > I am seeing boot failure on odroid-xu4 with 4.13-rc1 with
> > exynos_defconfig.
> > 4.12 is fine.
> > 
> > I am debugging this and based on this report it sounds like it
> > might be
> > easier for me to start with 4.12 and try to isolate the change.
> > Will keep
> > you posted.
> 
> I suspect that these CONFIG_PROVE_LOCKING=y issues are actually not
> kernel issues directly, but kernel size limitations based on the load
> addresses used in lab-collabora LAVA.
> 
> Comparing against my lab, I'm using load address: 0x4200 where as
> lab-collabora is using 0x4100.  I'm wondering if that is not
> enough
> room to ucompress down to the default boot address without
> overwriting itself?

Adjusting the load address for XU3 resolved the boot failure (changing
from memory start + 16M to memory start + 32M which is the default for
that board in u-boot/lava as well these days)

> I suspect the same problem in the lab-collabora panda boots since the
> load address differes from mine in the same way.

Correct, tuning the load addresses there results in a happy boot.

The kernel does relocate itself on boot if it detects decompression
will overwrite itself. On the panda though that seemingly meant the dtb
got overwritten, just bumping that out of the way made it boot as well.
I didn't check if the same happened on XU3 (which has/had far more
space between the kernel and the dtb).. That said, it's far safer (and
faster) to avoid running into that corner case :) 

> @Sjoerd: can someone in lab-collabora maybe have a closer look?
> 
> Kevin

-- 
Sjoerd Simons
Collabora Ltd.

Re: [PATCH 4.12 00/84] 4.12.3-stable review

[Kevin Hilman]

+ Sjoerd @ Collabora

Shuah Khan  writes:

> On 07/19/2017 10:29 AM, kernelci.org bot wrote:
>> stable-rc/linux-4.12.y boot: 166 boots: 5 failed, 161 passed 
>> (v4.12.2-85-g908a8d27d1c5)
>> 
>> Full Boot Summary: 
>> https://kernelci.org/boot/all/job/stable-rc/branch/linux-4.12.y/kernel/v4.12.2-85-g908a8d27d1c5/
>> Full Build Summary: 
>> https://kernelci.org/build/stable-rc/branch/linux-4.12.y/kernel/v4.12.2-85-g908a8d27d1c5/
>> 
>> Tree: stable-rc
>> Branch: linux-4.12.y
>> Git Describe: v4.12.2-85-g908a8d27d1c5
>> Git Commit: 908a8d27d1c52aaafab80d7267e8e61f9a174ecc
>> Git URL: 
>> http://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
>> Tested: 44 unique boards, 14 SoC families, 24 builds out of 204
>> 
>> Boot Regressions Detected:
>> 
>> arm:
>> 
>> multi_v7_defconfig+CONFIG_PROVE_LOCKING=y:
>> exynos5422-odroidxu3:
>> lab-collabora: new failure (last pass: v4.12.2)
>
> I am seeing boot failure on odroid-xu4 with 4.13-rc1 with exynos_defconfig.
> 4.12 is fine.
>
> I am debugging this and based on this report it sounds like it might be
> easier for me to start with 4.12 and try to isolate the change. Will keep
> you posted.

I suspect that these CONFIG_PROVE_LOCKING=y issues are actually not
kernel issues directly, but kernel size limitations based on the load
addresses used in lab-collabora LAVA.

Comparing against my lab, I'm using load address: 0x4200 where as
lab-collabora is using 0x4100.  I'm wondering if that is not enough
room to ucompress down to the default boot address without overwriting itself?

I suspect the same problem in the lab-collabora panda boots since the
load address differes from mine in the same way.

@Sjoerd: can someone in lab-collabora maybe have a closer look?

Kevin

Re: [PATCH 4.12 00/84] 4.12.3-stable review

[Kevin Hilman]

+ Sjoerd @ Collabora

Shuah Khan  writes:

> On 07/19/2017 10:29 AM, kernelci.org bot wrote:
>> stable-rc/linux-4.12.y boot: 166 boots: 5 failed, 161 passed 
>> (v4.12.2-85-g908a8d27d1c5)
>> 
>> Full Boot Summary: 
>> https://kernelci.org/boot/all/job/stable-rc/branch/linux-4.12.y/kernel/v4.12.2-85-g908a8d27d1c5/
>> Full Build Summary: 
>> https://kernelci.org/build/stable-rc/branch/linux-4.12.y/kernel/v4.12.2-85-g908a8d27d1c5/
>> 
>> Tree: stable-rc
>> Branch: linux-4.12.y
>> Git Describe: v4.12.2-85-g908a8d27d1c5
>> Git Commit: 908a8d27d1c52aaafab80d7267e8e61f9a174ecc
>> Git URL: 
>> http://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
>> Tested: 44 unique boards, 14 SoC families, 24 builds out of 204
>> 
>> Boot Regressions Detected:
>> 
>> arm:
>> 
>> multi_v7_defconfig+CONFIG_PROVE_LOCKING=y:
>> exynos5422-odroidxu3:
>> lab-collabora: new failure (last pass: v4.12.2)
>
> I am seeing boot failure on odroid-xu4 with 4.13-rc1 with exynos_defconfig.
> 4.12 is fine.
>
> I am debugging this and based on this report it sounds like it might be
> easier for me to start with 4.12 and try to isolate the change. Will keep
> you posted.

I suspect that these CONFIG_PROVE_LOCKING=y issues are actually not
kernel issues directly, but kernel size limitations based on the load
addresses used in lab-collabora LAVA.

Comparing against my lab, I'm using load address: 0x4200 where as
lab-collabora is using 0x4100.  I'm wondering if that is not enough
room to ucompress down to the default boot address without overwriting itself?

I suspect the same problem in the lab-collabora panda boots since the
load address differes from mine in the same way.

@Sjoerd: can someone in lab-collabora maybe have a closer look?

Kevin

Re: [PATCH 4.12 00/84] 4.12.3-stable review

[Greg Kroah-Hartman]

On Wed, Jul 19, 2017 at 01:35:59PM -0700, Guenter Roeck wrote:
> On Wed, Jul 19, 2017 at 11:43:06AM +0200, Greg Kroah-Hartman wrote:
> > This is the start of the stable review cycle for the 4.12.3 release.
> > There are 84 patches in this series, all will be posted as a response
> > to this one.  If anyone has any issues with these being applied, please
> > let me know.
> > 
> > Responses should be made by Fri Jul 21 09:22:37 UTC 2017.
> > Anything received after that time might be too late.
> > 
> Build results:
>   total: 145 pass: 145 fail: 0
> Qemu test results:
>   total: 122 pass: 122 fail: 0
> 
> Details are available at http://kerneltests.org/builders.

Great, thanks for letting me know about all of these.

greg k-h

Re: [PATCH 4.12 00/84] 4.12.3-stable review

[Greg Kroah-Hartman]

On Wed, Jul 19, 2017 at 01:35:59PM -0700, Guenter Roeck wrote:
> On Wed, Jul 19, 2017 at 11:43:06AM +0200, Greg Kroah-Hartman wrote:
> > This is the start of the stable review cycle for the 4.12.3 release.
> > There are 84 patches in this series, all will be posted as a response
> > to this one.  If anyone has any issues with these being applied, please
> > let me know.
> > 
> > Responses should be made by Fri Jul 21 09:22:37 UTC 2017.
> > Anything received after that time might be too late.
> > 
> Build results:
>   total: 145 pass: 145 fail: 0
> Qemu test results:
>   total: 122 pass: 122 fail: 0
> 
> Details are available at http://kerneltests.org/builders.

Great, thanks for letting me know about all of these.

greg k-h

Re: [PATCH 4.12 00/84] 4.12.3-stable review

[Shuah Khan]

On 07/19/2017 03:43 AM, Greg Kroah-Hartman wrote:
> This is the start of the stable review cycle for the 4.12.3 release.
> There are 84 patches in this series, all will be posted as a response
> to this one.  If anyone has any issues with these being applied, please
> let me know.
> 
> Responses should be made by Fri Jul 21 09:22:37 UTC 2017.
> Anything received after that time might be too late.
> 
> The whole patch series can be found in one patch at:
>   kernel.org/pub/linux/kernel/v4.x/stable-review/patch-4.12.3-rc1.gz
> or in the git tree and branch at:
>   git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git 
> linux-4.12.y
> and the diffstat can be found below.
> 
> thanks,
> 
> greg k-h
> 

Compiled and booted on my test system. No dmesg regressions.

thanks,
-- Shuah

Re: [PATCH 4.12 00/84] 4.12.3-stable review

[Shuah Khan]

On 07/19/2017 03:43 AM, Greg Kroah-Hartman wrote:
> This is the start of the stable review cycle for the 4.12.3 release.
> There are 84 patches in this series, all will be posted as a response
> to this one.  If anyone has any issues with these being applied, please
> let me know.
> 
> Responses should be made by Fri Jul 21 09:22:37 UTC 2017.
> Anything received after that time might be too late.
> 
> The whole patch series can be found in one patch at:
>   kernel.org/pub/linux/kernel/v4.x/stable-review/patch-4.12.3-rc1.gz
> or in the git tree and branch at:
>   git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git 
> linux-4.12.y
> and the diffstat can be found below.
> 
> thanks,
> 
> greg k-h
> 

Compiled and booted on my test system. No dmesg regressions.

thanks,
-- Shuah

Re: [PATCH 4.12 00/84] 4.12.3-stable review

[Guenter Roeck]

On Wed, Jul 19, 2017 at 11:43:06AM +0200, Greg Kroah-Hartman wrote:
> This is the start of the stable review cycle for the 4.12.3 release.
> There are 84 patches in this series, all will be posted as a response
> to this one.  If anyone has any issues with these being applied, please
> let me know.
> 
> Responses should be made by Fri Jul 21 09:22:37 UTC 2017.
> Anything received after that time might be too late.
> 
Build results:
total: 145 pass: 145 fail: 0
Qemu test results:
total: 122 pass: 122 fail: 0

Details are available at http://kerneltests.org/builders.

Guenter

Re: [PATCH 4.12 00/84] 4.12.3-stable review

[Guenter Roeck]

On Wed, Jul 19, 2017 at 11:43:06AM +0200, Greg Kroah-Hartman wrote:
> This is the start of the stable review cycle for the 4.12.3 release.
> There are 84 patches in this series, all will be posted as a response
> to this one.  If anyone has any issues with these being applied, please
> let me know.
> 
> Responses should be made by Fri Jul 21 09:22:37 UTC 2017.
> Anything received after that time might be too late.
> 
Build results:
total: 145 pass: 145 fail: 0
Qemu test results:
total: 122 pass: 122 fail: 0

Details are available at http://kerneltests.org/builders.

Guenter

Re: [PATCH 4.12 00/84] 4.12.3-stable review

[Shuah Khan]

On 07/19/2017 10:29 AM, kernelci.org bot wrote:
> stable-rc/linux-4.12.y boot: 166 boots: 5 failed, 161 passed 
> (v4.12.2-85-g908a8d27d1c5)
> 
> Full Boot Summary: 
> https://kernelci.org/boot/all/job/stable-rc/branch/linux-4.12.y/kernel/v4.12.2-85-g908a8d27d1c5/
> Full Build Summary: 
> https://kernelci.org/build/stable-rc/branch/linux-4.12.y/kernel/v4.12.2-85-g908a8d27d1c5/
> 
> Tree: stable-rc
> Branch: linux-4.12.y
> Git Describe: v4.12.2-85-g908a8d27d1c5
> Git Commit: 908a8d27d1c52aaafab80d7267e8e61f9a174ecc
> Git URL: 
> http://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
> Tested: 44 unique boards, 14 SoC families, 24 builds out of 204
> 
> Boot Regressions Detected:
> 
> arm:
> 
> multi_v7_defconfig+CONFIG_PROVE_LOCKING=y:
> exynos5422-odroidxu3:
> lab-collabora: new failure (last pass: v4.12.2)

I am seeing boot failure on odroid-xu4 with 4.13-rc1 with exynos_defconfig.
4.12 is fine.

I am debugging this and based on this report it sounds like it might be
easier for me to start with 4.12 and try to isolate the change. Will keep
you posted.

-- Shuah

> 
> Boot Failures Detected:
> 
> arm:
> 
> multi_v7_defconfig
> imx6ul-pico-hobbit_rootfs:nfs: 1 failed lab
> 
> tegra_defconfig
> tegra124-jetson-tk1_rootfs:nfs: 1 failed lab
> 
> mvebu_v5_defconfig
> kirkwood-openblocks_a7_rootfs:nfs: 1 failed lab
> 
> multi_v7_defconfig+CONFIG_PROVE_LOCKING=y
> exynos5422-odroidxu3: 1 failed lab
> omap4-panda: 1 failed lab
> 
> ---
> For more info write to 
> 

Re: [PATCH 4.12 00/84] 4.12.3-stable review

[Shuah Khan]

On 07/19/2017 10:29 AM, kernelci.org bot wrote:
> stable-rc/linux-4.12.y boot: 166 boots: 5 failed, 161 passed 
> (v4.12.2-85-g908a8d27d1c5)
> 
> Full Boot Summary: 
> https://kernelci.org/boot/all/job/stable-rc/branch/linux-4.12.y/kernel/v4.12.2-85-g908a8d27d1c5/
> Full Build Summary: 
> https://kernelci.org/build/stable-rc/branch/linux-4.12.y/kernel/v4.12.2-85-g908a8d27d1c5/
> 
> Tree: stable-rc
> Branch: linux-4.12.y
> Git Describe: v4.12.2-85-g908a8d27d1c5
> Git Commit: 908a8d27d1c52aaafab80d7267e8e61f9a174ecc
> Git URL: 
> http://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
> Tested: 44 unique boards, 14 SoC families, 24 builds out of 204
> 
> Boot Regressions Detected:
> 
> arm:
> 
> multi_v7_defconfig+CONFIG_PROVE_LOCKING=y:
> exynos5422-odroidxu3:
> lab-collabora: new failure (last pass: v4.12.2)

I am seeing boot failure on odroid-xu4 with 4.13-rc1 with exynos_defconfig.
4.12 is fine.

I am debugging this and based on this report it sounds like it might be
easier for me to start with 4.12 and try to isolate the change. Will keep
you posted.

-- Shuah

> 
> Boot Failures Detected:
> 
> arm:
> 
> multi_v7_defconfig
> imx6ul-pico-hobbit_rootfs:nfs: 1 failed lab
> 
> tegra_defconfig
> tegra124-jetson-tk1_rootfs:nfs: 1 failed lab
> 
> mvebu_v5_defconfig
> kirkwood-openblocks_a7_rootfs:nfs: 1 failed lab
> 
> multi_v7_defconfig+CONFIG_PROVE_LOCKING=y
> exynos5422-odroidxu3: 1 failed lab
> omap4-panda: 1 failed lab
> 
> ---
> For more info write to 
>