Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Menno Finlay-Smits]

Not really sorry. I'm not using this hardware any more.

On Thu, 1 Aug 2019, at 02:37, Martin Michlmayr wrote:
> * Martin Michlmayr  [2018-09-10 14:27]:
> > > > > Do you want me to help figure out which change to the kernel fixed the
> > > > > problem?
> > > > 
> > > > As far as I can tell and if I'm not mistaken, the fix is already 
> > > > identified
> > > > and is included in 4.9.86.
> > > > 
> > > > I've started working on it for Stretch, and at one point it will be 
> > > > uploaded
> > > > to -proposed-updates for inclusion in the next point release (9.5). 
> > > > When it's
> > > > done I'll probably ask you to try a test kernel to make sure it's really
> > > > fixed.
> > > 
> > > I'm happy to try it out when it's available.
> > 
> > Any update on this?  Debian stable has 4.9.110-1 now.
> 
> Menno, any chance you can test if your problem has gone away?
> 
> -- 
> Martin Michlmayr
> https://www.cyrius.com/
>

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Martin Michlmayr]

* Martin Michlmayr  [2018-09-10 14:27]:
> > > > Do you want me to help figure out which change to the kernel fixed the
> > > > problem?
> > > 
> > > As far as I can tell and if I'm not mistaken, the fix is already 
> > > identified
> > > and is included in 4.9.86.
> > > 
> > > I've started working on it for Stretch, and at one point it will be 
> > > uploaded
> > > to -proposed-updates for inclusion in the next point release (9.5). When 
> > > it's
> > > done I'll probably ask you to try a test kernel to make sure it's really
> > > fixed.
> > 
> > I'm happy to try it out when it's available.
> 
> Any update on this?  Debian stable has 4.9.110-1 now.

Menno, any chance you can test if your problem has gone away?

-- 
Martin Michlmayr
https://www.cyrius.com/

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Martin Michlmayr]

* Menno Finlay-Smits  [2018-03-12 09:59]:
> > > Do you want me to help figure out which change to the kernel fixed the
> > > problem?
> > 
> > As far as I can tell and if I'm not mistaken, the fix is already identified
> > and is included in 4.9.86.
> > 
> > I've started working on it for Stretch, and at one point it will be uploaded
> > to -proposed-updates for inclusion in the next point release (9.5). When 
> > it's
> > done I'll probably ask you to try a test kernel to make sure it's really
> > fixed.
> 
> I'm happy to try it out when it's available.

Any update on this?  Debian stable has 4.9.110-1 now.

-- 
Martin Michlmayr
https://www.cyrius.com/

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Menno Finlay-Smits]

On Mon, 12 Mar 2018, at 00:06, Yves-Alexis Perez wrote:
> On Mon, 2018-03-12 at 00:02 +1300, Menno Finlay-Smits wrote:
> > Do you want me to help figure out which change to the kernel fixed the
> > problem?
> 
> As far as I can tell and if I'm not mistaken, the fix is already identified
> and is included in 4.9.86.
> 
> I've started working on it for Stretch, and at one point it will be uploaded
> to -proposed-updates for inclusion in the next point release (9.5). When it's
> done I'll probably ask you to try a test kernel to make sure it's really
> fixed.

I'm happy to try it out when it's available.

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Yves-Alexis Perez]

On Mon, 2018-03-12 at 00:02 +1300, Menno Finlay-Smits wrote:
> Bingo. The problem doesn't seem to happen using linux-image-4.14.0-3-
> marvell_4.14.17-1 from sid. I've now successfully transferred 1.5TB from an
> external USB drive onto the internal hard disk. Previously the problem would
> show up within the first few 100MB of the transfer.
> 
> Do you want me to help figure out which change to the kernel fixed the
> problem?

As far as I can tell and if I'm not mistaken, the fix is already identified
and is included in 4.9.86.

I've started working on it for Stretch, and at one point it will be uploaded
to -proposed-updates for inclusion in the next point release (9.5). When it's
done I'll probably ask you to try a test kernel to make sure it's really
fixed.

Regards,
-- 
Yves-Alexis

signature.asc
Description: This is a digitally signed message part

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Menno Finlay-Smits]

> > > Could you also try linux-image-4.14.0-3-marvell from sid?
> > 
> > Can do. Should I just use the kernel packages from sid or update the whole 
> > system to sid?
> 
> Hi Menno
> 
> The kernel packages should be sufficient.

Bingo. The problem doesn't seem to happen using 
linux-image-4.14.0-3-marvell_4.14.17-1 from sid. I've now successfully 
transferred 1.5TB from an external USB drive onto the internal hard disk. 
Previously the problem would show up within the first few 100MB of the transfer.

Do you want me to help figure out which change to the kernel fixed the problem?

- Menno

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Andrew Lunn]

> > Hi Menno
> > 
> > Could you also try linux-image-4.14.0-3-marvell from sid?
> 
> Can do. Should I just use the kernel packages from sid or update the whole 
> system to sid?

Hi Menno

The kernel packages should be sufficient.

Andrew

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Yves-Alexis Perez]

On Fri, Mar 09, 2018 at 10:53:07PM +1300, Menno Finlay-Smits wrote:
> Can do. Should I just use the kernel packages from sid or update the whole 
> system to sid?

I think you should be able to just pick the kernel from sid.

I'm starting to work on updating 4.9 to later kernels but it's not
really a priority right now since we'll just have a point release this
week-end.

Regards,
-- 
Yves-Alexis Perez


signature.asc
Description: PGP signature

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Menno Finlay-Smits]

On Thu, 8 Mar 2018, at 11:27, Andrew Lunn wrote:
> On Thu, Mar 08, 2018 at 09:49:29AM +1300, Menno Finlay-Smits wrote:
> > On Thu, 8 Mar 2018, at 02:36, Andrew Lunn wrote:
> > > > What else can I try?
> > > 
> > > Do you have transparent huge pages enabled?
> > > 
> > > ~$ cat /sys/kernel/mm/transparent_hugepage/enabled 
> > > [always] madvise never
> > > 
> > > If so, could you disable it with:
> > > 
> > > echo never > /sys/kernel/mm/transparent_hugepage/enabled
> > > 
> > > and run your rsync command.
> 
> Hi Menno
> 
> Could you also try linux-image-4.14.0-3-marvell from sid?

Can do. Should I just use the kernel packages from sid or update the whole 
system to sid?

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Andrew Lunn]

On Thu, Mar 08, 2018 at 09:49:29AM +1300, Menno Finlay-Smits wrote:
> On Thu, 8 Mar 2018, at 02:36, Andrew Lunn wrote:
> > > What else can I try?
> > 
> > Do you have transparent huge pages enabled?
> > 
> > ~$ cat /sys/kernel/mm/transparent_hugepage/enabled 
> > [always] madvise never
> > 
> > If so, could you disable it with:
> > 
> > echo never > /sys/kernel/mm/transparent_hugepage/enabled
> > 
> > and run your rsync command.

Hi Menno

Could you also try linux-image-4.14.0-3-marvell from sid?

There does not appear to be a 4.15 yet for marvell.

  Andrew

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Menno Finlay-Smits]

On Thu, 8 Mar 2018, at 02:36, Andrew Lunn wrote:
> > What else can I try?
> 
> Do you have transparent huge pages enabled?
> 
> ~$ cat /sys/kernel/mm/transparent_hugepage/enabled 
> [always] madvise never
> 
> If so, could you disable it with:
> 
> echo never > /sys/kernel/mm/transparent_hugepage/enabled
> 
> and run your rsync command.

I'll check this at the next opportunity. 

Also, I'm comfortable with patching and compiling kernels if necessary.

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Andrew Lunn]

> What else can I try?

Do you have transparent huge pages enabled?

~$ cat /sys/kernel/mm/transparent_hugepage/enabled 
[always] madvise never

If so, could you disable it with:

echo never > /sys/kernel/mm/transparent_hugepage/enabled

and run your rsync command.

Thanks
Andrew

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Andrew Lunn]

> What else can I try? There doesn't appear to be a newer kernel in
> proposed right now.

Are you happy to apply patches, build a kernel, and test it?

Thanks
Andrew

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Menno Finlay-Smits]

On Tue, 6 Mar 2018, at 13:54, Menno Finlay-Smits wrote:
> On Tue, 6 Mar 2018, at 04:57, Yves-Alexis Perez wrote:
> > On Mon, 2018-03-05 at 15:28 +0100, Andrew Lunn wrote:
> > > Would it be possible to try to reproduce this problem with 4.9.86 on
> > > the hardware reporting the issue?
> > 
> > 4.9.82-1+deb9u3 is currently in the archive. Menno, could you give it a 
> > shot?
> 
> Will do. I need to get the NAS back to running Scratch as I went back to 
> Jessie for comparison (similar looking problems there too) but I'll get 
> it done as soon as I can.
> 
> Also, I can confirm that I was indeed copying from an external USB disk 
> with just "rsync -av  ".

The problem still happens with 4.9.82-1+deb9u3.

Here's the dump:

[  675.800163] usercopy: kernel memory overwrite attempt detected to c08f83a0 
() (4294933600 bytes)
[  675.810513] [ cut here ]
[  675.815144] kernel BUG at /build/linux-nLLkbA/linux-4.9.82/mm/usercopy.c:75!
[  675.822215] Internal error: Oops - BUG: 0 [#1] ARM
[  675.827020] Modules linked in: ses enclosure scsi_transport_sas uas 
usb_storage marvell ehci_orion mv643xx_eth mvmdio of_mdio ehci_hcd fixed_phy 
marvell_cesa libphy des_generic xhci_pci xhci_hcd sg usbcore orion_wdt 
usb_common m25p80 spi_nor kirkwood_thermal evdev gpio_keys sunrpc ip_tables 
x_tables ipv6 autofs4 ext4 crc16 jbd2 crc32c_generic fscrypto ecb mbcache 
sd_mod sata_mv libata scsi_mod
[  675.862376] CPU: 0 PID: 2050 Comm: rsync Not tainted 4.9.0-6-marvell #1 
Debian 4.9.82-1+deb9u3
[  675.871022] Hardware name: Marvell Kirkwood (Flattened Device Tree)
[  675.877310] task: c0af32c0 task.stack: c0a7e000
[  675.881857] PC is at __check_object_size+0x120/0x1d8
[  675.886840] LR is at __check_object_size+0x120/0x1d8
[  675.891821] pc : []lr : []psr: 6013
   sp : c0a7fdb8  ip :   fp : c0a7ff08
[  675.903339] r10: c0a7e000  r9 : 7c60  r8 : c08f83a0
[  675.908579] r7 : c08f  r6 :   r5 : 7c60  r4 : c08f83a0
[  675.915128] r3 : c0555080  r2 : c055a3a4  r1 : c05509f0  r0 : 0065
[  675.921677] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[  675.928835] Control: 0005397f  Table: 00b44000  DAC: 0051
[  675.934599] Process rsync (pid: 2050, stack limit = 0xc0a7e190)
[  675.940538] Stack: (0xc0a7fdb8 to 0xc0a8)
[  675.944908] fda0:   
c04634c4 7c60
[  675.953116] fdc0: 000103a8 7c60 8000 c0a7fec0 c08f83a0 c0202dd0 
0008 00cfbff0
[  675.961329] fde0: dfc09d00 c08e8000 0051 0008 c0a7fec0 8000 
0008 0008
[  675.969535] fe00: 8000  dead4e40 8008 c0496ed1 c02fcf5c 
dead4e40 c0a7fec0
[  675.977749] fe20: c0a7fec0  8008 dead4e40 c08be920 c0a7feb8 
0001 
[  675.985964] fe40: c08beb60 c03a2f80 c0a7fe64 0003 dec8e2e0 8000 
0008 8008
[  675.994178] fe60: 5a9f1704      
 
[  676.002392] fe80: c0ace700 c0a7feb8 dec8e2e0 dec8e2e0 c0a892a0 00cebc48 
c0a7e000 
[  676.010607] fea0: 00511e6c c02ef4a8 c0a7ff10 c0a7ff28 dec8e2e0 c02ef548 
 
[  676.018821] fec0: 0001 0008 8000 c0a7ff08 0001 3b9a9904 
 
[  676.027035] fee0: 0040 c0a7ff28   c0a892a0 c0a7ff88 
8008 c01144c0
[  676.035249] ff00: 8008  00cebc48 8008 0001  
8008 c0a7ff08
[  676.043454] ff20: 0001 3b9a9904 c0a892a0    
 
[  676.051660] ff40:    c0a892a0 8008  
c0a7ff88 c011512c
[  676.059865] ff60: c0a892a0 00cebc48 8008 c0a892a0 c0a892a0 00cebc48 
8008 c000f724
[  676.068071] ff80: c0a7e000 c0115fe0   8008 00511e6c 
bef86848 bef867c8
[  676.076277] ffa0: 0004 c000f560 00511e6c bef86848 0004 00cebc48 
8008 00cebc48
[  676.084490] ffc0: 00511e6c bef86848 bef867c8 0004 0051fa80 00511e84 
0050f95c 00511e6c
[  676.092696] ffe0:  bef8666c 004c5978 b6ef7d1c 4010 0004 
1fffd871 1fffdc71
[  676.100910] [] (__check_object_size) from [] 
(copy_page_from_iter+0x2e8/0x3d0)
[  676.109915] [] (copy_page_from_iter) from [] 
(skb_copy_datagram_from_iter+0xfc/0x188)
[  676.119524] [] (skb_copy_datagram_from_iter) from [] 
(unix_stream_sendmsg+0x208/0x2f8)
[  676.129218] [] (unix_stream_sendmsg) from [] 
(sock_sendmsg+0x3c/0x50)
[  676.137430] [] (sock_sendmsg) from [] 
(sock_write_iter+0x8c/0xb4)
[  676.145297] [] (sock_write_iter) from [] 
(new_sync_write+0xc0/0xe4)
[  676.153337] [] (new_sync_write) from [] 
(vfs_write+0xc0/0x194)
[  676.160941] [] (vfs_write) from [] (SyS_write+0x44/0x7c)
[  676.168023] [] (SyS_write) from [] 
(ret_fast_syscall+0x0/0x44)
[  676.175625] Code: e59f10a0 01a01000 e59f009c ebff0495 (e7f001f2)
[  676.181748] ---[ end trace 09357b62c70de3ea ]---

What else can I try? There doesn't appear to be a newer kernel in proposed 

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Menno Finlay-Smits]

On Tue, 6 Mar 2018, at 04:57, Yves-Alexis Perez wrote:
> On Mon, 2018-03-05 at 15:28 +0100, Andrew Lunn wrote:
> > Would it be possible to try to reproduce this problem with 4.9.86 on
> > the hardware reporting the issue?
> 
> 4.9.82-1+deb9u3 is currently in the archive. Menno, could you give it a shot?

Will do. I need to get the NAS back to running Scratch as I went back to Jessie 
for comparison (similar looking problems there too) but I'll get it done as 
soon as I can.

Also, I can confirm that I was indeed copying from an external USB disk with 
just "rsync -av  ".

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Yves-Alexis Perez]

On Mon, 2018-03-05 at 15:28 +0100, Andrew Lunn wrote:
> Would it be possible to try to reproduce this problem with 4.9.86 on
> the hardware reporting the issue?

4.9.82-1+deb9u3 is currently in the archive. Menno, could you give it a shot?

Regards,
-- 
Yves-Alexis

signature.asc
Description: This is a digitally signed message part

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Andrew Lunn]

On Sun, Mar 04, 2018 at 09:41:15PM +0100, Andrew Lunn wrote:
> On Sun, Mar 04, 2018 at 06:41:57PM +0100, Martin Michlmayr wrote:
> > A Debian user reported the following issue on QNAP TS-119P II with
> > 4.9.65:
> > 
> > * Menno Finlay-Smits  [2018-01-21 23:08]:
> > > Rsyncing files between 2 HDDs on a QNAP 119p with a fresh, minimal 
> > > install of
> > > stretch NAS (armel) causes the kernel to fail after ~20mins with a kernel
> > > memory overwrite attempt (full error below). 
> 
> Please can you give me the exact rsync command being used. Having a
> unix domain socket seems a bit odd for rsync'ing files on the same
> machine.

I played with this a bit. When you rsync with both source and target
on the same machine, it appears to fork two processes, and connect
them together via a unix domain socket.

I tried reproducing this with 4.9.86. I used the command

rsync -az /home /root/home

which copied 12G of data. No problems seen. But this is one disk. I
assume the machine giving the problem has one of the disks as USB,
since the QNAP TS-119P II is a single bay NAS. So it could be the USB
disk is slower than the internal disk, and there is a backlog building
up. First a backlog for writing out to the disk, then a backlog
forming on the Unix domain socket? So maybe that is why i cannot
reproduce it. Or the problem has been fixed between 4.9.65 and 4.9.86.

Would it be possible to try to reproduce this problem with 4.9.86 on
the hardware reporting the issue?

Thanks
Andrew

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Andrew Lunn]

On Sun, Mar 04, 2018 at 06:41:57PM +0100, Martin Michlmayr wrote:
> A Debian user reported the following issue on QNAP TS-119P II with
> 4.9.65:
> 
> * Menno Finlay-Smits  [2018-01-21 23:08]:
> > Rsyncing files between 2 HDDs on a QNAP 119p with a fresh, minimal install 
> > of
> > stretch NAS (armel) causes the kernel to fail after ~20mins with a kernel
> > memory overwrite attempt (full error below). 

Please can you give me the exact rsync command being used. Having a
unix domain socket seems a bit odd for rsync'ing files on the same
machine.

Thanks
Andrew

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Andrew Lunn]

On Sun, Mar 04, 2018 at 06:41:57PM +0100, Martin Michlmayr wrote:
> A Debian user reported the following issue on QNAP TS-119P II with
> 4.9.65:
> 
> * Menno Finlay-Smits  [2018-01-21 23:08]:
> > Rsyncing files between 2 HDDs on a QNAP 119p with a fresh, minimal install 
> > of
> > stretch NAS (armel) causes the kernel to fail after ~20mins with a kernel
> > memory overwrite attempt (full error below). 
> > 
> > This happens reliably for any large rsync attempt. I have about 1TB of data 
> > to
> > copy between these 2 HDDs and have not managed to copy more than ~2% of the
> > total amount.
> > 
> > ** Kernel log:
> > 
> > [ 2775.213733] usercopy: kernel memory overwrite attempt detected to 
> > c29454e0 () (4294802208 bytes)

Not seen this before.

My first thought is that this actually looks like a userspace
problem. Userspace is passing 4294802208 bytes to the kernel. But the
kernel should of already sanity checked that before trying to copy it
into kernel space. This is also a Unix domain socket, which sounds odd
for rsync. And this is all generic code, nothing specific to kirkwood.

Has there been any similar reports on other targets?

Andrew

> > [ 2775.224095] [ cut here ]
> > [ 2775.228728] kernel BUG at 
> > /build/linux-myVvPm/linux-4.9.65/mm/usercopy.c:75!
> > [ 2775.235800] Internal error: Oops - BUG: 0 [#1] ARM
> > [ 2775.240604] Modules linked in: marvell ehci_orion mvmdio mv643xx_eth 
> > ehci_hcd of_mdio fixed_phy xhci_pci xhci_hcd marvell_cesa des_generic sg 
> > usbcore libphy m25p80 spi_nor orion_wdt usb_common kirkwood_thermal evdev 
> > gpio_keys ip_tables x_tables ipv6 autofs4 ext4 crc16 jbd2 crc32c_generic 
> > fscrypto ecb mbcache sd_mod sata_mv libata scsi_mod
> > [ 2775.271023] CPU: 0 PID: 601 Comm: rsync Not tainted 4.9.0-5-marvell #1 
> > Debian 4.9.65-3+deb9u2
> > [ 2775.279582] Hardware name: Marvell Kirkwood (Flattened Device Tree)
> > [ 2775.285870] task: c0d496c0 task.stack: d5ffe000
> > [ 2775.290418] PC is at __check_object_size+0x120/0x1d8
> > [ 2775.295401] LR is at __check_object_size+0x120/0x1d8
> > [ 2775.300382] pc : []lr : []psr: 6013
> >sp : d5fffdb8  ip :   fp : d508
> > [ 2775.311908] r10: d5ffe000  r9 : fffd7b20  r8 : c29454e0
> > [ 2775.317148] r7 : c291d000  r6 :   r5 : fffd7b20  r4 : c29454e0
> > [ 2775.323697] r3 : c0554fa0  r2 : c055a20c  r1 : c055094c  r0 : 0065
> > [ 2775.330247] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment 
> > none
> > [ 2775.337405] Control: 0005397f  Table: 1481  DAC: 0051
> > [ 2775.343168] Process rsync (pid: 601, stack limit = 0xd5ffe190)
> > [ 2775.349020] Stack: (0xd5fffdb8 to 0xd600)
> > [ 2775.353390] fda0:   
> > c04623b8 fffd7b20
> > [ 2775.361598] fdc0: 000294e8 fffd7b20 1000 d5fffec0 c29454e0 c0202360 
> > 0008 008eafe8
> > [ 2775.369812] fde0: dfc4a380 c291c000 0051 6908 d5fffec0 8000 
> > 0008 0008
> > [ 2775.378026] fe00: 1000  c0c26b40 1008 c0495cf7 c02fc3d0 
> > c0c26b40 d5fffec0
> > [ 2775.386240] fe20: d5fffec0  8008 c0c26b40 df782d80 d5fffeb8 
> > 0001 
> > [ 2775.394445] fe40: df782b40 c03a21d0 d5fffe64 0003 de65b2c0 8000 
> > 0008 8008
> > [ 2775.402651] fe60: 5a644f89      
> >  
> > [ 2775.410866] fe80: d2bebb80 d5fffeb8 de65b2c0 de65b2c0 df79caa0 008c1b00 
> > d5ffe000 
> > [ 2775.419080] fea0: 00512e6c c02ee92c d510 d528 de65b2c0 c02ee9cc 
> >  
> > [ 2775.427294] fec0: 0001 0008 8000 d508 0001 3b9aa9ee 
> >  
> > [ 2775.435499] fee0: 0040 d528   df79caa0 d588 
> > 8008 c0114048
> > [ 2775.443705] ff00: 8008  008c1b00 8008 0001  
> > 8008 d508
> > [ 2775.451909] ff20: 0001 3b9aa9ee df79caa0    
> >  
> > [ 2775.460116] ff40:    df79caa0 8008  
> > d588 c0114cb4
> > [ 2775.468321] ff60: df79caa0 008c1b00 8008 df79caa0 df79caa0 008c1b00 
> > 8008 c000f704
> > [ 2775.476527] ff80: d5ffe000 c0115b68   8008 00512e6c 
> > bedfb878 bedfb7f8
> > [ 2775.484733] ffa0: 0004 c000f560 00512e6c bedfb878 0004 008c1b00 
> > 8008 008c1b00
> > [ 2775.492947] ffc0: 00512e6c bedfb878 bedfb7f8 0004 00520a80 00512e84 
> > 0051095c 00512e6c
> > [ 2775.501161] ffe0:  bedfb69c 004c6978 b6ea3d1c 4010 0004 
> > 624f 624f
> > [ 2775.509384] [] (__check_object_size) from [] 
> > (copy_page_from_iter+0x2e8/0x3d0)
> > [ 2775.518388] [] (copy_page_from_iter) from [] 
> > (skb_copy_datagram_from_iter+0xfc/0x188)
> > [ 2775.527997] [] (skb_copy_datagram_from_iter) from [] 
> > (unix_stream_sendmsg+0x208/0x2f8)
> > [ 2775.537691] [] 

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Martin Michlmayr]

A Debian user reported the following issue on QNAP TS-119P II with
4.9.65:

* Menno Finlay-Smits  [2018-01-21 23:08]:
> Rsyncing files between 2 HDDs on a QNAP 119p with a fresh, minimal install of
> stretch NAS (armel) causes the kernel to fail after ~20mins with a kernel
> memory overwrite attempt (full error below). 
> 
> This happens reliably for any large rsync attempt. I have about 1TB of data to
> copy between these 2 HDDs and have not managed to copy more than ~2% of the
> total amount.
> 
> ** Kernel log:
> 
> [ 2775.213733] usercopy: kernel memory overwrite attempt detected to c29454e0 
> () (4294802208 bytes)
> [ 2775.224095] [ cut here ]
> [ 2775.228728] kernel BUG at 
> /build/linux-myVvPm/linux-4.9.65/mm/usercopy.c:75!
> [ 2775.235800] Internal error: Oops - BUG: 0 [#1] ARM
> [ 2775.240604] Modules linked in: marvell ehci_orion mvmdio mv643xx_eth 
> ehci_hcd of_mdio fixed_phy xhci_pci xhci_hcd marvell_cesa des_generic sg 
> usbcore libphy m25p80 spi_nor orion_wdt usb_common kirkwood_thermal evdev 
> gpio_keys ip_tables x_tables ipv6 autofs4 ext4 crc16 jbd2 crc32c_generic 
> fscrypto ecb mbcache sd_mod sata_mv libata scsi_mod
> [ 2775.271023] CPU: 0 PID: 601 Comm: rsync Not tainted 4.9.0-5-marvell #1 
> Debian 4.9.65-3+deb9u2
> [ 2775.279582] Hardware name: Marvell Kirkwood (Flattened Device Tree)
> [ 2775.285870] task: c0d496c0 task.stack: d5ffe000
> [ 2775.290418] PC is at __check_object_size+0x120/0x1d8
> [ 2775.295401] LR is at __check_object_size+0x120/0x1d8
> [ 2775.300382] pc : []lr : []psr: 6013
>sp : d5fffdb8  ip :   fp : d508
> [ 2775.311908] r10: d5ffe000  r9 : fffd7b20  r8 : c29454e0
> [ 2775.317148] r7 : c291d000  r6 :   r5 : fffd7b20  r4 : c29454e0
> [ 2775.323697] r3 : c0554fa0  r2 : c055a20c  r1 : c055094c  r0 : 0065
> [ 2775.330247] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment 
> none
> [ 2775.337405] Control: 0005397f  Table: 1481  DAC: 0051
> [ 2775.343168] Process rsync (pid: 601, stack limit = 0xd5ffe190)
> [ 2775.349020] Stack: (0xd5fffdb8 to 0xd600)
> [ 2775.353390] fda0:   
> c04623b8 fffd7b20
> [ 2775.361598] fdc0: 000294e8 fffd7b20 1000 d5fffec0 c29454e0 c0202360 
> 0008 008eafe8
> [ 2775.369812] fde0: dfc4a380 c291c000 0051 6908 d5fffec0 8000 
> 0008 0008
> [ 2775.378026] fe00: 1000  c0c26b40 1008 c0495cf7 c02fc3d0 
> c0c26b40 d5fffec0
> [ 2775.386240] fe20: d5fffec0  8008 c0c26b40 df782d80 d5fffeb8 
> 0001 
> [ 2775.394445] fe40: df782b40 c03a21d0 d5fffe64 0003 de65b2c0 8000 
> 0008 8008
> [ 2775.402651] fe60: 5a644f89      
>  
> [ 2775.410866] fe80: d2bebb80 d5fffeb8 de65b2c0 de65b2c0 df79caa0 008c1b00 
> d5ffe000 
> [ 2775.419080] fea0: 00512e6c c02ee92c d510 d528 de65b2c0 c02ee9cc 
>  
> [ 2775.427294] fec0: 0001 0008 8000 d508 0001 3b9aa9ee 
>  
> [ 2775.435499] fee0: 0040 d528   df79caa0 d588 
> 8008 c0114048
> [ 2775.443705] ff00: 8008  008c1b00 8008 0001  
> 8008 d508
> [ 2775.451909] ff20: 0001 3b9aa9ee df79caa0    
>  
> [ 2775.460116] ff40:    df79caa0 8008  
> d588 c0114cb4
> [ 2775.468321] ff60: df79caa0 008c1b00 8008 df79caa0 df79caa0 008c1b00 
> 8008 c000f704
> [ 2775.476527] ff80: d5ffe000 c0115b68   8008 00512e6c 
> bedfb878 bedfb7f8
> [ 2775.484733] ffa0: 0004 c000f560 00512e6c bedfb878 0004 008c1b00 
> 8008 008c1b00
> [ 2775.492947] ffc0: 00512e6c bedfb878 bedfb7f8 0004 00520a80 00512e84 
> 0051095c 00512e6c
> [ 2775.501161] ffe0:  bedfb69c 004c6978 b6ea3d1c 4010 0004 
> 624f 624f
> [ 2775.509384] [] (__check_object_size) from [] 
> (copy_page_from_iter+0x2e8/0x3d0)
> [ 2775.518388] [] (copy_page_from_iter) from [] 
> (skb_copy_datagram_from_iter+0xfc/0x188)
> [ 2775.527997] [] (skb_copy_datagram_from_iter) from [] 
> (unix_stream_sendmsg+0x208/0x2f8)
> [ 2775.537691] [] (unix_stream_sendmsg) from [] 
> (sock_sendmsg+0x3c/0x50)
> [ 2775.545903] [] (sock_sendmsg) from [] 
> (sock_write_iter+0x8c/0xb4)
> [ 2775.553771] [] (sock_write_iter) from [] 
> (new_sync_write+0xc0/0xe4)
> [ 2775.561810] [] (new_sync_write) from [] 
> (vfs_write+0xc0/0x194)
> [ 2775.569414] [] (vfs_write) from [] 
> (SyS_write+0x44/0x7c)
> [ 2775.576497] [] (SyS_write) from [] 
> (ret_fast_syscall+0x0/0x38)
> [ 2775.584098] Code: e59f10a0 01a01000 e59f009c ebff04bf (e7f001f2)
> [ 2775.590218] ---[ end trace 9c6c6370c712b384 ]---

> 
> ** Network status:
> *** IP interfaces and addresses:
> 1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group 
> default qlen 1
> 

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Menno Finlay-Smits]

I messed up the NAS model number when editing the bug description. To be clear, 
it's a "QNAP TS-119p II".

Bug#887873: linux-image-4.9.0-5-marvell: frequent "usercopy: kernel memory overwrite attempt detected" on QNAP NAS (ARM)

[Menno Finlay-Smits]

Package: src:linux
Version: 4.9.65-3+deb9u2
Severity: critical
Justification: breaks the whole system

Dear Maintainer,

Rsyncing files between 2 HDDs on a QNAP 119p with a fresh, minimal install of
stretch NAS (armel) causes the kernel to fail after ~20mins with a kernel
memory overwrite attempt (full error below). 

This happens reliably for any large rsync attempt. I have about 1TB of data to
copy between these 2 HDDs and have not managed to copy more than ~2% of the
total amount.

-- Package-specific info:
** Version:
Linux version 4.9.0-5-marvell (debian-ker...@lists.debian.org) (gcc version 
6.3.0 20170516 (Debian 6.3.0-18) ) #1 Debian 4.9.65-3+deb9u2 (2018-01-04)

** Command line:
console=ttyS0,115200 root=/dev/ram initrd=0xa0,0x90 ramdisk=34816

** Not tainted

** Kernel log:

[ 2775.213733] usercopy: kernel memory overwrite attempt detected to c29454e0 
() (4294802208 bytes)
[ 2775.224095] [ cut here ]
[ 2775.228728] kernel BUG at /build/linux-myVvPm/linux-4.9.65/mm/usercopy.c:75!
[ 2775.235800] Internal error: Oops - BUG: 0 [#1] ARM
[ 2775.240604] Modules linked in: marvell ehci_orion mvmdio mv643xx_eth 
ehci_hcd of_mdio fixed_phy xhci_pci xhci_hcd marvell_cesa des_generic sg 
usbcore libphy m25p80 spi_nor orion_wdt usb_common kirkwood_thermal evdev 
gpio_keys ip_tables x_tables ipv6 autofs4 ext4 crc16 jbd2 crc32c_generic 
fscrypto ecb mbcache sd_mod sata_mv libata scsi_mod
[ 2775.271023] CPU: 0 PID: 601 Comm: rsync Not tainted 4.9.0-5-marvell #1 
Debian 4.9.65-3+deb9u2
[ 2775.279582] Hardware name: Marvell Kirkwood (Flattened Device Tree)
[ 2775.285870] task: c0d496c0 task.stack: d5ffe000
[ 2775.290418] PC is at __check_object_size+0x120/0x1d8
[ 2775.295401] LR is at __check_object_size+0x120/0x1d8
[ 2775.300382] pc : []lr : []psr: 6013
   sp : d5fffdb8  ip :   fp : d508
[ 2775.311908] r10: d5ffe000  r9 : fffd7b20  r8 : c29454e0
[ 2775.317148] r7 : c291d000  r6 :   r5 : fffd7b20  r4 : c29454e0
[ 2775.323697] r3 : c0554fa0  r2 : c055a20c  r1 : c055094c  r0 : 0065
[ 2775.330247] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[ 2775.337405] Control: 0005397f  Table: 1481  DAC: 0051
[ 2775.343168] Process rsync (pid: 601, stack limit = 0xd5ffe190)
[ 2775.349020] Stack: (0xd5fffdb8 to 0xd600)
[ 2775.353390] fda0:   
c04623b8 fffd7b20
[ 2775.361598] fdc0: 000294e8 fffd7b20 1000 d5fffec0 c29454e0 c0202360 
0008 008eafe8
[ 2775.369812] fde0: dfc4a380 c291c000 0051 6908 d5fffec0 8000 
0008 0008
[ 2775.378026] fe00: 1000  c0c26b40 1008 c0495cf7 c02fc3d0 
c0c26b40 d5fffec0
[ 2775.386240] fe20: d5fffec0  8008 c0c26b40 df782d80 d5fffeb8 
0001 
[ 2775.394445] fe40: df782b40 c03a21d0 d5fffe64 0003 de65b2c0 8000 
0008 8008
[ 2775.402651] fe60: 5a644f89      
 
[ 2775.410866] fe80: d2bebb80 d5fffeb8 de65b2c0 de65b2c0 df79caa0 008c1b00 
d5ffe000 
[ 2775.419080] fea0: 00512e6c c02ee92c d510 d528 de65b2c0 c02ee9cc 
 
[ 2775.427294] fec0: 0001 0008 8000 d508 0001 3b9aa9ee 
 
[ 2775.435499] fee0: 0040 d528   df79caa0 d588 
8008 c0114048
[ 2775.443705] ff00: 8008  008c1b00 8008 0001  
8008 d508
[ 2775.451909] ff20: 0001 3b9aa9ee df79caa0    
 
[ 2775.460116] ff40:    df79caa0 8008  
d588 c0114cb4
[ 2775.468321] ff60: df79caa0 008c1b00 8008 df79caa0 df79caa0 008c1b00 
8008 c000f704
[ 2775.476527] ff80: d5ffe000 c0115b68   8008 00512e6c 
bedfb878 bedfb7f8
[ 2775.484733] ffa0: 0004 c000f560 00512e6c bedfb878 0004 008c1b00 
8008 008c1b00
[ 2775.492947] ffc0: 00512e6c bedfb878 bedfb7f8 0004 00520a80 00512e84 
0051095c 00512e6c
[ 2775.501161] ffe0:  bedfb69c 004c6978 b6ea3d1c 4010 0004 
624f 624f
[ 2775.509384] [] (__check_object_size) from [] 
(copy_page_from_iter+0x2e8/0x3d0)
[ 2775.518388] [] (copy_page_from_iter) from [] 
(skb_copy_datagram_from_iter+0xfc/0x188)
[ 2775.527997] [] (skb_copy_datagram_from_iter) from [] 
(unix_stream_sendmsg+0x208/0x2f8)
[ 2775.537691] [] (unix_stream_sendmsg) from [] 
(sock_sendmsg+0x3c/0x50)
[ 2775.545903] [] (sock_sendmsg) from [] 
(sock_write_iter+0x8c/0xb4)
[ 2775.553771] [] (sock_write_iter) from [] 
(new_sync_write+0xc0/0xe4)
[ 2775.561810] [] (new_sync_write) from [] 
(vfs_write+0xc0/0x194)
[ 2775.569414] [] (vfs_write) from [] (SyS_write+0x44/0x7c)
[ 2775.576497] [] (SyS_write) from [] 
(ret_fast_syscall+0x0/0x38)
[ 2775.584098] Code: e59f10a0 01a01000 e59f009c ebff04bf (e7f001f2)
[ 2775.590218] ---[ end trace 9c6c6370c712b384 ]---


** Model information
Hardware: Marvell Kirkwood (Flattened