Re: [PATCH v2] fbdev: Detect integer underflow at "struct fbcon_ops"->clear_margins.

2020-07-24 Thread Bartlomiej Zolnierkiewicz
048576, for >>>> "32768 rows or cols" * "32 pixels per character" = 1045876 and >>>> vc_do_resize() accepts only >>>> rows and cols < 32768 ? >>>> >>>>> >>>>> Just to avoid confusion here. >&g

Re: [PATCH v2] fbdev: Detect integer underflow at "struct fbcon_ops"->clear_margins.

2020-07-23 Thread Greg Kroah-Hartman
"32768 rows or cols" * "32 pixels per character" = 1045876 and > > > vc_do_resize() accepts only > > > rows and cols < 32768 ? > > > > > > > > > > > Just to avoid confusion here. > > > > > > > >

Re: [PATCH v2] fbdev: Detect integer underflow at "struct fbcon_ops"->clear_margins.

2020-07-22 Thread Daniel Vetter
r > > "32768 rows or cols" * "32 pixels per character" = 1045876 and > > vc_do_resize() accepts only > > rows and cols < 32768 ? > > > > > > > > Just to avoid confusion here. > > > > > >> Anyway, my two patches are

Re: [PATCH v2] fbdev: Detect integer underflow at "struct fbcon_ops"->clear_margins.

2020-07-21 Thread Greg Kroah-Hartman
fusion here. > > > >> Anyway, my two patches are small and low cost; can we apply these patches > >> regardless > >> of basic checks? > > > > Which two patches where? > > [PATCH v3] vt: Reject zero-sized screen buffer size. > from > https://lkml.k

Re: [PATCH v2] fbdev: Detect integer underflow at "struct fbcon_ops"->clear_margins.

2020-07-17 Thread Tetsuo Handa
() accepts only rows and cols < 32768 ? > > Just to avoid confusion here. > >> Anyway, my two patches are small and low cost; can we apply these patches >> regardless >> of basic checks? > > Which two patches where? [PATCH v3] vt: Reject zero-sized screen buffer

Re: [PATCH v2] fbdev: Detect integer underflow at "struct fbcon_ops"->clear_margins.

2020-07-16 Thread Daniel Vetter
On Thu, Jul 16, 2020 at 12:29:00AM +0900, Tetsuo Handa wrote: > On 2020/07/16 0:12, Dan Carpenter wrote: > > I've complained about integer overflows in fbdev for a long time... > > > > What I'd like to see is something like the following maybe. I don't > > know how to get the vc_data in fbmem.c

Re: [PATCH v2] fbdev: Detect integer underflow at "struct fbcon_ops"->clear_margins.

2020-07-16 Thread Tetsuo Handa
On 2020/07/16 0:12, Dan Carpenter wrote: > I've complained about integer overflows in fbdev for a long time... > > What I'd like to see is something like the following maybe. I don't > know how to get the vc_data in fbmem.c so it doesn't include your checks > for negative. Yes. Like I said

Re: [PATCH v2] fbdev: Detect integer underflow at "struct fbcon_ops"->clear_margins.

2020-07-16 Thread Tetsuo Handa
On 2020/07/15 18:48, Dan Carpenter wrote: >> @@ -216,7 +216,7 @@ static void bit_clear_margins(struct vc_data *vc, struct >> fb_info *info, >> region.color = color; >> region.rop = ROP_COPY; >> >> -if (rw && !bottom_only) { >> +if ((int) rw > 0 && !bottom_only) { >>

Re: [PATCH v2] fbdev: Detect integer underflow at "struct fbcon_ops"->clear_margins.

2020-07-16 Thread Tetsuo Handa
On 2020/07/15 20:17, Tetsuo Handa wrote: > On 2020/07/15 18:48, Dan Carpenter wrote: >>> @@ -216,7 +216,7 @@ static void bit_clear_margins(struct vc_data *vc, >>> struct fb_info *info, >>> region.color = color; >>> region.rop = ROP_COPY; >>> >>> - if (rw && !bottom_only) { >>> + if

Re: [PATCH v2] fbdev: Detect integer underflow at "struct fbcon_ops"->clear_margins.

2020-07-15 Thread Dan Carpenter
On Wed, Jul 15, 2020 at 11:02:58PM +0900, Tetsuo Handa wrote: > On 2020/07/15 20:17, Tetsuo Handa wrote: > > On 2020/07/15 18:48, Dan Carpenter wrote: > >>> @@ -216,7 +216,7 @@ static void bit_clear_margins(struct vc_data *vc, > >>> struct fb_info *info, > >>> region.color = color; > >>>

Re: [PATCH v2] fbdev: Detect integer underflow at "struct fbcon_ops"->clear_margins.

2020-07-15 Thread Dan Carpenter
On Wed, Jul 15, 2020 at 10:51:02AM +0900, Tetsuo Handa wrote: > syzbot is reporting general protection fault in bitfill_aligned() [1] > caused by integer underflow in bit_clear_margins(). The cause of this > problem is when and how do_vc_resize() updates vc->vc_{cols,rows}. > > If vc_do_resize()

[PATCH v2] fbdev: Detect integer underflow at "struct fbcon_ops"->clear_margins.

2020-07-15 Thread Tetsuo Handa
syzbot is reporting general protection fault in bitfill_aligned() [1] caused by integer underflow in bit_clear_margins(). The cause of this problem is when and how do_vc_resize() updates vc->vc_{cols,rows}. If vc_do_resize() fails (e.g. kzalloc() fails) when var.xres or var.yres is going to