[PATCH] drm/vmwgfx: fix incorrect VRAM size check in vmw_kms_fb_create()

2011-12-10 Thread Xi Wang
The commit e133e737 didn't correctly fix the overflow issue. - unsigned int required_size; + u64 required_size; ... required_size = mode_cmd-pitch * mode_cmd-height; - if (unlikely(required_size dev_priv-vram_size)) { + if (unlikely(required_size (u64)

[PATCH] drm/vmwgfx: fix incorrect VRAM size check in vmw_kms_fb_create()

2011-12-08 Thread Xi Wang
The commit e133e737 didn't correctly fix the overflow issue. - unsigned int required_size; + u64 required_size; ... required_size = mode_cmd->pitch * mode_cmd->height; - if (unlikely(required_size > dev_priv->vram_size)) { + if (unlikely(required_size >