Well you answered my question.
Isn't three going to be the only non-two factor? (Which in my previous
email, hadn't thought possible -- I thought 24bpp was treated as 32bpp).
Anyway the better way (if still doing factoring) is to use a loop like
this:
x = mode->HDisplay
y = mode->VDisplay
z = scrp->videoRam
while ( (! (x * y & 0x01)) &&
(! (z & 0x01)) ) {
if (x & 0x01) {
y <<= 1;
} else {
x <<= 1;
}
z <<= 1;
}
if ( x*y > z )
return MODE_MEM;
// p.s. I left out the 1024, 8 and scrp->fbFormat.bitsPerPixel parts
// sorry
Anyway I would use the above only if still paranoid that a 64bit
number wouldn't work. And 64bit math by hand is not hard if it
doesn't have to be fast -- so that seems to be the best approach.
-Gryn (Adam Luter)
On Tue, Nov 05, 2002 at 03:06:09AM +0100, Alexander Stohr wrote:
> Adam Luter [mailto:luterac@;bellsouth.net] wrote:
>
> > Why not use math?
> >
> > factor = 1024 * 8 / scrp->fbFormat.bitsPerPixel;
> > if (mode->HDisplay * mode->VDisplay > scrp->videoRam * factor)
> > return MODE_MEM;
> >
> > This assumes that scrp->fbFormat.bitsPerPixel is a power of two.
> >
> > Also, really huge monochrome displays are still vunerable. In
> > other words, I would also use a 64bit number.
> >
> > -Gryn
>
> concerning your formula, I am applying
> a 24 bpp wide packed pixel format:
>
> 1024*8/24 = 341.333
>
> This case might lead to unwanted fractional loss.
> Using a special case handling for 24 bpp and similar
> would mean that any bit depth could go that way anyways.
> But then I do see no need for an "optimized" path at all.
>
> Your version further does not care about
> that HDisp and VDisp might multiply to
> a value bigger than 2 GB and therfore getting
> negatvie if input values were choosen evil minded.
>
> I considered such a division approach myself,
> but in the end this is only save if you
> can ensure limited ranges for all numbers.
>
> But we should not build up further limits
> since there were already two suggestions
> that do not need that at all
>
> One thing to discuss is the likeliness for
> 64 bit math support versus the support for
> integer division on the target platform.
>
> -Alex.
_______________________________________________
Xpert mailing list
[EMAIL PROTECTED]
http://XFree86.Org/mailman/listinfo/xpert
