On 09/18/2011 09:30 AM, Christoph Gohlke wrote: > Hello, > > matplotlib uses int(x*255) or np.array(x*255, np.uint8) to quantize > normalized floating point numbers x in the range [0.0 to 1.0] to > integers in the range [0 to 255]. This way only 1.0 is mapped to 255, > not for example 0.999. Is this really intended or would not the largest > floating point number below 256.0 be a better scale factor than 255? The > exact factor depends on the floating point precision (~255.999992 for > np.float32, ~255.93 for np.float16). > > Christoph
Christoph, It's a reasonable question; but do you have use cases in mind where it actually makes a difference? The simple scaling with truncation is used in many places, both in the python and the c++ code. Eric ------------------------------------------------------------------------------ BlackBerry® DevCon Americas, Oct. 18-20, San Francisco, CA http://p.sf.net/sfu/rim-devcon-copy2 _______________________________________________ Matplotlib-devel mailing list Matplotlib-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-devel
