On 12/23/2011 11:29 AM, Michael Büsch wrote:
> On Thu, 22 Dec 2011 15:16:59 +0100
> Michael Büsch<m...@bues.ch> wrote:
>
>> On Thu, 22 Dec 2011 07:40:19 -0600
>> Jeff Epler<jep...@unpythonic.net> wrote:
>>
>>> I understand the rest, but what's the + 0.1 part doing?
Curiously I was working on a similar issue using Java. I'm processing a
database of capacitors and resistors and converting them to enotation
then back and ran into rounding issues.
Here's an ugly attempt that's not fully tested, but gives the idea:
ndigits are the number of non-zero digits: I'll post a the final version
if interested.
I want to specify the number if sig digits I want for a fractional value:
//ex. 0.000000470266543763 9.21k 9210
static float Rounder(float in, int nDigits)
{
double mult = 1;
int len = 1;
String inStr = String.valueOf(in); //converts the input float
to a string enotation is automatically used if needed.
String strTemp1 = inStr; // todo was going to split the string
to further process.
len = inStr.length(); // not used
int exp = (int) Math.log10(in); // grab the exponent for
creating a x10 power multiplier to 'normalize' the value for us
mult = Math.pow(10,-exp + nDigits + 1);
//
float i = (float) (Math.round(in * mult) / mult);
return(i);
}
>> The +0.1 is to protect against floating point inaccuracies.
>> Imagine the ceil() operation yields a 200.0, for example. But
>> due to floating point inaccuracies it's actually represented
>> as 199.9999999999999. So the (implicit) int cast would (tail)-cast
>> it to 199, instead of the correct 200. The +0.1 avoids that.
>>
>>> fwiw the code was added to fix this bug:
>>> http://sourceforge.net/tracker/?func=detail&aid=2478266&group_id=6744&atid=106744
>>>
>>> I think that ceil, round or (implicit in integer division) floor are all
>>> OK as fixes to the original problem, so I'd be tempted to just drop the
>>> useless ceil() call instead and leave the behavior unchanged.
>> Ok, well. I'm don't really understand the code good enough to have an
>> opinion on this, but changing the code to this:
>>
>> servo_mult = traj_period_nsec / nsec;
>>
>> should maintain the current semantics, but avoid the floating point stuff.
>>
>> Please also note that some older compilers are confused by that floating
>> point stuff there and throw internal errors. (That's how I got attracted
>> to that code). So it should be fixed either way.
>
> Should I resend the patch, or are you going to apply that manually?
>
------------------------------------------------------------------------------
Write once. Port to many.
Get the SDK and tools to simplify cross-platform app development. Create
new or port existing apps to sell to consumers worldwide. Explore the
Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join
http://p.sf.net/sfu/intel-appdev
_______________________________________________
Emc-developers mailing list
Emc-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/emc-developers
