Taking the vector norm using the naïve method of adding up the squares and then taking the square root is known to be problematic in terms of overflow for very large arguments. There are always some small problems when using square-roots since it can be hard to get accuracy to 1ULPS without standing on your head.
Are you saying that this is the problem? Can you say more about the error you see? On 4/18/08 10:26 AM, "Sergey Chickin (JIRA)" <[EMAIL PROTECTED]> wrote: > I get an error caused by storing doubles in memory and resulting in > calculations error(actually taking square root when calculating vector norm)
