Hi Joe,
> On Nov 5, 2019, at 6:04 PM, Joe Darcy <joe.da...@oracle.com> wrote: > > Please review the changes to fix > > JDK-8233452: java.math.BigDecimal.sqrt() with RoundingMode.FLOOR results > in incorrect result > http://cr.openjdk.java.net/~darcy/8233452.0/ > <http://cr.openjdk.java.net/~darcy/8233452.0/> > > Some background on the problem and fix. > > The core of the BigDecimal.sqrt method is a Newton-Raphson loop. > […] > The Newton iteration reduces the error at each step and, as currently > written, it can settle on a value like 2, which is closer to the actual > answer, but *incorrect* according to the requested rounding mode. > > There are a few ways to fix this one. One is to run the Newton loop to a > higher-precision where these finer distinctions can be teased out. […] > > Another approach is to detect when the result is too large/too small and then > subtract/add an ulp as a fix-up. This is the approach taken for > BigDecimal.sqrt. […] This implementation change looks reasonable to me. > An approach not explored for this fix would be to arrange for the iteration > to start from the "right" side of the number line depending on the rounding > mode and then monotonically increase/decrease to approach the square root. > […] I believe this is technically possible, but would require some additional > analysis to setup. This approach seems more appealing and could perhaps be investigated at a later date. In the test I think you want to replace “down” with “up" at line 238. Brian
