from:"Raffaello Giulietti"

OK, from what I read so far from you is that you were surprised about 
the order, not about why verification happens even without initialization.


Verification is part of linking, which surely must happen before user 
code (e.g., initialization) is executed, which explains the order.


But once linked (thus, verified), there's no pressure to have 
initialization code (user code) executed. This can wait until the very 
last moment, when some other code of the class is invoked, or some field 
is accessed, or an instance is created, etc. It's a lazy execution 
strategy, which the JVM spec allows and which HotSpot implements this way.


HTH
Raffaello


On 2022-03-18 01:51, Cheng Jin wrote:

Hi Raffaello,

My concern is why the verification happens even without initialization in the 
test without mh.invoke() in the main(), which I don't think is covered or 
explained in the JVM Spec.
Put it in another way, my understanding is, when the class gets loaded, it is 
verified which doesn't necessarily lead to initialization, am I correct?

Best Regards
Cheng

-Original Message-
From: core-libs-dev  On Behalf Of 
Raffaello Giulietti
Sent: March 17, 2022 8:35 PM
To: core-libs-dev@openjdk.java.net
Subject: [EXTERNAL] Re: When to initialize the method's class for 
MethodHandles.Lookup.findStatic()?

Cheng,

initialization is the last thing that happens because it's where user provided 
code gets executed.

This has always been this way, as long as I can remember. See the JVMS for the 
gory details.


Greetings
Raffaello


On 2022-03-18 01:21, Cheng Jin wrote:

Hi David,

1) for the test with mh.invoke() in main(),  the log shows:
[0.262s][info][class,init] Start class verification for: Test_1
[0.262s][info][class,init] End class verification for: Test_1
[0.263s][info][class,init] 282 Initializing 'Test_1' (0x000800c00800)
[0.263s][info][class,init] Start class verification for: Test_2
[0.263s][info][class,init] End class verification for: Test_2
[0.272s][info][class,init] 366 Initializing 'Test_2' (0x000800c00a08) 
<--

2) for the test without  mh.invoke() in main(),  the log shows:
[0.296s][info][class,init] Start class verification for: Test_1
[0.296s][info][class,init] End class verification for: Test_1
[0.297s][info][class,init] 282 Initializing 'Test_1' (0x000800c00800)
[0.297s][info][class,init] Start class verification for: Test_2
[0.297s][info][class,init] End class verification for: Test_2
(Test_2 was verified but didn't get initialized)

The comparison above literally surprised me that the bytecode verification 
happened prior to the class initialization, which means
the class got verified at first even without initialization coz I previously 
thought the initialization should trigger the verification rather than in the 
reversed order.

Could you explain a little more about why it goes in this way?

Best Regards
Cheng


-Original Message-
From: core-libs-dev  On Behalf Of David 
Holmes
Sent: March 17, 2022 7:46 PM
To: Raffaello Giulietti ; 
core-libs-dev@openjdk.java.net
Subject: [EXTERNAL] Re: When to initialize the method's class for 
MethodHandles.Lookup.findStatic()?

Run with -Xlog:class+init=info to see the classes that get initialized and in 
what order.

David

On 18/03/2022 5:53 am, Raffaello Giulietti wrote:

Hi again,

here's code that shows that initialization doesn't happen during
lookup but only upon invoking the method handle. (I'm on Java 17.)

As long as the 2nd line in main() is commented, you don't see the
message "Test_2 initialized", which shows that the lookup doesn't
initialize Test_2.
When you uncomment the line in main(), the message will appear.

So, as advertised, it's the invocation of the method handle that can
trigger initialization, not the lookup.


HTH
Raffaello



import java.lang.invoke.*;

public class Test_1 {

       static MethodHandle mh;

       static {
       try {
       mh = MethodHandles.lookup().findStatic(Test_2.class,
"testMethod", MethodType.methodType(int.class, int.class));
       } catch (NoSuchMethodException | IllegalAccessException e) {
       e.printStackTrace();
       }
       }

       public static void main(String[] args) throws Throwable {
       System.out.println(mh);
       // System.out.println(Test_1.mh.invoke(0));
       }

}




public class Test_2 {

       static {
       System.out.println("Test_2 initialized");
       }

       static int testMethod(int value) { return (value + 1); }

}




On 2022-03-17 20:38, Raffaello Giulietti wrote:

Hi,

as far as I can see, the code should not even compile, as there's a
static field in Test_1 which is initialized with an expression that
throws checked exceptions (findStatic(..)).

In addition, it seems to me that there's nothing in your code that
reveals whether Test_2 has been initialized during the lookup. How
can you tell?

Finally, the method handle invocation in Test_1 will thro

Re: When to initialize the method's class for MethodHandles.Lookup.findStatic()?


Cheng,

initialization is the last thing that happens because it's where user 
provided code gets executed.


This has always been this way, as long as I can remember. See the JVMS 
for the gory details.



Greetings
Raffaello


On 2022-03-18 01:21, Cheng Jin wrote:

Hi David,

1) for the test with mh.invoke() in main(),  the log shows:
[0.262s][info][class,init] Start class verification for: Test_1
[0.262s][info][class,init] End class verification for: Test_1
[0.263s][info][class,init] 282 Initializing 'Test_1' (0x000800c00800)
[0.263s][info][class,init] Start class verification for: Test_2
[0.263s][info][class,init] End class verification for: Test_2
[0.272s][info][class,init] 366 Initializing 'Test_2' (0x000800c00a08) 
<--

2) for the test without  mh.invoke() in main(),  the log shows:
[0.296s][info][class,init] Start class verification for: Test_1
[0.296s][info][class,init] End class verification for: Test_1
[0.297s][info][class,init] 282 Initializing 'Test_1' (0x000800c00800)
[0.297s][info][class,init] Start class verification for: Test_2
[0.297s][info][class,init] End class verification for: Test_2
(Test_2 was verified but didn't get initialized)

The comparison above literally surprised me that the bytecode verification 
happened prior to the class initialization, which means
the class got verified at first even without initialization coz I previously 
thought the initialization should trigger the verification rather than in the 
reversed order.

Could you explain a little more about why it goes in this way?

Best Regards
Cheng


-Original Message-
From: core-libs-dev  On Behalf Of David 
Holmes
Sent: March 17, 2022 7:46 PM
To: Raffaello Giulietti ; 
core-libs-dev@openjdk.java.net
Subject: [EXTERNAL] Re: When to initialize the method's class for 
MethodHandles.Lookup.findStatic()?

Run with -Xlog:class+init=info to see the classes that get initialized and in 
what order.

David

On 18/03/2022 5:53 am, Raffaello Giulietti wrote:

Hi again,

here's code that shows that initialization doesn't happen during
lookup but only upon invoking the method handle. (I'm on Java 17.)

As long as the 2nd line in main() is commented, you don't see the
message "Test_2 initialized", which shows that the lookup doesn't
initialize Test_2.
When you uncomment the line in main(), the message will appear.

So, as advertised, it's the invocation of the method handle that can
trigger initialization, not the lookup.


HTH
Raffaello



import java.lang.invoke.*;

public class Test_1 {

      static MethodHandle mh;

      static {
      try {
      mh = MethodHandles.lookup().findStatic(Test_2.class,
"testMethod", MethodType.methodType(int.class, int.class));
      } catch (NoSuchMethodException | IllegalAccessException e) {
      e.printStackTrace();
      }
      }

      public static void main(String[] args) throws Throwable {
      System.out.println(mh);
      // System.out.println(Test_1.mh.invoke(0));
      }

}




public class Test_2 {

      static {
      System.out.println("Test_2 initialized");
      }

      static int testMethod(int value) { return (value + 1); }

}




On 2022-03-17 20:38, Raffaello Giulietti wrote:

Hi,

as far as I can see, the code should not even compile, as there's a
static field in Test_1 which is initialized with an expression that
throws checked exceptions (findStatic(..)).

In addition, it seems to me that there's nothing in your code that
reveals whether Test_2 has been initialized during the lookup. How
can you tell?

Finally, the method handle invocation in Test_1 will throw, as you
don't pass any argument to a handle that expects one.

Can you perhaps add more details?


Greetings
Raffaello



On 2022-03-17 17:42, Cheng Jin wrote:

Hi there,

The document of
INVALID URI REMOVED
_en_java_javase_17_docs_api_java.base_java_lang_invoke_MethodHandles
.Lookup.html-23findStatic-28java.lang.Class-2Cjava.lang.String-2Cjav
a.lang.invoke.MethodType-29=DwIDaQ=jf_iaSHvJObTbx-siA1ZOg=X90f
3XIRXAH8hbNam6bIUlWfF_qUAezL9ue7M7bFuPQ=RvhguidNJ90V-HK-3Ctl-kUZE5
cIfo_nt3_r8VZ0Fcc=tw_ph6oUkS0eCvzITWi9zEkarss5yNeHDrAIfvd3s3g=
in the Java API is ambiguous in terms of when to initialize the
method's class as follows (the same description as in other OpenJDK
versions)

If the returned method handle is invoked, the method's class will be
initialized, if it has not already been initialized.


It occurs to me that the method's class should be initialized when
invoking the method handle but OpenJDK actually chooses to do the
initialization in lookup.findStatic() rather than in mh.invoke()
e.g.
import java.lang.invoke.*;

public class Test_1 {
  static MethodHandle mh =
MethodHandles.lookup().findStatic(Test_2.class, "testMethod",
MethodType.methodType(int.class, int.class)); <---
Test_2.class gets initialized and verified.

  public static void main(String[] args)

Re: When to initialize the method's class for MethodHandles.Lookup.findStatic()?

Sure, which again shows that Test_2 is not initialized by the lookup of 
the mh but only upon its invocation.


Raffaello


On 2022-03-18 00:46, David Holmes wrote:
Run with -Xlog:class+init=info to see the classes that get initialized 
and in what order.


David

On 18/03/2022 5:53 am, Raffaello Giulietti wrote:

Hi again,

here's code that shows that initialization doesn't happen during 
lookup but only upon invoking the method handle. (I'm on Java 17.)


As long as the 2nd line in main() is commented, you don't see the 
message "Test_2 initialized", which shows that the lookup doesn't 
initialize Test_2.

When you uncomment the line in main(), the message will appear.

So, as advertised, it's the invocation of the method handle that can 
trigger initialization, not the lookup.



HTH
Raffaello



import java.lang.invoke.*;

public class Test_1 {

 static MethodHandle mh;

 static {
 try {
 mh = MethodHandles.lookup().findStatic(Test_2.class, 
"testMethod", MethodType.methodType(int.class, int.class));

 } catch (NoSuchMethodException | IllegalAccessException e) {
 e.printStackTrace();
 }
 }

 public static void main(String[] args) throws Throwable {
 System.out.println(mh);
 // System.out.println(Test_1.mh.invoke(0));
 }

}




public class Test_2 {

 static {
 System.out.println("Test_2 initialized");
 }

 static int testMethod(int value) { return (value + 1); }

}




On 2022-03-17 20:38, Raffaello Giulietti wrote:

Hi,

as far as I can see, the code should not even compile, as there's a 
static field in Test_1 which is initialized with an expression that 
throws checked exceptions (findStatic(..)).


In addition, it seems to me that there's nothing in your code that 
reveals whether Test_2 has been initialized during the lookup. How 
can you tell?


Finally, the method handle invocation in Test_1 will throw, as you 
don't pass any argument to a handle that expects one.


Can you perhaps add more details?


Greetings
Raffaello



On 2022-03-17 17:42, Cheng Jin wrote:

Hi there,

The document of 
https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/invoke/MethodHandles.Lookup.html#findStatic(java.lang.Class,java.lang.String,java.lang.invoke.MethodType) 
in the Java API is ambiguous in terms of when to initialize the 
method's class as follows (the same description as in other OpenJDK 
versions)


If the returned method handle is invoked, the method's class will be 
initialized, if it has not already been initialized.



It occurs to me that the method's class should be initialized when 
invoking the method handle but OpenJDK actually chooses to do the 
initialization in lookup.findStatic() rather than in mh.invoke()

e.g.
import java.lang.invoke.*;

public class Test_1 {
 static MethodHandle mh = 
MethodHandles.lookup().findStatic(Test_2.class, "testMethod", 
MethodType.methodType(int.class, int.class)); <--- 
Test_2.class gets initialized and verified.


 public static void main(String[] args) throws Throwable {
 Test_1.mh.invoke();
 }
}

public class Test_2 {
 static int testMethod(int value) { return (value + 1); }
}

So there should be more clear explanation what is the correct or 
expected behaviour at this point and why OpenJDK doesn't comply with 
the document to delay the initialization of the method's class to 
mh.invoke().


Best Regards
Cheng Jin

Re: When to initialize the method's class for MethodHandles.Lookup.findStatic()?


Hi Cheng,

I'm not sure I understand your point. By class initialization I mean 
what's specified in the JVMS [1].


Are you concerned with initialization or with verification (which 
precedes initialization)?



Raffaello



[1] https://docs.oracle.com/javase/specs/jvms/se17/html/jvms-5.html#jvms-5.5



On 2022-03-18 00:02, Cheng Jin wrote:

Hi Raffaello,

The code snippet I posted previously was not the original test I verified on 
Java 17 (which was generated via asmtools to trigger verification)

Here's the pretty much the test I used:

Test_1.java
import java.lang.invoke.*;

public class Test_1 {

  static MethodHandle mh;

  static {
  try {
  mh = MethodHandles.lookup().findStatic(Test_2.class,
"testMethod", MethodType.methodType(int[].class));
  } catch (NoSuchMethodException | IllegalAccessException e) {
  e.printStackTrace();
  }
  }

  public static void main(String[] args) throws Throwable {
  //Test_1.mh.invoke();
  }

}

Test_2.jasm
super public class Test_2
version 52:0
{
   static Method "":"()V"
stack 5 locals 12
   {
bipush  -2;
 istore 7;
iload   7;
sipush  997;
if_icmplt   L127;
iconst_0;
istore  8;
L127:   stack_frame_type full;
locals_map int, class "[B", class "[B", class "[Ljava/lang/Class;", class "[I", 
class "[I", class "[I", int, bogus, bogus, float, float;
iload   8;
 istore 8;
 return;
   }
   public Method "":"()V"
stack 1 locals 1
   {
aload_0;
invokespecial   Method java/lang/Object."":"()V";
return;
   }

   static Method testMethod:"()[I"
stack 6 locals 1
   {
getstatic   Field testArray:"[I";
areturn;
   }

} // end Class Test_2

Jdk17/bin/java  -jar asmtools.jar jasm  Test_2.jasm   // create the .class file 
of Test_2
Jdk17/bin/java  Test_1
java.lang.IllegalAccessException: no such method: 
Test_2.testMethod()int[]/invokeStatic
 at 
java.base/java.lang.invoke.MemberName.makeAccessException(MemberName.java:972)
 at 
java.base/java.lang.invoke.MemberName$Factory.resolveOrFail(MemberName.java:1117)
 at 
java.base/java.lang.invoke.MethodHandles$Lookup.resolveOrFail(MethodHandles.java:3649)
 at 
java.base/java.lang.invoke.MethodHandles$Lookup.findStatic(MethodHandles.java:2588)
 at Test_1.(Test_1.java:9)
Caused by: java.lang.VerifyError: Inconsistent stackmap frames at branch target 
15
Exception Details:
   Location:
 Test_2.()V @15: iload
   Reason:
 Type top (current frame, locals[0]) is not assignable to integer (stack 
map, locals[0])
   Current Frame:
 bci: @9
 flags: { }
 locals: { top, top, top, top, top, top, top, integer }
 stack: { integer, integer }
   Stackmap Frame:
 bci: @15
 flags: { }
 locals: { integer, '[B', '[B', '[Ljava/lang/Class;', '[I', '[I', '[I', 
integer, top, top, float, float }
 stack: { }
   Bytecode:
 000: 10fe 3607 1507 1103 e5a1 0006 0336 0815
 010: 0836 08b1
   Stackmap Table:
 
full_frame(@15,{Integer,Object[#11],Object[#11],Object[#20],Object[#5],Object[#5],Object[#5],Integer,Top,Top,Float,Float},{})

 at java.base/java.lang.invoke.MethodHandleNatives.resolve(Native 
Method)
 at 
java.base/java.lang.invoke.MemberName$Factory.resolve(MemberName.java:1085)
 at 
java.base/java.lang.invoke.MemberName$Factory.resolveOrFail(MemberName.java:1114)
 ... 3 more

Test_2 was intentionally modified to throw VerifyError if initialized.
As you see above,  mh.inovke() was commented out in Test_1.main() but the 
Test_2 was still verified in the test.
So it is impossible to verify Test_2 if it is not initialized, which only means 
Test_2 is initialized during the lookup.findstatic rather than mh.invoke().

Best Regards
Cheng

- Original Message -

From: "Cheng Jin" 
To: "core-libs-dev" 
Sent: Thursday, March 17, 2022 5:42:57 PM
Subject: When to initialize the method's class for 
MethodHandles.Lookup.findStatic()?



Hi there,

The document of
INVALID URI REMOVED
n_java_javase_17_docs_api_java.base_java_lang_invoke_MethodHandles.Loo
kup.html-23findStatic-28java.lang.Class-2Cjava.lang.String-2Cjava.lang
.invoke.MethodType-29=DwIFaQ=jf_iaSHvJObTbx-siA1ZOg=X90f3XIRXAH8
hbNam6bIUlWfF_qUAezL9ue7M7bFuPQ=Xt-10pHYoPWnY6dByKowR4yeLtEs7kZkKUgt
bxKUGvM=dPAMGMxphLLXT9N4ZdukiNvWyvRPAGkcGCBLTy_sm0c=
in the Java API is ambiguous in terms of when to initialize the
method's class as follows (the same description as in other OpenJDK
versions)

If the returned method handle is invoked, the method's class will be
initialized, if it has not already been initialized.


It occurs to me that the method's class should be initialized when
invoking the method handle but

Re: When to initialize the method's class for MethodHandles.Lookup.findStatic()?


Hi again,

here's code that shows that initialization doesn't happen during lookup 
but only upon invoking the method handle. (I'm on Java 17.)


As long as the 2nd line in main() is commented, you don't see the 
message "Test_2 initialized", which shows that the lookup doesn't 
initialize Test_2.

When you uncomment the line in main(), the message will appear.

So, as advertised, it's the invocation of the method handle that can 
trigger initialization, not the lookup.



HTH
Raffaello



import java.lang.invoke.*;

public class Test_1 {

static MethodHandle mh;

static {
try {
mh = MethodHandles.lookup().findStatic(Test_2.class, 
"testMethod", MethodType.methodType(int.class, int.class));

} catch (NoSuchMethodException | IllegalAccessException e) {
e.printStackTrace();
}
}

public static void main(String[] args) throws Throwable {
System.out.println(mh);
// System.out.println(Test_1.mh.invoke(0));
}

}




public class Test_2 {

static {
System.out.println("Test_2 initialized");
}

static int testMethod(int value) { return (value + 1); }

}




On 2022-03-17 20:38, Raffaello Giulietti wrote:

Hi,

as far as I can see, the code should not even compile, as there's a 
static field in Test_1 which is initialized with an expression that 
throws checked exceptions (findStatic(..)).


In addition, it seems to me that there's nothing in your code that 
reveals whether Test_2 has been initialized during the lookup. How can 
you tell?


Finally, the method handle invocation in Test_1 will throw, as you don't 
pass any argument to a handle that expects one.


Can you perhaps add more details?


Greetings
Raffaello



On 2022-03-17 17:42, Cheng Jin wrote:

Hi there,

The document of  
https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/invoke/MethodHandles.Lookup.html#findStatic(java.lang.Class,java.lang.String,java.lang.invoke.MethodType) 
in the Java API is ambiguous in terms of when to initialize the 
method's class as follows (the same description as in other OpenJDK 
versions)


If the returned method handle is invoked, the method's class will be 
initialized, if it has not already been initialized.



It occurs to me that the method's class should be initialized when 
invoking the method handle but OpenJDK actually chooses to do the 
initialization in lookup.findStatic() rather than in mh.invoke()

e.g.
import java.lang.invoke.*;

public class Test_1 {
 static MethodHandle mh = 
MethodHandles.lookup().findStatic(Test_2.class, "testMethod", 
MethodType.methodType(int.class, int.class)); <--- 
Test_2.class gets initialized and verified.


 public static void main(String[] args) throws Throwable {
 Test_1.mh.invoke();
 }
}

public class Test_2 {
 static int testMethod(int value) { return (value + 1); }
}

So there should be more clear explanation what is the correct or 
expected behaviour at this point and why OpenJDK doesn't comply with 
the document to delay the initialization of the method's class to 
mh.invoke().


Best Regards
Cheng Jin

Re: When to initialize the method's class for MethodHandles.Lookup.findStatic()?


Hi,

as far as I can see, the code should not even compile, as there's a 
static field in Test_1 which is initialized with an expression that 
throws checked exceptions (findStatic(..)).


In addition, it seems to me that there's nothing in your code that 
reveals whether Test_2 has been initialized during the lookup. How can 
you tell?


Finally, the method handle invocation in Test_1 will throw, as you don't 
pass any argument to a handle that expects one.


Can you perhaps add more details?


Greetings
Raffaello



On 2022-03-17 17:42, Cheng Jin wrote:

Hi there,

The document of  
https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/invoke/MethodHandles.Lookup.html#findStatic(java.lang.Class,java.lang.String,java.lang.invoke.MethodType)
 in the Java API is ambiguous in terms of when to initialize the method's class 
as follows (the same description as in other OpenJDK versions)

If the returned method handle is invoked, the method's class will be 
initialized, if it has not already been initialized.


It occurs to me that the method's class should be initialized when invoking the 
method handle but OpenJDK actually chooses to do the initialization in 
lookup.findStatic() rather than in mh.invoke()
e.g.
import java.lang.invoke.*;

public class Test_1 {
 static MethodHandle mh = MethodHandles.lookup().findStatic(Test_2.class, 
"testMethod", MethodType.methodType(int.class, int.class)); <--- 
Test_2.class gets initialized and verified.

 public static void main(String[] args) throws Throwable {
 Test_1.mh.invoke();
 }
}

public class Test_2 {
 static int testMethod(int value) { return (value + 1); }
}

So there should be more clear explanation what is the correct or expected 
behaviour at this point and why OpenJDK doesn't comply with the document to 
delay the initialization of the method's class to mh.invoke().

Best Regards
Cheng Jin

Re: Making enum hashcodes consistent across JVM's


Hi,

specifying a fixed hashCode() algorithm has the important drawback that 
it prevents evolution.


For example, as you note, String has a strictly specified algorithm [1] 
that was perhaps a good one in the early days of Java (around 1995) but 
has shown its deficiencies over the years. In particular, it's easy to 
construct collisions, which might impact the performance of popular and 
heavily used hash based data structures, like HashMap and HashSet, under 
some forms of cyberattack.


Unfortunately, while better String hash algorithms have been proposed, 
they cannot be adopted, as too much code out there relies on the 
specific details of String::hashCode. (Even bytecode for switches over 
String relies on that particular spec.) It's far too late now to change 
the spec, even for the better.


In retrospect and IMO, fixing String::hashCode in the spec was a 
(understandable) mistake.



More generally, client code that relies on details of any popular 
library hashCode() implementation has a dangerous dependency on such 
details and impairs evolution of the library itself (as the example with 
String shows).


To leave the door open for better implementations proposed in some 
future, it is best to avoid specifying the details of hashCode(), as 
done in Record::hashCode [2] ("Implementation Requirements"). After all, 
the general contract reads [3]: "This integer [the hash] need not remain 
consistent from one execution of an application to another execution of 
the same application."



But what is the use case you have for which you would like a firmly 
specified Enum::hashCode?



Greetings
Raffaello



[1] 
https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/String.html#hashCode()
[2] 
https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Record.html#hashCode()
[3] 
https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Object.html#hashCode()




On 2022-03-17 13:49, dfranken@gmail.com wrote:

Dear all,

Currently enums do not have a well-defined hashCode() implementation so
they defer to Object.hashCode() which just uses an internal mechanism
to determine the hashcode, likely based on the object's place in the
heap.

This may confuse a lot of developers as other classes such as String do
have a well-defined hashcode which is consistent across multiple JVM's.

Could it be a good idea to make enums' hashCode() method more reliably
and predictable? For instance by using the hashcode of the name of its
value?

For example:

class MyEnum { A, B }
-> A.hashCode() == A.name().hashCode()

Kind regards,

Dave Franken

Re: Java floating point and StrictMath improvement?

2022-03-16 Thread Raffaello Giulietti


"Computer scientists of the world, unite!

Let's get rid of binary floating-point arithmetic in Java, after 27 
years of honorable service!


Let's adopt decimal floating-point arithmetic, where 1 / 3 + 1 / 3 is 
still different from 2 / 3, but who cares? At least we have 0.1 * 0.1 = 
0.01, as Mathematics commands!"



Just kidding...
More seriously, let's stop this useless discussion.


Raffaello




On 2022-03-16 06:47, A Z wrote:

To core-libs-dev@openjdk.java.net,


float and double should be mutually enhanced or

defaultedly changed, to uphold the following three

properties:


1) Base ten arithmetic on float and double, via operators.

2) Base Ten elementary function calls, like

those made on java.lang.StrictMath,

on double or float values.

3) Base ten Comparison operators. ==, !=, >, <, >=,<=.



Martin has said:


'It is inherently, mathematically *impossible* to emulate decimal

behaviour with binary IEEE 754 arithmetic. That's why Java also

offers java.math.BigDecimal, a decimal floating-point library

implemented in software.'


Then don't use IEEE 754 alone! Or, have a dual mode, with an

active alternative. There are binary alternatives, particularly

that include the likes of SSE, that in fact can.


The only standards necessary, and really, the 'authoritative standard',

are those prevalent in mathematics itself. Base 10 arithmetic, algebra,

elementary functions, range limited real values.


BigDecimal, BigInteger, and https://github.com/eobermuhlner big-math,

with its calculator class, take up more room in memory than necessary.

They are slower, and they do not allow for the use of operators.

Worse yet, they are not immediately reified in the rest of the language,

ie all class and interface libraries.


Raffaello has said:

'Exact representation of 0.1 using base 2 is mathematically impossible,

no matter the language (it is a periodic number in base 2).'


Consideration of the article:

https://people.eecs.berkeley.edu/~wkahan/JAVAhurt.pdf


Which is admittedly older, but still accurate, indicates otherwise.


I contend that within an upper and lower range, that it is possible.

While C++ can't use the immediate ==, !=, >,<,>=,<= operators

immediately accurately, Java has.


Using 32 or 64 bit buffers, and an SSE or similar buffer, and the

right algorithms, unto limited range, you can have as much of

a range representation as required for float or double, even

though you ultimately don't have 'exact' representation.


Bernd Eckenfels has said:

'The need to round floating point numbers for commercial math

(and the risk involved in doing so) is nothing new, it predates

the IEEE standard and should be subject for even basic comp

sci curriculums all over the world.'


That is true, but it is the case that range termination

of decimal values, by means of truncation, is more accurate

than this. By any processing of rounding, you start gaining

inaccuracy, which can grow and grow and overpopulate the

entire number. In Computer Programming, you can need range

continuous range accuracy. Such as with 2D and 3D processing.

The present Java approach is slower and a waste of memory.


Given all this, and given that humans, and representations

of the real world rely on base 10 first, shouldn't the 3

properties I outlined at the beginning of my post be

at least mutually, compatibly, instituted in Java,

if not defaultedly?

Re: Discussion about Java Floating Point?

2022-03-15 Thread Raffaello Giulietti


Sorry, for some reason a line was missing in the C++ code


#include 

using namespace std;

int main()
{
cout << "Program has started..." << endl;
double a = 0.1D;
double b = 0.1D;
double c = a*b;
double d = 0.01D;
cout << endl << "0.1D*0.1D " << (c == d ? "==" : "!=") << " 0.01D" 
<< endl << endl;

cout << "0.1D*0.1D : ";
cout.precision(30);
cout << c << endl;
cout << "0.01D : ";
cout.precision(30);
cout << d << endl << endl;
cout << "Program has finished." << endl;
return 0;
}



On 3/15/22 11:03, Raffaello Giulietti wrote:

Hi,

Java, as well as most implementations of most languages, including C and 
C++, have adopted the IEEE 754 floating-point standard, first issued in 
1985. The standard specifies the formats, the outcome of operators and 
comparisons up to the last bit.


Try out this C/C++ code (I'm on Linux with gcc)



#include 

using namespace std;

int main()
{
     cout << "Program has started..." << endl;
     double a = 0.1D;
     double b = 0.1D;
     double c = a*b;
     double d = 0.01D;
     cout << endl << "0.1D*0.1D " << (c == d ? "==" : "!=") << " 0.01D" 
<< endl << endl;

     cout.precision(30);
     cout << c << endl;
     cout << "0.01D : ";
     cout.precision(30);
     cout << d << endl << endl;
     cout << "Program has finished." << endl;
     return 0;
}



This is the output I see:



Program has started...

0.1D*0.1D != 0.01D

0.1D*0.1D : 0.010001942890293094
0.01D : 0.012081668171172

Program has finished.



So 0.1D*0.1D != 0.01D, as stated on my previous post. That is, 
expectations on decimal behavior are usually wrong. It is inherently, 
mathematically *impossible* to emulate decimal behavior with binary IEEE 
754 arithmetic. That's why Java also offers java.math.BigDecimal, a 
decimal floating-point library implemented in software.


Also, as suggested in that post, incrementing the precision of the 
output by means of precision(30) as done above, for example, also 
clearly shows where the two values start being different, with 0.1D*0.1D 
being slightly greater than 0.01D.


Note that the precision only directs how many digits appear in the 
decimal output: it doesn't affect the underlying floating-point 
arithmetic at all. The latter follows the spec in full. In fact, all 
operators and comparisons are implemented directly in hardware, not in 
libraries. All CPUs designed in the last couple of decades implement 
IEEE 754.


This shows that the default precision for decimal output in C/C++ is not 
sufficient to expose the difference between the values. You have to set 
a higher precision to see differences.


This also shows the advantage of the default Java conversion routine, as 
opposed to the default precision in C/C++: different values are 
converted to different decimal outcome, without you having to set 
anything, because Java adapts the precision of the output automatically, 
depending on the value.


Finally, while Wikipedia is an exceptional source of information, the 
ultimate authoritative reference are the IEEE 754 spec, the C/C++ specs 
and other languages specs, and their related standard libraries specs.



Greetings
Raffaello


On 3/15/22 05:23, A Z wrote:

To core-libs-dev@openjdk.java.net,

In terms of floating point, it seems there are thus
three (3) phenomena that are relevant.

1) Arithmetic on float and double, via operators.
2) Elementary function calls, namely
    those made from java.lang.StrictMath, as it is,
    on double values.
3) Comparison operators.  ==, !=, >, <, >=,<=.

Java floating point successfully has 3) the way required,
even though other languages, particularly C++ , do not
compare their floats and doubles through those comparison
operators.  The point at issue is Java although, so Java is
satisfactory at point 3).

My point of contention is that Java does not have 1) and 2)
operating as they should, even though C++ and other
languages do, in those two areas, namely C++:

Martin has submitted the following:
?The following statement is not entirely true when using finite floating
point precision:


(?snip?) It is a mathematical fact that, for
consistent, necessary and even fast term, 10% of 10% must
always precisely be 1%, and by no means anything else."

/*
#include 

using namespace std;

int main()
{
 cout << "Program has started..." << endl;
 double a = 0.1D;
 double b = 0.1D;
 double c = a*b;
 cout << endl << c << endl << endl;
 float d = 0.1F;
 float e = 0.1F;
 float f = d*e;

Re: RFR: JDK-8283143: Use minimal-length literals to initialize PI and E constants

2022-03-15 Thread Raffaello Giulietti

The new decimal literals have indeed the minimal length required to 
recover the doubles closest to the true mathematical values.


Raffaello



On 3/15/22 02:43, Joe Darcy wrote:

Depending on the range of the number line, a double value has between 15 and 17 
digits of decimal precision. The literals used to initialize Math.PI and Math.E 
have several digits more precision than that maximum.

That is potentially confusing to readers of the code and the minimum length 
strings to exactly represent the value in question should be used instead.

-

Commit messages:
  - JDK-8283143: Use minimal-length literals to initialize PI and E constants

Changes: https://git.openjdk.java.net/jdk/pull/7814/files
  Webrev: https://webrevs.openjdk.java.net/?repo=jdk=7814=00
   Issue: https://bugs.openjdk.java.net/browse/JDK-8283143
   Stats: 4 lines in 2 files changed: 0 ins; 0 del; 4 mod
   Patch: https://git.openjdk.java.net/jdk/pull/7814.diff
   Fetch: git fetch https://git.openjdk.java.net/jdk pull/7814/head:pull/7814

PR: https://git.openjdk.java.net/jdk/pull/7814

Re: Discussion about Java Floating Point?

2022-03-15 Thread Raffaello Giulietti


Hi,

Java, as well as most implementations of most languages, including C and 
C++, have adopted the IEEE 754 floating-point standard, first issued in 
1985. The standard specifies the formats, the outcome of operators and 
comparisons up to the last bit.


Try out this C/C++ code (I'm on Linux with gcc)



#include 

using namespace std;

int main()
{
cout << "Program has started..." << endl;
double a = 0.1D;
double b = 0.1D;
double c = a*b;
double d = 0.01D;
cout << endl << "0.1D*0.1D " << (c == d ? "==" : "!=") << " 0.01D" 
<< endl << endl;

cout.precision(30);
cout << c << endl;
cout << "0.01D : ";
cout.precision(30);
cout << d << endl << endl;
cout << "Program has finished." << endl;
return 0;
}



This is the output I see:



Program has started...

0.1D*0.1D != 0.01D

0.1D*0.1D : 0.010001942890293094
0.01D : 0.012081668171172

Program has finished.



So 0.1D*0.1D != 0.01D, as stated on my previous post. That is, 
expectations on decimal behavior are usually wrong. It is inherently, 
mathematically *impossible* to emulate decimal behavior with binary IEEE 
754 arithmetic. That's why Java also offers java.math.BigDecimal, a 
decimal floating-point library implemented in software.


Also, as suggested in that post, incrementing the precision of the 
output by means of precision(30) as done above, for example, also 
clearly shows where the two values start being different, with 0.1D*0.1D 
being slightly greater than 0.01D.


Note that the precision only directs how many digits appear in the 
decimal output: it doesn't affect the underlying floating-point 
arithmetic at all. The latter follows the spec in full. In fact, all 
operators and comparisons are implemented directly in hardware, not in 
libraries. All CPUs designed in the last couple of decades implement 
IEEE 754.


This shows that the default precision for decimal output in C/C++ is not 
sufficient to expose the difference between the values. You have to set 
a higher precision to see differences.


This also shows the advantage of the default Java conversion routine, as 
opposed to the default precision in C/C++: different values are 
converted to different decimal outcome, without you having to set 
anything, because Java adapts the precision of the output automatically, 
depending on the value.


Finally, while Wikipedia is an exceptional source of information, the 
ultimate authoritative reference are the IEEE 754 spec, the C/C++ specs 
and other languages specs, and their related standard libraries specs.



Greetings
Raffaello


On 3/15/22 05:23, A Z wrote:

To core-libs-dev@openjdk.java.net,

In terms of floating point, it seems there are thus
three (3) phenomena that are relevant.

1) Arithmetic on float and double, via operators.
2) Elementary function calls, namely
those made from java.lang.StrictMath, as it is,
on double values.
3) Comparison operators.  ==, !=, >, <, >=,<=.

Java floating point successfully has 3) the way required,
even though other languages, particularly C++ , do not
compare their floats and doubles through those comparison
operators.  The point at issue is Java although, so Java is
satisfactory at point 3).

My point of contention is that Java does not have 1) and 2)
operating as they should, even though C++ and other
languages do, in those two areas, namely C++:

Martin has submitted the following:
?The following statement is not entirely true when using finite floating
point precision:


(?snip?) It is a mathematical fact that, for
consistent, necessary and even fast term, 10% of 10% must
always precisely be 1%, and by no means anything else."

/*
#include 

using namespace std;

int main()
{
 cout << "Program has started..." << endl;
 double a = 0.1D;
 double b = 0.1D;
 double c = a*b;
 cout << endl << c << endl << endl;
 float d = 0.1F;
 float e = 0.1F;
 float f = d*e;
 cout << f << endl << endl;
 cout << "Program has Finished.";
 return 0;
}
*/
/*
Program has started...

0.01

0.01

Program has Finished.
*/

This is actually not fully or finally true, including SSE, SSE algorithm logic 
and further.
The included C++ example of mine above here disproves this.  Even though I do
contextually admit these answers can only work within the range of the
involved types, it is still actually the case that 10% of 10% can and always
precisely be 1%, within the range of the type used, and by no means
anything else.  Because of the laws of decimal arithmetic.

See https://en.wikipedia.org/wiki/Floating-point_arithmetic
and the starting sentence:

'In computing, floating-point arithmetic (FP) is arithmetic using formulaic
representation of real numbers as an approximation to support a
trade-off betwee range and precision.'

However, it simply isn't the case that reduced precision has to exist inside
the range, certainly not any more.  Rationally, one would have to think,
with the

Re: RFR: JDK-8283124: Add constant for tau to Math and StrictMath

Right, and PI with 16 digits (or 17).

On 3/14/22 22:51, Hans Boehm wrote:
Couldn't the apiNote just say TAU == 2 * PI instead? I think the fact 
that this is actually a guaranteed floating point equality aids clarity.

On Mon, Mar 14, 2022 at 2:33 PM Raffaello Giulietti 
mailto:raffaello.giulie...@gmail.com>> 
wrote:

Hello,

I find it a bit disturbing that PI is specified with 21 digits whereas
TAU has 16.
I think that specifying PI as
      public static final double PI = 3.141592653589793;
doesn't harm anybody and makes it visually more consistent with TAU-

Greetings
Raffaello

On 3/14/22 22:13, Brian Burkhalter wrote:
 > On Mon, 14 Mar 2022 20:52:39 GMT, Joe Darcy mailto:da...@openjdk.org>> wrote:
 >
 >> Add a constant for tau, 2*pi, to Math and StrictMath. Since 2*pi
is a very common value in mathematical formulas, it is helpful to
give it a distinct constant.
 >>
 >> Please also review the CSR
https://bugs.openjdk.java.net/browse/JDK-8283136
<https://bugs.openjdk.java.net/browse/JDK-8283136>
 >
 > Marked as reviewed by bpb (Reviewer).
 >
 > -
 >
 > PR: https://git.openjdk.java.net/jdk/pull/7813
<https://git.openjdk.java.net/jdk/pull/7813>

Re: RFR: JDK-8283124: Add constant for tau to Math and StrictMath


Hello,

I find it a bit disturbing that PI is specified with 21 digits whereas 
TAU has 16.

I think that specifying PI as
public static final double PI = 3.141592653589793;
doesn't harm anybody and makes it visually more consistent with TAU-


Greetings
Raffaello



On 3/14/22 22:13, Brian Burkhalter wrote:

On Mon, 14 Mar 2022 20:52:39 GMT, Joe Darcy  wrote:


Add a constant for tau, 2*pi, to Math and StrictMath. Since 2*pi is a very 
common value in mathematical formulas, it is helpful to give it a distinct 
constant.

Please also review the CSR https://bugs.openjdk.java.net/browse/JDK-8283136


Marked as reviewed by bpb (Reviewer).

-

PR: https://git.openjdk.java.net/jdk/pull/7813

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v6]

On Tue, 8 Feb 2022 22:11:34 GMT, Raffaello Giulietti  
wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request with a new target base due 
> to a merge or a rebase. The pull request now contains 12 commits:
> 
>  - 4511638: Double.toString(double) sometimes produces incorrect results
>
>Adapted hashes in ElementStructureTest.java
>  - 4511638: Double.toString(double) sometimes produces incorrect results
>
>Merge branch 'master' into JDK-4511638
>  - 4511638: Double.toString(double) sometimes produces incorrect results
>
>Enhanced intervals in MathUtils.
>Updated references to Schubfach v4.
>  - 4511638: Double.toString(double) sometimes produces incorrect results
>
>Merge branch 'master' into JDK-4511638
>  - 4511638: Double.toString(double) sometimes produces incorrect results
>
>Slight adjustments to Javadoc as suggested in the JDK-8202555 (CSR) 
> comments.
>  - 4511638: Double.toString(double) sometimes produces incorrect results
>
>Adjusted hashes in test/langtools/tools/javac/sym/ElementStructureTest.java
>  - 4511638: Double.toString(double) sometimes produces incorrect results
>
>Merge branch 'master' into JDK-4511638
>  - 4511638: Double.toString(double) sometimes produces incorrect results
>  - 4511638: Double.toString(double) sometimes produces incorrect results
>
>Refactored test classes to better match OpenJDK conventions.
>Added tests recommended by Guy Steele and Paul Zimmermann.
>  - Changed MAX_CHARS to static
>  - ... and 2 more: 
> https://git.openjdk.java.net/jdk/compare/92f4f40d...c29dff76

Hi Suminda,

before merging, an OpenJDK PR needs to be properly reviewed by 
officially nominated reviewers other than the author of a PR.

In addition, this PR also needs a CSR approval because the proposed spec 
has changed.

As you can see on the top of the PR, neither of these two prerequisites 
for merging are checked as of today.


Greetings
Raffaello

On 3/10/22 07:37, Suminda Sirinath Salpitikorala Dharmasena wrote:
>
> Why is this still not merged?
>
> —
> Reply to this email directly, view it on GitHub 
> <https://github.com/openjdk/jdk/pull/3402#issuecomment-1063713753>, or 
> unsubscribe 
> <https://github.com/notifications/unsubscribe-auth/AQ3TDG2HH4HZA4AFVFNFEYLU7GKBPANCNFSM42TWJIOA>.
> Triage notifications on the go with GitHub Mobile for iOS 
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email=8=524675>
>  
> or Android 
> <https://play.google.com/store/apps/details?id=com.github.android=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
>  
>
> You are receiving this because you were mentioned.Message ID: 
> ***@***.***>
>

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: Questions about enhancement and Correction to Java OpenJDK Floating Point?

Hi Terry,

as pointed out by Martin, the real issue is using *binary*
floating-point arithmetic, like float or double, to emulate *decimal*
arithmetic.

When you write 0.1D in Java, C or C++, what happens is that this decimal
number is rounded to the double closest to the mathematical value 1/10.
There's no double that is exactly 1/10, so you start with a value that
is already rounded and inexact. Multiplication rounds as well, so you
end up with a value that was subject to 3 roundings: twice for the
conversion of the two openads from 0.1D to the closest doubles and once
for the multiplication. The result is slightly different than the
naively "expected" 0.01D, which is subject to one rounding only during
conversion to the closest double. In other words, 0.1D*0.1D != 0.01D,
even in C/C++ and most programming languages/environments.

In Java, however, when you convert a double to a decimal string by means
of System.out.print[ln](), the library outputs just as many digits as
necessary, and no less, for an input routine to be able to recover the
original double. C and C++ do *not* ensure this. In Java, 0.01D (1
rounding) is correctly converted to "0.01" while 0.1D*0.1D (3 roundings)
is correctly converted to "0.010002".

In C/C++, try to output both 0.1D*0.1D and 0.01D with 20 digits, say,
instead of the default 6 and you'll see a difference.

As observed by Rémi, Java offers formatting similar to C/C++ if that is
what you want.

To summarize, Java uses IEEE 754 binary arithmetic as by specification,
as do most other languages, including C/C++. It is however fundamentally
wrong to use binary floating-point arithmetic to emulate decimal
behavior. Also, pay attention to the output routines that convert float
and double values to a decimal representation. Usually, C and C++ will
have information loss by default, as in your case.

HTH
Raffaello

On 3/14/22 07:49, A Z wrote:

To whom it may concern,

Having noticed

https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8190947
https://bugs.openjdk.java.net/browse/JDK-8190991

and similar, at
https://community.oracle.com/tech/developers/discussion/4126262/big-issue-with-float-double-java-floating-point

I have been referred on to the core-libs-dev area.

The software development company I represent wishes to keep its name
confidential,
and no-mentioned, at this time.

A number of us at our end have found that floating point and StrictMath
arithmetic
on both float and double does not result in range accuracy, but produces
denormal
and pronormal values.

We are aware of the Java Language Specification, and IEEE 754 specifications,
to these issues, but are still finding that they are not the most relevant or
great issue.

While we are aware of the BigDecimal and BigInteger workarounds, and
furthermore, the calculator class including big-math
https://github.com/eobermuhlner,
we are finding in the development, debugging, and editing of our Java programs,
that using other classes to operate and exchange for the lack of range accuracy
from float,
double and java.lang.StrictMath, that we are getting bogged down with what
turns into
more inferior software. The known and available workaround approaches are
becoming
stop-gap measures, perforcedly put in place, while introducing other problems
into OpenJDK or Java software that don't have any particular, immediate,
solutions.

Substituting float and double data in and out of BigDecimal and BigInteger
produces
source code which is much messier, complicated, error prone, difficult to
understand
and to change, is definitely slower, and is an inferior substitute when float
and double are more than enough in the overwhelming majority of corresponding
cases.
This is particularly showing up in 2D and 3D Graphics software, by the default
OpenJDK Libraries, but also through JMonkeyEngine 3.5.

Possessing the option to immediately deal with the precondition, postcondition
and field types of float and double is far superior and more than ideal.
All this is before the massive advantage of being able to use operators,
but the change case becomes overwhelming when along a range accurate,
double (or maybe float, also) supporting Scientific Calculator class.

If I want to discuss (at least OpenJDK) change in this area, I have
been pointed to the core-libs area, by one of the respondents
of the article:

https://community.oracle.com/tech/developers/discussion/4126262/big-issue-with-float-double-java-floating-point.

Is there anyone here, at core-libs-dev, who can point
me in a better Oracle or OpenJDK direction, to discuss further
and see about Java float and double and StrictMath floating point
arithmetic denormal and pronormal values being repaired away
and being made range accurate for all evaluation operations
with them?

Certainly since other languages already have, that are open source
and open resource file ones. It is a mathematical fact that, for
consistent, necessary and

Re: Proposed JEP: Safer Process Launch by ProcessBuilder and Runtime.exec

2022-02-24 Thread Raffaello Giulietti


Hi,

on Windows, the mechanism to launch a new program is for the parent to 
call CreateProcess(). This function accepts, among others parameters, a 
lpApplicationName string and a lpCommandLine string. There's a *single* 
lpCommandLine that encodes all the arguments to pass to the new program.


The exact encoding of the arguments in the command line is imposed by 
how the *new* program parses (decodes) the command line to get its 
constituent parts. There are no fixed rules on how this happens. There 
are some more or less well documented rules for some runtimes (e.g., the 
C/C++ runtime) or for some specific programs (e.g., cmd.exe., 
wscript.exe). In general, however, a program can decode in any way it 
deems useful.


Because the encoding is dictated by the target program's decoding, and 
because the latter is really arbitrary, there's no safe, universal 
procedure to encode a list of string arguments to a single command line 
string. It is only when the decoding rules in the target program are 
known that encoding in the parent becomes feasible.


Thus, it might be more useful on Windows platforms to avoid the API 
points that expose List or String[] for the arguments to the 
target program and use the ones that accept a single String, instead. 
The client of those API points would then have to deal with the encoding 
specific for that program. This is a better match with the underlying OS 
mechanism, namely CreateProcess(), which accepts a single, already 
encoded string.


In addition, to assist programmers unfamiliar with specific encodings, 
widely used specific encoders (e.g., for the C/C++ runtime [1], for 
cmd.exe, etc.) can be implemented separately.



Greetings
Raffaello



[1] 
https://mail.openjdk.java.net/pipermail/core-libs-dev/2022-February/086105.html

Re: RFR: 8282008: Incorrect handling of quoted arguments in ProcessBuilder [v3]

2022-02-24 Thread Raffaello Giulietti


Hi,

as far as I know, on Windows every program can obtain the lpCommandLine 
argument, used in the call of CreateProcess() from its parent, by 
calling GetCommandLine() and parse that string as it sees fit. This is 
in stark contrast with how Unix-like systems create and execute 
programs, where the system call execve(2) accepts an array of arguments, 
not a single command line.


There are no fixed rules on how to parse the command line, as witnessed 
by the different strategies implemented in the C/C++ runtime (which 
splits the command line according to the rules outlined in [1] to 
populate the argv[] array in main()), the cmd.exe shell, the wscript.exe 
runtime, etc.


Consequently, there are no fixed rules on how to encode a command line 
(specifically, the lpCommandLine argument to CreateProcess()) because it 
really is up to the invoked program to parse it, whether explicitly or 
implicitly, according to its own, directly or indirectly implemented 
rules. Even a C/C++ console program could ignore the result that the 
runtime automatically provides in argv[] and parse the command line 
directly, as obtained by GetCommandLine().


Without knowing the parsing rules of the target program, it is not 
possible to encode a command line correctly for CreateProcess(). I doubt 
there's a "common denominator" which would cover most cases encountered 
in practice.


The best we can hope is to implement encoders (and decoders) for 
specific, widely used runtimes. (BTW, for the C/C++ runtime I prepared 
an implementation mentioned in [2].)



Greetings
Raffaello



[1] 
https://docs.microsoft.com/en-us/cpp/c-language/parsing-c-command-line-arguments
[2] 
https://mail.openjdk.java.net/pipermail/core-libs-dev/2022-February/086105.html




On 2022-02-24 20:18, Olga Mikhaltsova wrote:

On Fri, 18 Feb 2022 17:21:48 GMT, Olga Mikhaltsova  
wrote:


This fix made equal processing of strings such as ""C:\\Program Files\\Git\\"" 
before and after JDK-8250568.

For example, it's needed to execute the following command on Windows:
`C:\Windows\SysWOW64\WScript.exe "MyVB.vbs" "C:\Program Files\Git" "Test"`
it's equal to:
`new ProcessBuilder("C:\\Windows\\SysWOW64\\WScript.exe", "MyVB.vbs", ""C:\\Program 
Files\\Git\\"", "Test").start();`

While processing, the 3rd argument ""C:\\Program Files\\Git\\"" treated as 
unquoted due to the condition added in JDK-8250568.

 private static String unQuote(String str) {
 ..
if (str.endsWith("\\"")) {
 return str;// not properly quoted, treat as unquoted
 }
 ..
 }


that leads to the additional surrounding by quotes in 
ProcessImpl::createCommandLine(..) because needsEscaping(..) returns true due 
to the space inside the string argument.
As a result the native function CreateProcessW 
(src/java.base/windows/native/libjava/ProcessImpl_md.c) gets the incorrectly 
quoted argument:

pcmd = C:\Windows\SysWOW64\WScript.exe MyVB.vbs ""C:\Program Files\Git"" Test
(jdk.lang.Process.allowAmbiguousCommands = true)
pcmd = "C:\Windows\SysWOW64\WScript.exe" MyVB.vbs ""C:\Program Files\Git\\"" 
Test
(jdk.lang.Process.allowAmbiguousCommands = false)


Obviously, a string ending with `"\\""` must not be started with `"""` to treat 
as unquoted overwise it’s should be treated as properly quoted.


Olga Mikhaltsova has updated the pull request incrementally with one additional 
commit since the last revision:

   Add a new line to the end of test file for JDK-8282008


Roger, writing a test via echo was not a good idea obviously for this particular case 
because of the fact well shown in the doc "4. Everyone Parses Differently", 
https://daviddeley.com/autohotkey/parameters/parameters.htm#WINCRULESMSEX. The task is 
more complicated than it seems at the first glance. The same command line correctly 
parsed in an app written in C/C++ might be incorrectly parsed in a VBS app etc.

The suggestion not to use the path-argument surroundings with `'"'` doesn't fix the issue 
in case of VBS. It leads to a resulting path-argument ending with the doubled backslash in a 
VBS app according to the rules "10.1 The WSH Command Line Parameter Parsing Rules", 
https://daviddeley.com/autohotkey/parameters/parameters.htm#WSH.

Below there are some experiments with an app attached to JDK-8282008:

NO FIX

1. new ProcessBuilder("C:\\Windows\\SysWOW64\\WScript.exe", "MyVB.vbs", ""C:\\Program 
Files\\Git\\"", "Test").start();

1.1 allowAmbiguousCommands = false
arg[0] = \C:\Program
arg[1] = Files\Git1\\\
CreateProcessW: pcmd = "C:\Windows\SysWOW64\WScript.exe" MyVB.vbs ""C:\Program 
Files\Git1\\"" Test

1.2 allowAmbiguousCommands = true
arg[0] = C:\Program
arg[1] = Files\Git1\
CreateProcessW: pcmd = C:\Windows\SysWOW64\WScript.exe MyVB.vbs ""C:\Program 
Files\Git1"" Test

2. new ProcessBuilder("C:\\Windows\\SysWOW64\\WScript.exe", "MyVB.vbs", "C:\\Program 
Files\\Git\", "Test").start();

2.1 allowAmbiguousCommands = false
arg[0] = C:\Program Files\Git1\\

Re: Proposed JEP: Safer Process Launch by ProcessBuilder and Runtime.exec

2022-02-18 Thread Raffaello Giulietti

Hello,

to overcome some of the problems with parsing and generating Windows
command lines, I implemented two classes [1] that attempt to provide a
more sophisticated solution. To be clear, they do not create processes
or launch programs. They only serve as a parser and an "escaper".

Currently, they are completely outside the OpenJDK codebase to avoid
interfering with the current behavior. The intent is to have a concrete
basis for a more thorough discussion and some code to experiment with.
Later, the code can be integrated into OpenJDK if so desired.

Both classes perform a straightforward, one-pass left-to-right
processing (each character is read only once) without back-patching.
They only make use String, StringBuilder and ArrayList.

Two important technical aspects must be kept in mind when later using
the outcomes of these classes to start new processes on Windows. Both
are related in the interplay between the Windows function
CreateProcess() [2] and the C/C++ runtime [3]:

* A program can parse the command line as it deems useful. There are no
hard rules, only conventions. These classes assume that the program
denoted on the command line will perform parsing as done by the Windows
C/C++ runtime conventions [3]. If this assumption is invalid, there's no
point in using these classes.

* In particular, the "shell" cmd.exe parses the command line in a
different way. While not currently exposed in these classes, it would be
easy to add a specific parser and escaper for cmd.exe as well.

* Absent the application name, the initial section of the command line
passed to CreateProcess() is parsed by it to locate the program to
launch. The way it parses the program part when it is unquoted is too
cumbersome and depends on the content of the filesystem [2]. Trying to
re-implement this parsing would introduce a potential source of troubles
that could later lead in launching an unintended program. Thus, for
simplification and caution, these classes assume that the program part
is always quoted, throwing otherwise.

Greetings
Raffaello

[1] https://github.com/rgiulietti/experiments
[2]
https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createprocessa
[3]
https://docs.microsoft.com/en-us/cpp/c-language/parsing-c-command-line-arguments

On 2022-01-20 19:05, Roger Riggs wrote:
A JEP to Improve safety of process launch by ProcessBuilder and
Runtime.exec on Windows[1].

Argument encoding errors have been problematic on Windows systems due to
improperly quoted command arguments.

The idea is to tighten up quoting and encoding of command line arguments.

Comments appreciated, Roger

[1] https://bugs.openjdk.java.net/browse/JDK-8263697

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v6]

> Hello,
> 
> here's a PR for a patch submitted on March 2020 
> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was a 
> thing.
> 
> The patch has been edited to adhere to OpenJDK code conventions about 
> multi-line (block) comments. Nothing in the code proper has changed, except 
> for the addition of redundant but clarifying parentheses in some expressions.
> 
> 
> Greetings
> Raffaello

Raffaello Giulietti has updated the pull request with a new target base due to 
a merge or a rebase. The pull request now contains 12 commits:

 - 4511638: Double.toString(double) sometimes produces incorrect results
   
   Adapted hashes in ElementStructureTest.java
 - 4511638: Double.toString(double) sometimes produces incorrect results
   
   Merge branch 'master' into JDK-4511638
 - 4511638: Double.toString(double) sometimes produces incorrect results
   
   Enhanced intervals in MathUtils.
   Updated references to Schubfach v4.
 - 4511638: Double.toString(double) sometimes produces incorrect results
   
   Merge branch 'master' into JDK-4511638
 - 4511638: Double.toString(double) sometimes produces incorrect results
   
   Slight adjustments to Javadoc as suggested in the JDK-8202555 (CSR) comments.
 - 4511638: Double.toString(double) sometimes produces incorrect results
   
   Adjusted hashes in test/langtools/tools/javac/sym/ElementStructureTest.java
 - 4511638: Double.toString(double) sometimes produces incorrect results
   
   Merge branch 'master' into JDK-4511638
 - 4511638: Double.toString(double) sometimes produces incorrect results
 - 4511638: Double.toString(double) sometimes produces incorrect results
   
   Refactored test classes to better match OpenJDK conventions.
   Added tests recommended by Guy Steele and Paul Zimmermann.
 - Changed MAX_CHARS to static
 - ... and 2 more: https://git.openjdk.java.net/jdk/compare/92f4f40d...c29dff76

-

Changes: https://git.openjdk.java.net/jdk/pull/3402/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk=3402=05
  Stats: 23939 lines in 16 files changed: 23809 ins; 54 del; 76 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3402.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3402/head:pull/3402

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v5]

On Tue, 8 Feb 2022 20:18:22 GMT, Brian Burkhalter  wrote:

>> The work (the code) has been published here on GitHub, so afaik the 
>> copyright year cannot be simply changed without modifying something (beside 
>> the year itself, which would be self-justifying)
>
> @rgiulietti You are correct: no year update needed unless there's a change to 
> the file.

@bplb I'm not even sure that just correcting a typo is considered enough of a 
change to be entitled to postpone the copyright year. But mileage may vary 
between jurisdictions ;-)

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v5]

On Mon, 15 Nov 2021 19:31:09 GMT, Raffaello Giulietti  
wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results
>   
>   Enhanced intervals in MathUtils.
>   Updated references to Schubfach v4.

The work (the code) has been published here on GitHub, so afaik the copyright 
year cannot be simply changed without modifying something (beside the year 
itself, which would be self-justifying)

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v5]

On Tue, 8 Feb 2022 19:47:25 GMT, Brian Burkhalter  wrote:

>> @bplb The last commit was pushed on Nov 15.
>> I guess I would have to merge against the latest master because of
>> test/langtools/tools/javac/sym/ElementStructureTest.java
>> and the intervening switch from JDK18 to JDK19.
>
> @rgiulietti Also the copyright year. So no further substantive updates needed?

@bplb Why the copyright year? Nothing is expected to change, except for the 
test mentioned.

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v5]

On Tue, 8 Feb 2022 19:23:54 GMT, Brian Burkhalter  wrote:

>> En attendant go! go!
>
> @rgiulietti Should we expect any more commits from you in this PR? Thanks.

@bplb The last commit was pushed on Nov 15.
I guess I would have to merge against the latest master because of
test/langtools/tools/javac/sym/ElementStructureTest.java
and the intervening switch from JDK18 to JDK19.

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: Constant methods in Java

2022-02-02 Thread Raffaello Giulietti


Hello,

I don't get why the author of myMethod() would/should be interested in 
forcing the caller of the method to declare the variable b to be final.

Stated otherwise, what is the problem addressed by this suggestion?
Have you some specific case in mind?


Greetings
Raffaello


On 2022-02-02 20:27, Alberto Otero Rodríguez wrote:

I have a suggestion. I think it would be interesting creating constant methods 
in Java.

I mean methods declared like this:

public const String myMethod() {
   String a = "a";
   a = a + "b";
   return a;
}

So that the response of the method is forced to be assigned to a final variable.

This would be ok:
final String b = myMethod();

But this would throw a compilation error:
String c = myMethod();

What do you think? It's just an idea.

Re: Proposed JEP: Safer Process Launch by ProcessBuilder and Runtime.exec

2022-01-28 Thread Raffaello Giulietti


Hi Roger,

I'm trying the following (ugly) code on JDK 17/Win, where Args.exe does 
nothing else than writing out its argv[], redirecting to a log file.


public static void main(String[] args) throws IOException, 
InterruptedException {

String[] command = {
"C:\\Users\\alpha\\Args.exe",
"",
"a",
"",
"b",
"",
};
var processBuilder = new ProcessBuilder(command);
processBuilder.redirectOutput(new 
File("C:\\Users\\alpha\\my.log"));

var process = processBuilder.start();
Thread.sleep(2_000);
System.out.println("process.exitValue() = " + process.exitValue());
}


Here's the log file

argv[0] = [C:\Users\alpha\Args.exe]
argv[1] = []
argv[2] = [a]
argv[3] = []
argv[4] = [b]
argv[5] = []

so empty args seem to work correctly, at least in this plain example.

Have you specific examples that behave incorrectly?
I'm asking because I'd like to setup a simple set of rules to solve the 
issue on Windows altogether.






On 2022-01-28 16:48, Roger Riggs wrote:

Hi Raffaello,

For .exe executables, one example is an empty string in the list of 
arguments to ProcessBuilder.
The empty string is not visible in the generated command line. For 
position sensitive commands, it appears the argument is dropped.
An argument in ProcessBuilder with mismatched quotes can cause the 
argument to be joined with the next in the generated command line.
A stray "\" at the end of an argument can cause the following character 
to be quoted, possibly joining the argument with the next.


For .cmd executables, cmd.exe interprets more characters as argument 
separators and will split arguments.
For example, an argument with a semi-colon or comma, (unquoted) will be 
split into two arguments when parsed by cmd.exe.
The goal is to improve the integrity and robustness of the command 
encoding.


Thanks, Roger


On 1/28/22 4:07 AM, Raffaello Giulietti wrote:

Hello,

if I understand correctly, the issue addressed here (on Windows) is 
how to assemble a single command string from an array of argument 
strings to pass to CreateProcess() in a way that the individual 
argument strings can be fully recovered in the invoked program.

Similarly when the command string is passed to an instance of cmd.exe.

Are there known (non security critical) examples that do not work 
correctly JDK 18 or earlier?



Greetings
Raffaello


On 2022-01-20 19:05, Roger Riggs wrote:
A JEP to Improve safety of process launch by ProcessBuilder and 
Runtime.exec on Windows[1].


Argument encoding errors have been problematic on Windows systems due to
improperly quoted command arguments.

The idea is to tighten up quoting and encoding of command line 
arguments.


Comments appreciated,  Roger

[1] https://bugs.openjdk.java.net/browse/JDK-8263697

Re: Proposed JEP: Safer Process Launch by ProcessBuilder and Runtime.exec

2022-01-28 Thread Raffaello Giulietti


Hello,

if I understand correctly, the issue addressed here (on Windows) is how 
to assemble a single command string from an array of argument strings to 
pass to CreateProcess() in a way that the individual argument strings 
can be fully recovered in the invoked program.

Similarly when the command string is passed to an instance of cmd.exe.

Are there known (non security critical) examples that do not work 
correctly JDK 18 or earlier?



Greetings
Raffaello


On 2022-01-20 19:05, Roger Riggs wrote:
A JEP to Improve safety of process launch by ProcessBuilder and 
Runtime.exec on Windows[1].


Argument encoding errors have been problematic on Windows systems due to
improperly quoted command arguments.

The idea is to tighten up quoting and encoding of command line arguments.

Comments appreciated,  Roger

[1] https://bugs.openjdk.java.net/browse/JDK-8263697

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v5]

2022-01-11 Thread Raffaello Giulietti

On Mon, 15 Nov 2021 19:31:09 GMT, Raffaello Giulietti  
wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results
>   
>   Enhanced intervals in MathUtils.
>   Updated references to Schubfach v4.

En attendant go! go!

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: a quick question about String

2022-01-01 Thread Raffaello Giulietti


Hello,

the simple answer is that
new String(...) == new String(...)
*never* evaluates to true )it might throw, though), whatever 
constructors you are using and whatever arguments you pass to to them.


Some garbage collectors *might* de-duplicate the underlying arrays (but 
you cannot tell), but the references to the Strings will be different.



HTH
Raffaello


On 2021-12-23 18:59, Alan Snyder wrote:

Do the public constructors of String actually do what their documentation says 
(allocate a new instance), or is there some kind of compiler magic that might 
avoid allocation?

Re: Type narrowing security leak

2022-01-01 Thread Raffaello Giulietti


Hi,

if I understand correctly, you are supposing that method changeAmount() 
is part of a security sensitive application under attack.


Its implementation first attempts a data validation by invoking 
isUserIdAllowedOrThrowException(), then transforms validated data in a 
lossy way (narrowing, then widening) for further usage (invoking 
doChangeAmount()).
A seriously conducted review of security related code, whether by 
experienced engineers or by means of tools, would reject this code as 
fatally flawed in the first place, code that should never become productive.


(BTW, (1L << Integer.SIZE) + 63 is a more efficient way to produce e 
rebound of 63 for your example.)


Yours is just one example of the more general class of integer overflow 
attacks that potentially affects modular integer arithmetic in Java and 
many other languages. To prevent such attacks, operations on integer 
values should throw on overflow. Modifying the Java spec to mandate 
throwing on overflow would be a substantial change with major 
compatibility issues and preventing many legitimate usages of modular 
arithmetic as it is currently specified.


Security sensitive code can (and does) make good use of 
j.l.Math.*Exact() methods (addExact(), toIntExact(), etc.)



Greetings
Raffaello


On 2021-12-29 07:43, Fabrice Tiercelin wrote:

Greetings,
Any Java application may be concerned by a hacker attack using a type narrowing 
leak. If a program does the following things in this order:
  - Assert that a numerical id is allowed - Do a type narrowing among other 
things, even followed by a type widening - Do an action with the numerical id
...the hacker can do forbidden actions. Let's say that a given user doesn't 
have rights to change an amount for the id 63:
public void changeAmount(long userId, double newAmount) throws 
IllegalArgumentException {  isUserIdAllowedOrThrowException(userId); // userId 
= 4294967359

    int theUserId = (int) userId; // theUserId = 63    userId = theUserId; // 
userId = 63
   doChangeAmount(userId, newAmount); // userId = 63
}
It will fail passing 63 but it will success passing 4294967359 because 
4_294_967_359 is narrowed into 63. Let's call 4_294_967_359  a rebound of 63. 
4294967359 can be retrieved in few seconds by a basic program like this:
public class MyClass {
     public static void main(String args[]) {
   long targettedNumber = 63;
   
    for (long rebound = Integer.MAX_VALUE + 1; true; rebound++) {  int typeNarrowing = (int) rebound;

   long typeWidening = typeNarrowing;
   
   if (typeWidening == targettedNumber) {

   System.out.println("Rebound for " + targettedNumber + " found: " 
+ rebound);
   return;
   }
   }
     }
}
And it can be optimized. It works for any type narrowing. It not only works for 
numerical id but also for flags. If a numerical value should contain or not 
several flags, you can search a rebound among billions of rebounds until you 
find one with the perfect features. All the Java versions are concerned. The 
security layer can even be coded in another programming language.

To fix it, I suggest to add a test just before the type narrowing in the 
bytecode. The test verifies if the type narrowing will alter the numerical 
value. If true, it throws an unchecked exception or an error. Otherwise, it 
continues as currently. Note that it changes the behavior but the current 
behavior is dangerous, useless and is a failing case. You can add a compiler 
option to restore the original behavior but the new behavior should be the 
default.

Best regards,Fabrice TIERCELIN

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v5]

2021-11-17 Thread Raffaello Giulietti


Hi Suminda,

please read my reply to you from May
https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-May/077081.html


Greetings
Raffaello


On 2021-11-17 14:54, Suminda Sirinath Salpitikorala Dharmasena wrote:

On Mon, 15 Nov 2021 19:31:09 GMT, Raffaello Giulietti  
wrote:


Hello,

here's a PR for a patch submitted on March 2020 
[1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was a 
thing.

The patch has been edited to adhere to OpenJDK code conventions about 
multi-line (block) comments. Nothing in the code proper has changed, except for 
the addition of redundant but clarifying parentheses in some expressions.


Greetings
Raffaello


Raffaello Giulietti has updated the pull request incrementally with one 
additional commit since the last revision:

   4511638: Double.toString(double) sometimes produces incorrect results
   
   Enhanced intervals in MathUtils.

   Updated references to Schubfach v4.


Also there is Dragonbox algorithm which might be faster: 
https://github.com/jk-jeon/dragonbox/blob/master/other_files/Dragonbox.pdf

-

PR: https://git.openjdk.java.net/jdk/pull/3402

A guide to PR3402 ("[Double,Float].toString()") code reviewers

2021-11-16 Thread Raffaello Giulietti


Hello,

as mentioned in [1], the Schubfach writing accompanying JBS issues [2] 
and [3] has recently undergone a thorough review by renowned researchers 
(in addition to Dmitry Nadezhin back in 2018-19 and myself on a 
continuous basis) [4].


For the *code* reviewers of PR [5] this means that there's no need for 
them to deal with the more mathematical aspects of the writing. They can 
instead concentrate in crosschecking that the Java code is in sync with 
the pseudo-code in the writing, as well as about its readability.


To facilitate this review, there are comments scattered in the code that 
refer back to the writing for easy reference.



# MathUtils

For the flog*() methods in MathUtils refer to section 9.1.1.

The large lookup table in MathUtils is defined in result 24 as well as 
in javadoc.
MathUtilsChecker contains fully commented code that tests the 
correctness of the table as well as the flog*() methods to an extent 
sufficient for the conversions of doubles.
(The table has also been checked independently by one of the article 
reviewers.)



# DoubleToDecimal

With regard to DoubleToDecimal, the fast path described in 8.3 is coded 
as part of toDecimal(double).


In DoubleToDecimal, toDecimal(int,long,int) is a Java translation of the 
Schubfach skeleton in figure 7 combined with the suggestions in figure 9.


Figure 8 is embodied in rop(long,long,long).

§10 about digits extraction is implemented in toChars(long,int) and its 
dependencies.


The rest of the code is just glue and should be easy to understand.


# FloatToDecimal

The code of FloatToDecimal is almost identical but adapted to the 32 bit 
case. Method rop(long,long) is from figure 11.



# Test classes

I hope it turns out that the test classes are either commented well 
enough to guide the reader or are kind of straightforward to understand.




There are some package private constants in the code that are not 
referenced anywhere else in src/ and could thus be removed. However, 
since there's no great harm retaining them and since they might turn out 
to be useful in later work, I would suggest not to get rid of them.



Of course, I'm at the disposal of this community for any question 
related to the PR and the writing.



HTH
Raffaello



[1] 
https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-November/083536.html

[2] https://bugs.openjdk.java.net/browse/JDK-4511638
[3] https://bugs.openjdk.java.net/browse/JDK-8202555
[4] https://drive.google.com/file/d/1IEeATSVnEE6TkrHlCYNY2GjaraBjOT4f
[5] https://github.com/openjdk/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v5]

2021-11-15 Thread Raffaello Giulietti

> Hello,
> 
> here's a PR for a patch submitted on March 2020 
> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was a 
> thing.
> 
> The patch has been edited to adhere to OpenJDK code conventions about 
> multi-line (block) comments. Nothing in the code proper has changed, except 
> for the addition of redundant but clarifying parentheses in some expressions.
> 
> 
> Greetings
> Raffaello

Raffaello Giulietti has updated the pull request incrementally with one 
additional commit since the last revision:

  4511638: Double.toString(double) sometimes produces incorrect results
  
  Enhanced intervals in MathUtils.
  Updated references to Schubfach v4.

-

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/3402/files
  - new: https://git.openjdk.java.net/jdk/pull/3402/files/37acd52a..45881cd4

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk=3402=04
 - incr: https://webrevs.openjdk.java.net/?repo=jdk=3402=03-04

  Stats: 26 lines in 3 files changed: 0 ins; 0 del; 26 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3402.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3402/head:pull/3402

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR (CSR): 8202555: Double.toString(double) sometimes produces incorrect results

2021-11-15 Thread Raffaello Giulietti


Hello,

the 2nd version of the article [1] accompanying this issue [2] and issue 
[3] has been kindly and thoroughly reviewed by the following renowned 
world-class researchers:


* Guy Steele Jr [4], Oracle Labs
* Paul Zimmermann [5], INRIA
* Jean-Michel Muller [6], Ecole Normale Supérieure de Lyon

Thanks to the improvements contributed by these fine reviewers, the 
newest 4th version [7] is now in "preview" form, the intent being to 
publish it on a more permanent platform (academic journal or similar).


In addition, they also suggested and provided new hard tests which are 
included in the PR [8].


I hope this will help in progressing both the CSR and the PR.


Greetings
Raffaello



[1] https://drive.google.com/file/d/1luHhyQF9zKlM8yJ1nebU0OgVYhfC6CBN
[2] https://bugs.openjdk.java.net/browse/JDK-8202555
[3] https://bugs.openjdk.java.net/browse/JDK-4511638
[4] https://labs.oracle.com/pls/apex/f?p=labs%3Abio%3A0%3A120
[5] https://members.loria.fr/PZimmermann/
[6] https://perso.ens-lyon.fr/jean-michel.muller/
[7] https://drive.google.com/file/d/1IEeATSVnEE6TkrHlCYNY2GjaraBjOT4f
[8] https://github.com/openjdk/jdk/pull/3402



On 2021-10-26 21:28, Raffaello Giulietti wrote:

Hello,

PR [1] and the accompanying article [2] have been subject to some 
positive reactions in the last couple of weeks. A fresh set of about 20 
thousand additional hard test cases kindly provided by Paul Zimmermann 
of INRIA and other tests proposed by Guy Steele have been added to the 
code.


The corresponding CSR [3] is in Finalized state for review. The proposed 
spec has been carefully written to uniquely define the outcomes and the 
code in the PR has been extensively tested to match the proposed spec.


Behavioral backward compatibility has been a major goal for the CSR. In 
fact, in the vast majority of cases, the CSR and the current 
implementation agree on the outcomes.


As the CSR is a prerequisite for the advancement of the PR, I beg 
everybody entitled (and interested) to review and approve it and/or 
discuss it further.



Greetings
Raffaello



[1] https://github.com/openjdk/jdk/pull/3402
[2] https://drive.google.com/file/d/1luHhyQF9zKlM8yJ1nebU0OgVYhfC6CBN/view
[3] https://bugs.openjdk.java.net/browse/JDK-8202555

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v4]

2021-10-27 Thread Raffaello Giulietti

> Hello,
> 
> here's a PR for a patch submitted on March 2020 
> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was a 
> thing.
> 
> The patch has been edited to adhere to OpenJDK code conventions about 
> multi-line (block) comments. Nothing in the code proper has changed, except 
> for the addition of redundant but clarifying parentheses in some expressions.
> 
> 
> Greetings
> Raffaello

Raffaello Giulietti has updated the pull request with a new target base due to 
a merge or a rebase. The pull request now contains ten commits:

 - 4511638: Double.toString(double) sometimes produces incorrect results
   
   Merge branch 'master' into JDK-4511638
 - 4511638: Double.toString(double) sometimes produces incorrect results
   
   Slight adjustments to Javadoc as suggested in the JDK-8202555 (CSR) comments.
 - 4511638: Double.toString(double) sometimes produces incorrect results
   
   Adjusted hashes in test/langtools/tools/javac/sym/ElementStructureTest.java
 - 4511638: Double.toString(double) sometimes produces incorrect results
   
   Merge branch 'master' into JDK-4511638
 - 4511638: Double.toString(double) sometimes produces incorrect results
 - 4511638: Double.toString(double) sometimes produces incorrect results
   
   Refactored test classes to better match OpenJDK conventions.
   Added tests recommended by Guy Steele and Paul Zimmermann.
 - Changed MAX_CHARS to static
 - 4511638: Double.toString(double) sometimes produces incorrect results
 - 4511638: Double.toString(double) sometimes produces incorrect results

-

Changes: https://git.openjdk.java.net/jdk/pull/3402/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk=3402=03
  Stats: 23945 lines in 16 files changed: 23809 ins; 61 del; 75 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3402.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3402/head:pull/3402

PR: https://git.openjdk.java.net/jdk/pull/3402

RFR (CSR): 8202555: Double.toString(double) sometimes produces incorrect results

2021-10-26 Thread Raffaello Giulietti


Hello,

PR [1] and the accompanying article [2] have been subject to some 
positive reactions in the last couple of weeks. A fresh set of about 20 
thousand additional hard test cases kindly provided by Paul Zimmermann 
of INRIA and other tests proposed by Guy Steele have been added to the code.


The corresponding CSR [3] is in Finalized state for review. The proposed 
spec has been carefully written to uniquely define the outcomes and the 
code in the PR has been extensively tested to match the proposed spec.


Behavioral backward compatibility has been a major goal for the CSR. In 
fact, in the vast majority of cases, the CSR and the current 
implementation agree on the outcomes.


As the CSR is a prerequisite for the advancement of the PR, I beg 
everybody entitled (and interested) to review and approve it and/or 
discuss it further.



Greetings
Raffaello



[1] https://github.com/openjdk/jdk/pull/3402
[2] https://drive.google.com/file/d/1luHhyQF9zKlM8yJ1nebU0OgVYhfC6CBN/view
[3] https://bugs.openjdk.java.net/browse/JDK-8202555

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v3]

2021-10-21 Thread Raffaello Giulietti

> Hello,
> 
> here's a PR for a patch submitted on March 2020 
> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was a 
> thing.
> 
> The patch has been edited to adhere to OpenJDK code conventions about 
> multi-line (block) comments. Nothing in the code proper has changed, except 
> for the addition of redundant but clarifying parentheses in some expressions.
> 
> 
> Greetings
> Raffaello

Raffaello Giulietti has updated the pull request with a new target base due to 
a merge or a rebase. The pull request now contains seven commits:

 - 4511638: Double.toString(double) sometimes produces incorrect results
   
   Adjusted hashes in test/langtools/tools/javac/sym/ElementStructureTest.java
 - 4511638: Double.toString(double) sometimes produces incorrect results
   
   Merge branch 'master' into JDK-4511638
 - 4511638: Double.toString(double) sometimes produces incorrect results
 - 4511638: Double.toString(double) sometimes produces incorrect results
   
   Refactored test classes to better match OpenJDK conventions.
   Added tests recommended by Guy Steele and Paul Zimmermann.
 - Changed MAX_CHARS to static
 - 4511638: Double.toString(double) sometimes produces incorrect results
 - 4511638: Double.toString(double) sometimes produces incorrect results

-

Changes: https://git.openjdk.java.net/jdk/pull/3402/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk=3402=02
  Stats: 23941 lines in 16 files changed: 23805 ins; 61 del; 75 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3402.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3402/head:pull/3402

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

2021-10-15 Thread Raffaello Giulietti

On Fri, 16 Apr 2021 11:30:32 GMT, Raffaello Giulietti  
wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results

"All have the form 2^q10^(-k)"

(Paul Zimmermann's reply)

not quite, for example 0x1.3eaba12cap-804 = 2251808225167717/2^855 is not 
of this form, but these cases come from the continued fraction expansion of 
2^q10^(-k).

(my reply)

Right, it is correctly stated in the test class (to be pushed) but incorrectly 
in my previous post
Raffaello

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

2021-10-14 Thread Raffaello Giulietti

On Fri, 16 Apr 2021 11:30:32 GMT, Raffaello Giulietti  
wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results

Hello,

Paul Zimmermann of INRIA kindly provided me 19'545 doubles generated on purpose 
as hard test cases.
All have the form 2^q*10^(-k), with k as in my writing 
(https://drive.google.com/file/d/1luHhyQF9zKlM8yJ1nebU0OgVYhfC6CBN/view)

They all pass on my machine and will be added to the this PR asap.

Greetings
Raffaello

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

2021-10-13 Thread Raffaello Giulietti

On Fri, 16 Apr 2021 11:30:32 GMT, Raffaello Giulietti  
wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results

Hi Guy,

> I have now read through the Paxson paper. Does this refer to the values 
> listed in his Tables 3 and 4, or to other values instead or in addition?

Right.

>> Shortening my 27 pages writing and re-formating it to meet a journal 
>> standards for publication would require investing yet another substantial 
>> amount of time. I'm not sure I'm prepared for that, given that I've no 
>> personal interest in a journal publication: I'm not an academic, I'm not 
>> pursuing a career...
>> But I promise I'll think about reconsidering my position on this ;-)
> 
> Please do think about reconsidering.

OK, thanks for the useful suggestions.


Greetings
Raffaello

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

2021-10-13 Thread Raffaello Giulietti

On Fri, 16 Apr 2021 11:30:32 GMT, Raffaello Giulietti  
wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results

(Replies from Guy Steele)


> Hi Guy,
> 
> for some reason your comments still appear garbled on the GitHub PR page and 
> don't make it to the core-libs-dev mailing list at all. Luckily, they appear 
> intelligible in my mailbox, so I'll keep going with prepending your comments 
> in my replies: not ideal but good enough.
> 
> Thanks so much for re-reading my "paper".
> 
> printf()
> 
> There are some issues to consider when trying to apply Schubfach to printf(), 
> the main one being that printf() allows to specify an arbitrary length for 
> the resulting decimal. This means, for example, that unlimited precision 
> arithmetic is unavoidable. But it might be worthwhile to investigate using 
> Schubfach for lengths up to H (9 and 17 for float and double, resp.) and fall 
> back to unlimited precision beyond that.
> Before that, however, I would prefer to finally push Schubfach in the OpenJDK 
> codebase for the toString() cases and close this PR.

I completely agree that using Schubfach to solve only the toString() problems 
would be a _major_ improvement in the situation, and this should not wait for 
exploration of the printf problem. But I suspect that using Schubfach for 
lengths up to H would cover a very large fraction of actual usage, and would 
improve both quality and speed, and therefore would be worth exploring later.

> Tests
> 
> Below, by "extensive test" I mean not only that the outcomes convert back 
> without loss of information, but that they fully meet the spec about minimal 
> number of digits, closeness, correct formatting (normal viz scientific), 
> character set, etc.
> 
> All currently available tests are in the contributed code of this PR and will 
> be part of the OpenJDK once integrated.
> 
> * All powers of 2 and the extreme values are already extensively tested.
> * All values closest to powers of 10 are extensively tested.
> * All values proposed by Paxson [1] are extensively tested.

I have now read through the Paxson paper. Does this refer to the values listed 
in his Tables 3 and 4, or to other values instead or in addition?

> * A configurable number of random values are tested at each round (currently 
> 4 * 1'000'000 random values). Should a value fail, there's enough diagnostic 
> information for further investigation.
> 
> I'll add extensive tests for the values you propose in point (1) and (2), 
> setting Z = Y = 1024.
> 

I do think that would lend further confidence.

> As for comparison with the current JDK behavior, there are already a bunch of 
> values for which extensive tests fail on the current JDK but pass with 
> Schubfach.

Yes, thanks for supplying some of those.

> It would be cumbersome, if possible at all, to have both the current JDK and 
> Schubfach implementations in the same OpenJDK codebase to be able to compare 
> the outcomes. I performed comparisons in a different constellation, with 
> Schubfach as an external library, but this is hardly a possibility in the 
> core-libs. Needless to say, Schubfach outcomes are either the same as in JDK 
> or better (shorter or closest to the fp value).

Okay.

I will mention here, for the record, that there is one other sort of test that 
could be performed that I think I have not yet seen you mention: a monotonicity 
test of the kind used by David Hough’s Testbase (mentioned by Paxson). However, 
a little thought reveals that such a test made unnecessary by the round-trip 
test. So a monotonicity test would be a good idea when testing printf case, but 
is not needed for the toString case.

Therefore, if you add the few tests I have suggested, I think that we can say 
with the best certainty we can have, short of separately testing every possible 
double value, that Schubfach is extremely well tested and ready for adoption 
into Java.

> Peer reviewed publication
> 
> Shortening my 27 pages writing and re-formating it to meet a journal 
> standards for publication would require investing yet another substantial 
> amount of time. I'm not sure I'm prep

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

2021-10-13 Thread Raffaello Giulietti

On Fri, 16 Apr 2021 11:30:32 GMT, Raffaello Giulietti  
wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results

(Guy Steele reply to a previous comment of mine)


Yes, thank you, I stated my suggested criterion incorrectly.

—Guy

On Oct 11, 2021, at 4:16 AM, Raffaello Giulietti ***@***.**@***.***>> 
wrote:> Hi Guy,
> 
> while implementing the additional test recommended in your point (2), it 
> occurred to me that the numbers of the form y 10^n, y in D_k (k = 1, 2, 3, 4) 
> end up being of the form y' 10^n', where y' = y / 10^k, n' = n + k, plus the 
> 2 * Y values around these. Such numbers do not seem to show any special 
> structure worth a dedicated test, so I'm wondering if you mean something else 
> instead.
> 
> Perhaps you mean y to have at most 4 digits, i.e., 0 <= y < 10^4?
> 
> Greetings
> Raffaello
> 
> P.S. The test recommended in point (1) pass successfully.
>

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

2021-10-11 Thread Raffaello Giulietti

On Fri, 16 Apr 2021 11:30:32 GMT, Raffaello Giulietti 
 wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results

Hi Guy,

while implementing the additional test recommended in your point (2), it 
occurred to me that the numbers of the form y 10^n, y in D_k (k = 1, 2, 3, 4) 
end up being of the form y' 10^n', where y' = y / 10^k, n' = n + k, plus the 2 
* Y values around these. Such numbers do not seem to show any special structure 
worth a dedicated test, so I'm wondering if you mean something else instead.

Perhaps you mean y to have at most 4 digits, i.e., 0 <= y < 10^4?

Greetings
Raffaello

P.S. The test recommended in point (1) pass successfully.

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

2021-10-09 Thread Raffaello Giulietti

On Fri, 16 Apr 2021 11:30:32 GMT, Raffaello Giulietti 
 wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results

(comment by Guy Steele)

Hi, Raffaello,

I will try to compose this message in plain ASCII and hope it does not get 
garbled.

I have now re-read the Schubfach paper, and with the extra information in your 
previous email, I think I understood it a lot better this time. I am very 
enthusiastic about the approach. This is exactly what JonL White was trying to 
puzzle out some five decades ago: just how many digits do you need in your 
powers of ten to get accurate print-read round-trips?

I am especially appreciative of the attention paid in the Schubfach paper to 
parameter M, which ensures that two-digit possibilities are considered in cases 
where it is required to print two significant digits anyway (scientific 
notation).

I have also read the two papers I could find about Ryu. Schubfach is clearly an 
improvement over Ryu because it avoids the iteration, among other reasons. But 
the second paper, about Ryu for printf, addresses the specific difficulties of 
generating digits for printf format specifiers, and raises the interesting 
question of whether it would be worthwhile also to design a version of 
Schubfach that handles printf requirements.

So I think it would be well worth incorporating the Schubfach algorithm into 
the JDK codebase. I am reassured that Schubfach has undergone extensive testing 
on trillions of randomly chosen values. But I would be further reassured by a 
statement that two important classes of values have been _exhaustively_ tested:

(1) All positive powers of 2 representable in the double format, plus the 
maximum representable value. (These are the values that are surrounded by 
irregular spacing.) Furthermore, out of caution one should also test every fp 
value within Z ulps of such a value; perhaps Z should be 64 or even 1024.

(2) All representable fp values that are the result of rounding-to-nearest some 
decimal value of the form y•10^n, where y is a member of, say, either D_1, D_2, 
D_3, or D_4. (These are values that may be especially susceptible to generation 
of too many digits by an insufficiently careful algorithm, and one reason might 
be insufficient precision or other algorithmic error in connection with a table 
of powers of ten.) Furthermore, out of caution one should also test every fp 
value within Y ulps of such a value; perhaps Y should be 64 or even 1024.

In every case, the testing should include (a) ensuring round-trip print-read 
behavior; (b) comparing to what is produced by the current JDK algorithm, and 
investigating any cases that differ.

And if other similar “edge cases” can be identified from the structure of the 
code, those would be worth focusing on as well.

If this testing has been or could be done, I would say, yes, certainly adopt 
Schubfach for Java. I would also recommend submitting the Schubfach paper to an 
appropriate conference or journal to get the benefit of further peer review of 
the algorithm and its write-up.

—Guy

(my reply)

Hi Guy,

for some reason your comments still appear garbled on the GitHub PR page and 
don't make it to the core-libs-dev mailing list at all. Luckily, they appear 
intelligible in my mailbox, so I'll keep going with prepending your comments in 
my replies: not ideal but good enough.

Thanks so much for re-reading my "paper".

printf()

There are some issues to consider when trying to apply Schubfach to printf(), 
the main one being that printf() allows to specify an arbitrary length for the 
resulting decimal. This means, for example, that unlimited precision arithmetic 
is unavoidable. But it might be worthwhile to investigate using Schubfach for 
lengths up to H (9 and 17 for float and double, resp.) and fall back to 
unlimited precision beyond that.
Before that, however, I would prefer to finally push Schubfach in the OpenJDK 
codebase for the toString() cases and close this PR.

Tests

Below, by "extensive test" I mean not only that the outcomes convert back 
without loss of information, but that they fully meet the spec about minimal 
number of digits, closeness, correct formatting (normal viz scientific), 
character set, etc.

All currently available tests are in the contributed code o

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

2021-10-06 Thread Raffaello Giulietti

On Fri, 16 Apr 2021 11:30:32 GMT, Raffaello Giulietti 
 wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results

The last time I ran JMH over the whole range of bitwise uniformly distributed 
random doubles, I measured a speedup of about 17.7x
https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-April/065921.html

This is about 70 ns/conversion on a 2013 consumer-grade laptop.

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

2021-10-06 Thread Raffaello Giulietti

On Fri, 16 Apr 2021 11:30:32 GMT, Raffaello Giulietti 
 wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results

(ungarbled comment from Guy Steele)

Hi, thanks for writing to me! I am glad to “meet” you. Schubfach looks like an 
excellent idea; I have simply been trying to assess just how confident we can 
be about the correctness of the final code. The explanations you have just 
provided to me about the proof process and testing are very reassuring. I will 
keep those in mind as I finish re-reading your paper, and will have more to say 
on Friday. And if you happen to have a list of a few examples where the current 
Java implementation of FP print fails but Schubfach succeeds, I would be very 
interested to study them. Thanks!

—Guy

(my answer)

Hi,

it's my pleasure to virtually meet the mythical Guy Steele!

For some reason, your comments end up garbled on this PR GitHub page and don't 
appear at all on the core-libs-dev mailing list. You may perhaps try to post 
them in plain ASCII?

You can find some of the anomalies in the bug report from exactly 20 years ago! 
[1]

A tiny collection of the anomalies are coded in the tests [2], [3]. Here's the 
relevant excerpt for doubles:

/*
 * There are tons of doubles that are rendered incorrectly by older JDK.
 * While the renderings correctly round back to the original value,
 * they are longer than needed or are not the closest decimal to the double.
 * Here are just a very few examples.
 */
private static final String[] Anomalies = {
/* JDK renders these, and others, with 18 digits! */
"2.82879384806159E17", "1.387364135037754E18",
"1.45800632428665E17",

/* JDK renders these longer than needed */
"1.6E-322", "6.3E-322",
"7.3879E20", "2.0E23", "7.0E22", "9.2E22",
"9.5E21", "3.1E22", "5.63E21", "8.41E21",

/* JDK does not render these, and many others, as the closest */
"9.9E-324", "9.9E-323",
"1.9400994884341945E25", "3.6131332396758635E25",
"2.5138990223946153E25",
};

To be clear and as emphasized in the comment, the outputs of the current JDK 
implementation are all information preserving. I could never find a real error 
in this regard.
However, typing 2e23 and getting back 1.9998E23 as a response is 
surprising and unnecessary if one knows David Matula's results from the late 
'60-ies.

Also, many powers of 2 are output with way too many digits on current JDK.

Schubfach's outputs pass the stringent tests.

Greetings
Raffaello

[1] https://bugs.openjdk.java.net/browse/JDK-4511638
[2] 
https://github.com/openjdk/jdk/pull/3402/files#diff-ee9b0bcfb171cb200f6aa1e3de972b61ca363f95bb8079255af912ae30026937
[3] 
https://github.com/openjdk/jdk/pull/3402/files#diff-c4a866bc9e7d1beb5342823f9f6cff1fe4f69c96b61620b96cc26d739f9c999f

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

2021-10-05 Thread Raffaello Giulietti

On Fri, 16 Apr 2021 11:30:32 GMT, Raffaello Giulietti 
 wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results

Hi,

here's Guy's original email which I got intact in my mailbox but which appears 
completely garbled on this PR page (if you are reading it) or does not appear 
at all in this mailing list (if you are reading that).

I repeat my comment as well

HTH

Hi, sorry for the silence. Back in April 2021 I did read the entire Schubfach 
paper as well as two papers about Ryu by Adams. Schubfach looks intriguing, but 
it depends (as so many of these algorithms do) on a subtle proof. I am glad to 
see that the proof, or parts of it, are machine-checked in some way. I would be 
a lot more confident if this proof were also subjected to peer review. These 
algorithms are very much like the buggy Pentium FDIV: nearly all the cases are 
easy, but the cases that are hard (and therefore may potentially fail if you 
get that part of the algorithm wrong) are very rare, so doing good testing is 
tricky and deserves attention. If we can identify those hard cases (typically 
related to edge cases in the table entries) perhaps we can develop a good 
testing methodology. I would suspect that these hard cases relate to situations 
where a power of (1/5) is very close to a power of (1/2) (and therefore the 
corresponding power of 5 is very close to a power of 2), but t
 he details matter.

Now that I have finished addressing bug 
https://bugs.openjdk.java.net/browse/JDK-8273792 (JumpableGenerator.rngs() 
documentation refers to wrong method) and bug 
https://bugs.openjdk.java.net/browse/JDK-8273056 (java.util.random does not 
correctly sample exponential or Gaussian distributions), I am now re-reading 
the Schubfach paper and am also investigating mentions of an implementation of 
that algorithm called DragonBox. I will have more to say by Friday (I have a 
Thursday deadline for something else).

—Guy

Hello,

thanks for reading my writing.

In some way, Ryu and Schubfach are similar in the sense that both are based on 
the observation that good, fixed size approximations of powers of 10 lead to 
extremely efficient algorithms. Both algorithms end up with a lookup table of 
617 approximations.

The proof of the central theorem of Schubfach is based on continued fractions. 
It was prepared on ACL2 by the late Dmitry Nadezhin, who was a member of 
Oracle's formal verification group. Dmitry also knew my writing inside-out: not 
a formal peer review but perhaps even more insightful.

Schubfach has been exhaustively tested on all 2^32 floats, with approximations 
of powers of 10 of 63 bits each, rather than the full 126 bits used for doubles 
(the minimum size for doubles is 123 bits).
It has also been tested for months on doubles, accumulating several trillions 
of witnesses and no failure.
The exhaustive test on floats is an optionally executed part of the contributed 
code.
After more than 25 years the current JDK implementation still contains 
anomalies, so I guess it isn't tested as well as Schubfach.

A complete Schubfach conversion only depends on some dozens of primitive 
operations (it doesn't even need any sort of hardware division at all) and on a 
lookup table (fully tested for correctness in the contributed code).
The current implementation in the JDK relies on the correctness of BigInteger 
and related classes, which are way more complex to thoroughly test and to be 
confident upon.

Checking whether the 617 tabulated powers of 10 are very close to powers of 2 
would be very easy, so I'm not sure I understand your point.
Identifying the hard cases would require even more theory and would be an 
endeavor in itself. This theory could, in turn, contain small errors and would 
probably require identifying its hard cases...

According to its author, DragonBox combines Schubfach and Grisu. AFAIK, there's 
no new fundamental underlying theory.
During development of Schubfach, I experimented with a similar mix: on the JVM 
of the time (2018) performance was worse than pure Schubfach and the code more 
complex, so I gave up and switched back to the simpler design, which C2 likes 
better.

Greetings
Raffaello

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

2021-10-05 Thread Raffaello Giulietti

On Fri, 16 Apr 2021 11:30:32 GMT, Raffaello Giulietti 
 wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results

Hello,

thanks for reading my writing.

In some way, Ryu and Schubfach are similar in the sense that both are based on 
the observation that good, fixed size approximations of powers of 10 lead to 
extremely efficient algorithms. Both algorithms end up with a lookup table of 
617 approximations.

The proof of the central theorem of Schubfach is based on continued fractions. 
It was prepared on ACL2 by the late Dmitry Nadezhin, who was a member of 
Oracle's formal verification group. Dmitry also knew my writing inside-out: not 
a formal peer review but perhaps even more insightful.

Schubfach has been exhaustively tested on all 2^32 floats, with approximations 
of powers of 10 of 63 bits each, rather than the full 126 bits used for doubles 
(the minimum size for doubles is 123 bits).
It has also been tested for months on doubles, accumulating several trillions 
of witnesses and no failure.
The exhaustive test on floats is an optionally executed part of the contributed 
code.
After more than 25 years the current JDK implementation still contains 
anomalies, so I guess it isn't tested as well as Schubfach.

A complete Schubfach conversion only depends on some dozens of primitive 
operations (it doesn't even need any sort of hardware division at all) and on a 
lookup table (fully tested for correctness in the contributed code).
The current implementation in the JDK relies on the correctness of BigInteger 
and related classes, which are way more complex to thoroughly test and to be 
confident upon.

Checking whether the 617 tabulated powers of 10 are very close to powers of 2 
would be very easy, so I'm not sure I understand your point.
Identifying the hard cases would require even more theory and would be an 
endeavor in itself. This theory could, in turn, contain small errors and would 
probably require identifying its hard cases...

According to its author, DragonBox combines Schubfach and Grisu. AFAIK, there's 
no new fundamental underlying theory.
During development of Schubfach, I experimented with a similar mix: on the JVM 
of the time (2018) performance was worse than pure Schubfach and the code more 
complex, so I gave up and switched back to the simpler design, which C2 likes 
better.

Greetings
Raffaello

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

2021-09-28 Thread Raffaello Giulietti

On Fri, 16 Apr 2021 11:30:32 GMT, Raffaello Giulietti 
 wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results

Thanks for helping me keeping this PR alive.

I was pondering whether to propose adding a launch-time option on the command 
line to choose between the current and the proposed implementation for some 
time, so to help the community gaining confidence in the new algorithm and 
still have the current one as a fallback, just in case. (AFAIU, a launch-time 
option can be constant folded by the JIT compiler to help it eliminate dead 
code, right?)

On the one hand, this adds complexity. On the other hand, it could help revive 
this dormant PR.

What do you think? Would this be a viable option?

Greetings
Raffaello

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Integrated: 8271602: Add Math.ceilDiv() family parallel to Math.floorDiv() family

2021-09-22 Thread Raffaello Giulietti

On Wed, 1 Sep 2021 20:13:38 GMT, Raffaello Giulietti 
 wrote:

> This PR ideally continues #5285, which has been closed as a consequence of 
> inadvertently removing the branch on my repo. See there for initial 
> discussion.
> 
> Sorry for the mess.

This pull request has now been integrated.

Changeset: da38ced3
Author:Raffaello Giulietti 
Committer: Brian Burkhalter 
URL:   
https://git.openjdk.java.net/jdk/commit/da38ced3299cbd02f210bea4130990a43f6104bf
Stats: 1093 lines in 4 files changed: 1040 ins; 0 del; 53 mod

8271602: Add Math.ceilDiv() family parallel to Math.floorDiv() family

Reviewed-by: bpb

-

PR: https://git.openjdk.java.net/jdk/pull/5341

Re: RFR: 8271602: Add Math.ceilDiv() family parallel to Math.floorDiv() family [v2]

2021-09-10 Thread Raffaello Giulietti

On Thu, 9 Sep 2021 23:44:49 GMT, Brian Burkhalter  wrote:

>> Raffaello Giulietti has updated the pull request incrementally with one 
>> additional commit since the last revision:
>> 
>>   8271602: Add Math.ceilDiv() family parallel to Math.floorDiv() family
>
> src/java.base/share/classes/java/lang/StrictMath.java line 1465:
> 
>> 1463:  * 
>> 1464:  * The floor modulus is {@code r = x - (ceilDiv(x, y) * y)},
>> 1465:  * has the same sign as the divisor {@code y} or is zero, and
> 
> In line 1465 I think `same` should be `opposite`.

Just pushed the corrections.

-

PR: https://git.openjdk.java.net/jdk/pull/5341

Re: RFR: 8271602: Add Math.ceilDiv() family parallel to Math.floorDiv() family [v2]

2021-09-10 Thread Raffaello Giulietti

> This PR ideally continues #5285, which has been closed as a consequence of 
> inadvertently removing the branch on my repo. See there for initial 
> discussion.
> 
> Sorry for the mess.

Raffaello Giulietti has updated the pull request incrementally with one 
additional commit since the last revision:

  8271602: Add Math.ceilDiv() family parallel to Math.floorDiv() family

-

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/5341/files
  - new: https://git.openjdk.java.net/jdk/pull/5341/files/7afe3cf0..8e368361

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk=5341=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk=5341=00-01

  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.java.net/jdk/pull/5341.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/5341/head:pull/5341

PR: https://git.openjdk.java.net/jdk/pull/5341

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

2021-09-09 Thread Raffaello Giulietti

On Fri, 16 Apr 2021 11:30:32 GMT, Raffaello Giulietti 
 wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results

The usual refresh comment to keep this PR to remain active.
Nothing new.

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: better random numbers

2021-09-07 Thread Raffaello Giulietti





On 2021-09-07 13:48, Stefan Zobel wrote:


On this blog entry (year 2017), Lemire is not giving any technical or
scientific argument in favor or against PCG.

He also refers to, and quotes from, a blog entry (year 2015) of an
influential researcher (whose work he respects) suggesting the entry has
harsh words about PCG. The fact is, that entry doesn't mention PCG or
O'Neill at all and the quotation if not found there.



That "influential researcher" is probably Sebastiano Vigna who has indeed
harsh words on PCG: https://pcg.di.unimi.it/pcg.php



Perhaps.

But the page you refer to is (unfortunately) not dated, except for a 
mention of a year 2020 result from INRIA. Lemire's blog post is from 2017.


Besides, the quotation on Lemire's blog entry is not found here. Two 
search engines don't know about the quotation either, except for 
Lemire's blog.

Re: better random numbers

2021-09-07 Thread Raffaello Giulietti


Hello,


On 2021-09-05 16:43, Andrew Haley wrote:

On 9/3/21 12:35 AM, John Rose wrote:


The reference I’d like to give here is to Dr. Melissa O’Neill’s
website and articles:


I'm quite sceptical. Anyone who says a (non-cryptographic) random-
number generator is "hard to predict" is either quite naive or in a
state of sin, (;-) and while O’Neill’s argument seems sound, it
doesn't seem to have convinced the academic world.

Lemire is thoughtful:
https://lemire.me/blog/2017/08/15/on-melissa-oneills-pcg-random-number-generator/



On this blog entry (year 2017), Lemire is not giving any technical or 
scientific argument in favor or against PCG.


He also refers to, and quotes from, a blog entry (year 2015) of an 
influential researcher (whose work he respects) suggesting the entry has 
harsh words about PCG. The fact is, that entry doesn't mention PCG or 
O'Neill at all and the quotation if not found there.


Looking at Lemire's formal papers, they don't seem to be about PRNGs, 
except for one (curiously written with O'Neill herself in 2019!) about 
statistical tests of variants of Xorshift and Xoroshiro.


I'm not competent on PRNGs at all. Still, I wouldn't rely on Lemire's 
blog entry when it comes to PCG. I'd rather look for other (rare) 
re/sources.






I wonder about AES, which can do (on Apple M1) 2 parallel rounds per
clock cycle. I'm quite tempted to try a reduced- round AES on the
TestU01 statistical tests. Maybe 6 rounds? However, there can be a
long latency between FPU and integer CPU, so perhaps it's not such a
great idea. Also, you have to load the key registers before you can
generate a random number, so it only really works if you want to
generate a lot of bits at a time. But it is maybe 128 randomish bits
per a few clock cycles.

RFR: 8271602: Add Math.ceilDiv() family parallel to Math.floorDiv() family

2021-09-01 Thread Raffaello Giulietti

This PR ideally continues #5285, which has been closed as a consequence of 
inadvertently removing the branch on my repo. See there for initial discussion.

Sorry for the mess.

-

Commit messages:
 - 8271602: Add Math.ceilDiv() family parallel to Math.floorDiv() family

Changes: https://git.openjdk.java.net/jdk/pull/5341/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk=5341=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8271602
  Stats: 1093 lines in 4 files changed: 1040 ins; 0 del; 53 mod
  Patch: https://git.openjdk.java.net/jdk/pull/5341.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/5341/head:pull/5341

PR: https://git.openjdk.java.net/jdk/pull/5341

Withdrawn: 8271602: Add Math.ceilDiv() family parallel to Math.floorDiv() family

2021-09-01 Thread Raffaello Giulietti

On Fri, 27 Aug 2021 18:53:23 GMT, Raffaello Giulietti 
 wrote:

> Please review this PR to add officially endorsed `ceilDiv()` and `ceilMod()` 
> methods do `j.l.[Strict]Math`.
> 
> Beside adding fresh tests to `test/jdk/java/lang/Math/DivModTests.java`, this 
> PR also corrects small typos in it and exercises tests that were already 
> present but which were never invoked.
> Let me know if this is acceptable for a test or if there's a need of a 
> separate issue in the JBS.

This pull request has been closed without being integrated.

-

PR: https://git.openjdk.java.net/jdk/pull/5285

Re: RFR: 8271602: Add Math.ceilDiv() family parallel to Math.floorDiv() family

2021-08-31 Thread Raffaello Giulietti

On Mon, 30 Aug 2021 22:25:09 GMT, Brian Burkhalter  wrote:

>> Please review this PR to add officially endorsed `ceilDiv()` and `ceilMod()` 
>> methods do `j.l.[Strict]Math`.
>> 
>> Beside adding fresh tests to `test/jdk/java/lang/Math/DivModTests.java`, 
>> this PR also corrects small typos in it and exercises tests that were 
>> already present but which were never invoked.
>> Let me know if this is acceptable for a test or if there's a need of a 
>> separate issue in the JBS.
>
> src/java.base/share/classes/java/lang/Math.java line 1501:
> 
>> 1499: // if the signs are the same and modulo not zero, round up
>> 1500: if ((x ^ y) >= 0 && (q * y != x)) {
>> 1501: return q + 1;
> 
> In `floorDiv()` this line is `r--` and there is only one return statement in 
> the method. I think the accepted convention is to return as soon as possible 
> like is done here, so perhaps it would be good to change `floorDiv()` to 
> match? In any cases the two should be consistent.

Yes, I tend to return as soon as possible (btw, it's a q (for quotient) rather 
than a r (for remainder).
I can of course modify the floorDiv() family to match this convention but I 
would like not to open an issue just for that. What is best?

> src/java.base/share/classes/java/lang/Math.java line 1589:
> 
>> 1587:  * 
>> 1588:  *   Regardless of the signs of the arguments, {@code 
>> ceilMod}(x, y)
>> 1589:  *   is zero exactly when {@code x % y} is zero as well.
> 
> Do you intend to say "`ceilMod(x, y)` is zero if and only if `x % y` is 
> zero"? That is to say "exactly when" intends "if and only if"?

This is the same wording as in floorMod().
"If and only if", "exactly when", "precisely when" are usually regarded the 
same afaik. (e.g., see https://en.wikipedia.org/wiki/If_and_only_if)

> src/java.base/share/classes/java/lang/Math.java line 1591:
> 
>> 1589:  *   is zero exactly when {@code x % y} is zero as well.
>> 1590:  *   If neither of {@code ceilMod}(x, y) or {@code x % y} is 
>> zero,
>> 1591:  *   their results differ exactly when the signs of the 
>> arguments are the same.
> 
> I would say "If neither `ceilMod(x, y)`nor `x % y` is zero".  Also same 
> question as above: by "exactly when" do you intend "if any only if"?

OK, but for consistency then even floorMod() should be changed. Would this 
require another JBS issue (or CSR)?

> src/java.base/share/classes/java/lang/Math.java line 1612:
> 
>> 1610: // if the signs are the same and modulo not zero, adjust result
>> 1611: if ((x ^ y) >= 0 && r != 0) {
>> 1612: return r - y;
> 
> Similar comment as for `floorDIv()` above: `floorMod()` does `mod += y` and 
> there is only one return.

Similar comment as for floorDiv() as well.

-

PR: https://git.openjdk.java.net/jdk/pull/5285

Integrated: 8273091: Doc of [Strict]Math.floorDiv(long, int) erroneously documents int in @return tag

2021-08-27 Thread Raffaello Giulietti

On Fri, 27 Aug 2021 20:02:48 GMT, Raffaello Giulietti 
 wrote:

> This PR corrects a typo in the JavaDoc of `[Strict]Math.floorDiv(long,int)`.

This pull request has now been integrated.

Changeset: 51167846
Author:    Raffaello Giulietti 
Committer: Brian Burkhalter 
URL:   
https://git.openjdk.java.net/jdk/commit/51167846cb5a60dfb31b4f8dfa214ba26640175c
Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod

8273091: Doc of [Strict]Math.floorDiv(long,int) erroneously documents int in 
@return tag

Reviewed-by: darcy, bpb

-

PR: https://git.openjdk.java.net/jdk/pull/5287

RFR: 8273091: Doc of [Strict]Math.floorDiv(long, int) erroneously documents int in @return tag

2021-08-27 Thread Raffaello Giulietti

This PR corrects a typo in the JavaDoc of `[Strict]Math.floorDiv(long,int)`.

-

Commit messages:
 - 8273091: Doc of [Strict]Math.floorDiv(long,int) erroneously documents int in 
@return tag

Changes: https://git.openjdk.java.net/jdk/pull/5287/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk=5287=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8273091
  Stats: 2 lines in 2 files changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.java.net/jdk/pull/5287.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/5287/head:pull/5287

PR: https://git.openjdk.java.net/jdk/pull/5287

RFR: 8271602: Add Math.ceilDiv() family parallel to Math.floorDiv() family

2021-08-27 Thread Raffaello Giulietti

Please review this PR to add officially endorsed `ceilDiv()` and `ceilMod()` 
methods do `j.l.[Strict]Math`.

Beside adding fresh tests to `test/jdk/java/lang/Math/DivModTests.java`, this 
PR also corrects small typos in it and exercises tests that were already 
present but which were never invoked.
Let me know if this is acceptable for a test or if there's a need of a separate 
issue in the JBS.

-

Commit messages:
 - 8271602: Add Math.ceilDiv() family parallel to Math.floorDiv() family

Changes: https://git.openjdk.java.net/jdk/pull/5285/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk=5285=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8271602
  Stats: 871 lines in 3 files changed: 850 ins; 0 del; 21 mod
  Patch: https://git.openjdk.java.net/jdk/pull/5285.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/5285/head:pull/5285

PR: https://git.openjdk.java.net/jdk/pull/5285

Integrated: 8271601: Math.floorMod(int, int) and Math.floorMod(long, long) differ in their logic

2021-08-10 Thread Raffaello Giulietti

On Mon, 2 Aug 2021 19:59:57 GMT, Raffaello Giulietti 
 wrote:

> Hello,
> 
> please review this tiny change in the implementation of 
> j.l.Math.floorMod(int, int).
> 
> While the results are unaffected, all of
> floorDiv(int, int)
> floorDiv(long, long)
> floorMod(long, long)
> use x ^ y in the tests to correct the result if needed.
> 
> Not only is the proposed change more consistent with the other methods, but 
> it might benefit later stages in the cpu to proceed with the evaluation of x 
> ^ y in parallel with the previous x % y and, depending of the outcome, even 
> further down.
> 
> 
> Greetings
> Raffaello

This pull request has now been integrated.

Changeset: 66d1faa7
Author:Raffaello Giulietti 
Committer: Brian Burkhalter 
URL:   
https://git.openjdk.java.net/jdk/commit/66d1faa7847b645f20ab2e966adf0a523e3ffeb2
Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod

8271601: Math.floorMod(int, int) and Math.floorMod(long, long) differ in their 
logic

Reviewed-by: bpb

-

PR: https://git.openjdk.java.net/jdk/pull/4962

Integrated: 8271599: Javadoc of floorDiv() and floorMod() families is inaccurate in some places

2021-08-04 Thread Raffaello Giulietti

On Mon, 2 Aug 2021 22:34:58 GMT, Raffaello Giulietti 
 wrote:

> Hello,
> 
> please review the changes in documentation of floorDiv() and floorMod() in 
> all their variants. Some are clarifications, some are corrections.
> 
> Here's an example of a confusing formulation in the current doc
>> "the / operator returns the integer closest to zero"
> 
> Literally, the integer closest to zero is zero!
> 
> 
> The doc of floorMod(int, int) also states:
>> "If the signs of arguments are unknown and a positive modulus
>> is needed it can be computed as (floorMod(x, y) + abs(y)) % abs(y)."
> 
> That's a questionable advice, as the sum in parentheses can lead to 
> irrecoverable overflow (beside requiring two divisions instead of one).
> E.g., setting x = Integer.MAX_VALUE - 1, y = Integer.MAX_VALUE leads to the 
> quite incorrect result -3, which is not even positive!
> 
> 
> Greetings
> Raffaello

This pull request has now been integrated.

Changeset: 9f1edafa
Author:Raffaello Giulietti 
Committer: Brian Burkhalter 
URL:   
https://git.openjdk.java.net/jdk/commit/9f1edafac4f096977ea6ce075ae7a6b0c2112b7d
Stats: 50 lines in 2 files changed: 6 ins; 7 del; 37 mod

8271599: Javadoc of floorDiv() and floorMod() families is inaccurate in some 
places

Reviewed-by: darcy, bpb

-

PR: https://git.openjdk.java.net/jdk/pull/4963

Re: RFR: 8271599: Javadoc of floorDiv() and floorMod() families is inaccurate in some places [v4]

2021-08-04 Thread Raffaello Giulietti

On Wed, 4 Aug 2021 00:57:16 GMT, Brian Burkhalter  wrote:

>> Raffaello Giulietti has updated the pull request incrementally with one 
>> additional commit since the last revision:
>> 
>>   8271599: Javadoc of floorDiv() and floorMod() families is inaccurate in 
>> some places
>
> Marked as reviewed by bpb (Reviewer).

@bplb or @jddarcy could you please file a CSR on the JBS for this PR? As I 
don't have Author status, I'm not entitled to help myself.

Thanks

-

PR: https://git.openjdk.java.net/jdk/pull/4963

Re: RFR: 8271599: Javadoc of floorDiv() and floorMod() families is inaccurate in some places [v4]

> Hello,
> 
> please review the changes in documentation of floorDiv() and floorMod() in 
> all their variants. Some are clarifications, some are corrections.
> 
> Here's an example of a confusing formulation in the current doc
>> "the / operator returns the integer closest to zero"
> 
> Literally, the integer closest to zero is zero!
> 
> 
> The doc of floorMod(int, int) also states:
>> "If the signs of arguments are unknown and a positive modulus
>> is needed it can be computed as (floorMod(x, y) + abs(y)) % abs(y)."
> 
> That's a questionable advice, as the sum in parentheses can lead to 
> irrecoverable overflow (beside requiring two divisions instead of one).
> E.g., setting x = Integer.MAX_VALUE - 1, y = Integer.MAX_VALUE leads to the 
> quite incorrect result -3, which is not even positive!
> 
> 
> Greetings
> Raffaello

Raffaello Giulietti has updated the pull request incrementally with one 
additional commit since the last revision:

  8271599: Javadoc of floorDiv() and floorMod() families is inaccurate in some 
places

-

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4963/files
  - new: https://git.openjdk.java.net/jdk/pull/4963/files/dc81446f..21e4f370

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk=4963=03
 - incr: https://webrevs.openjdk.java.net/?repo=jdk=4963=02-03

  Stats: 7 lines in 2 files changed: 0 ins; 0 del; 7 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4963.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4963/head:pull/4963

PR: https://git.openjdk.java.net/jdk/pull/4963

Re: RFR: 8271599: Javadoc of floorDiv() and floorMod() families is inaccurate in some places [v3]

> Hello,
> 
> please review the changes in documentation of floorDiv() and floorMod() in 
> all their variants. Some are clarifications, some are corrections.
> 
> Here's an example of a confusing formulation in the current doc
>> "the / operator returns the integer closest to zero"
> 
> Literally, the integer closest to zero is zero!
> 
> 
> The doc of floorMod(int, int) also states:
>> "If the signs of arguments are unknown and a positive modulus
>> is needed it can be computed as (floorMod(x, y) + abs(y)) % abs(y)."
> 
> That's a questionable advice, as the sum in parentheses can lead to 
> irrecoverable overflow (beside requiring two divisions instead of one).
> E.g., setting x = Integer.MAX_VALUE - 1, y = Integer.MAX_VALUE leads to the 
> quite incorrect result -3, which is not even positive!
> 
> 
> Greetings
> Raffaello

Raffaello Giulietti has updated the pull request incrementally with one 
additional commit since the last revision:

  8271599: Javadoc of floorDiv() and floorMod() families is inaccurate in some 
places

-

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4963/files
  - new: https://git.openjdk.java.net/jdk/pull/4963/files/cab4c54c..dc81446f

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk=4963=02
 - incr: https://webrevs.openjdk.java.net/?repo=jdk=4963=01-02

  Stats: 2 lines in 1 file changed: 0 ins; 0 del; 2 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4963.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4963/head:pull/4963

PR: https://git.openjdk.java.net/jdk/pull/4963

Re: RFR: 8271599: Javadoc of floorDiv() and floorMod() families is inaccurate in some places [v2]

> Hello,
> 
> please review the changes in documentation of floorDiv() and floorMod() in 
> all their variants. Some are clarifications, some are corrections.
> 
> Here's an example of a confusing formulation in the current doc
>> "the / operator returns the integer closest to zero"
> 
> Literally, the integer closest to zero is zero!
> 
> 
> The doc of floorMod(int, int) also states:
>> "If the signs of arguments are unknown and a positive modulus
>> is needed it can be computed as (floorMod(x, y) + abs(y)) % abs(y)."
> 
> That's a questionable advice, as the sum in parentheses can lead to 
> irrecoverable overflow (beside requiring two divisions instead of one).
> E.g., setting x = Integer.MAX_VALUE - 1, y = Integer.MAX_VALUE leads to the 
> quite incorrect result -3, which is not even positive!
> 
> 
> Greetings
> Raffaello

Raffaello Giulietti has updated the pull request incrementally with one 
additional commit since the last revision:

  8271599: Javadoc of floorDiv() and floorMod() families is inaccurate in some 
places

-

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/4963/files
  - new: https://git.openjdk.java.net/jdk/pull/4963/files/423af129..cab4c54c

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk=4963=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk=4963=00-01

  Stats: 21 lines in 2 files changed: 2 ins; 0 del; 19 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4963.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4963/head:pull/4963

PR: https://git.openjdk.java.net/jdk/pull/4963

Re: RFR: 8271225: Add floorDivExact() method to java.lang.[Strict]Math

On Thu, 29 Jul 2021 23:28:07 GMT, Brian Burkhalter  wrote:

>> Add methods `floorDivExact(int,int)` and `floorDivExact(long,long)` to 
>> `java.lang.Math` and `java.lang.StrictMath`.
>
> The `floorDivExact()` methods are identical to their `floorDiv()` 
> counterparts aside from that they throw an `ArithmeticException` when the 
> dividend is `MIN_VALUE` and the divisor is `-1`. These methods behave with 
> respect to the methods `floorDiv()` as the methods `divideExact()` behave 
> with respect to the division operator `/`.

Hello @bplb,

the doc of floorDivExact() is not, well..., exact.

For example,
> The floor rounding mode gives different results from truncation
when the exact result is negative.

The truth is that the results are different when the exact quotient _is not an 
integer and is negative_.
The description following this sentence is imprecise as well.

The doc seems to have been copied and adapted from the doc of floorDiv(), which 
is imprecise in the first place.
You might want to take a look at [PR 
4963](https://github.com/openjdk/jdk/pull/4963) which addresses this and wait 
until that one is thoroughly discussed for mathematical accuracy and correct 
wording.

Greetings
Raffaello

-

PR: https://git.openjdk.java.net/jdk/pull/4941

RFR: 8271599: Javadoc of floorDiv() and floorMod() families is inaccurate in some places

2021-08-02 Thread Raffaello Giulietti

Hello,

please review the changes in documentation of floorDiv() and floorMod() in all 
their variants. Some are clarifications, some are corrections.

Here's an example of a confusing formulation in the current doc
> "the / operator returns the integer closest to zero"

Literally, the integer closest to zero is zero!


The doc of floorMod(int, int) also states:
> "If the signs of arguments are unknown and a positive modulus
> is needed it can be computed as (floorMod(x, y) + abs(y)) % abs(y)."

That's a questionable advice, as the sum in parentheses can lead to 
irrecoverable overflow (beside requiring two divisions instead of one).
E.g., setting x = Integer.MAX_VALUE - 1, y = Integer.MAX_VALUE leads to the 
quite incorrect result -3, which is not even positive!


Greetings
Raffaello

-

Commit messages:
 - 8271599: Javadoc of floorDiv() and floorMod() families is inaccurate in some 
places

Changes: https://git.openjdk.java.net/jdk/pull/4963/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk=4963=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8271599
  Stats: 31 lines in 1 file changed: 4 ins; 7 del; 20 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4963.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4963/head:pull/4963

PR: https://git.openjdk.java.net/jdk/pull/4963

RFR: 8271601: Math.floorMod(int, int) and Math.floorMod(long, long) differ in their logic

2021-08-02 Thread Raffaello Giulietti

Hello,

please review this tiny change in the implementation of j.l.Math.floorMod(int, 
int).

While the results are unaffected, all of
floorDiv(int, int)
floorDiv(long, long)
floorMod(long, long)
use x ^ y in the tests to correct the result if needed.

Not only is the proposed change more consistent with the other methods, but it 
might benefit later stages in the cpu to proceed with the evaluation of x ^ y 
in parallel with the previous x % y and, depending of the outcome, even further 
down.


Greetings
Raffaello

-

Commit messages:
 - 8271601: Math.floorMod(int, int) and Math.floorMod(long, long) differ in 
their logic

Changes: https://git.openjdk.java.net/jdk/pull/4962/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk=4962=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8271601
  Stats: 1 line in 1 file changed: 0 ins; 0 del; 1 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4962.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4962/head:pull/4962

PR: https://git.openjdk.java.net/jdk/pull/4962

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

2021-07-16 Thread Raffaello Giulietti

On Fri, 16 Apr 2021 11:30:32 GMT, Raffaello Giulietti 
 wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results

This is just to keep [1] alive. No news.

[1] https://github.com/openjdk/jdk/pull/3402

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 8075806: divideExact is missing in java.lang.Math

2021-07-14 Thread Raffaello Giulietti


Mysteries of JIT compilation...


On 2021-07-14 21:53, Brian Burkhalter wrote:

On Tue, 13 Jul 2021 17:21:52 GMT, Brian Burkhalter  wrote:


Please consider this proposal to add `divideExact()` methods for integral data types to 
`java.lang.Math` thereby rounding out "exact" support to all four basic 
arithmetic operations.


@rgiulietti Actually your trick is the fastest: `yours > via negateExact() > 
original`. Thanks!

-

PR: https://git.openjdk.java.net/jdk/pull/4770

Re: RFR: 8075806: divideExact is missing in java.lang.Math

2021-07-14 Thread Raffaello Giulietti


Sorry, typo

int q = x / y;
if ((x & y & q) >= 0) {
 return q;  // q, not r
}
throw new ArithmeticException("integer overflow");


On 2021-07-14 21:26, Raffaello Giulietti wrote:

One could also divide first and then check the result:

int q = x / y;
if ((x & y & q) >= 0) {
     return r;
}
throw new ArithmeticException("integer overflow");

No idea about relative perf.



On 2021-07-13 19:32, Brian Burkhalter wrote:
Please consider this proposal to add `divideExact()` methods for 
integral data types to `java.lang.Math` thereby rounding out "exact" 
support to all four basic arithmetic operations.


-

Commit messages:
  - 8075806: divideExact is missing in java.lang.Math

Changes: https://git.openjdk.java.net/jdk/pull/4770/files
  Webrev: https://webrevs.openjdk.java.net/?repo=jdk=4770=00
   Issue: https://bugs.openjdk.java.net/browse/JDK-8075806
   Stats: 109 lines in 2 files changed: 100 ins; 5 del; 4 mod
   Patch: https://git.openjdk.java.net/jdk/pull/4770.diff
   Fetch: git fetch https://git.openjdk.java.net/jdk 
pull/4770/head:pull/4770


PR: https://git.openjdk.java.net/jdk/pull/4770

Re: RFR: 8075806: divideExact is missing in java.lang.Math

2021-07-14 Thread Raffaello Giulietti


One could also divide first and then check the result:

int q = x / y;
if ((x & y & q) >= 0) {
return r;
}
throw new ArithmeticException("integer overflow");

No idea about relative perf.



On 2021-07-13 19:32, Brian Burkhalter wrote:

Please consider this proposal to add `divideExact()` methods for integral data types to 
`java.lang.Math` thereby rounding out "exact" support to all four basic 
arithmetic operations.

-

Commit messages:
  - 8075806: divideExact is missing in java.lang.Math

Changes: https://git.openjdk.java.net/jdk/pull/4770/files
  Webrev: https://webrevs.openjdk.java.net/?repo=jdk=4770=00
   Issue: https://bugs.openjdk.java.net/browse/JDK-8075806
   Stats: 109 lines in 2 files changed: 100 ins; 5 del; 4 mod
   Patch: https://git.openjdk.java.net/jdk/pull/4770.diff
   Fetch: git fetch https://git.openjdk.java.net/jdk pull/4770/head:pull/4770

PR: https://git.openjdk.java.net/jdk/pull/4770

Re: RFR: 8188044: We need Math.unsignedMultiplyHigh [v2]

2021-07-02 Thread Raffaello Giulietti


FWIW, adinn's branchless code together with
PR https://git.openjdk.java.net/jdk/pull/4660
make both methods less vulnerable to timing attacks.


Greetings
Raffaello


On 2021-07-02 15:50, Andrew Haley wrote:

On Fri, 2 Jul 2021 11:06:06 GMT, Andrew Dinn  wrote:


You can also do that branchlessly which might prove better

```
  long result = Math.multiplyHigh(x, y);
  result += (y & (x >> 63));
  result += (x & (y >> 63));
  return result;
```

I doubt very much that it would be better, because these days branch prediction 
is excellent, and we also have conditional select instructions. Exposing the 
condition helps C2 to eliminate it if the range of args is known. The `if` code 
is easier to understand.

Benchmark results, with one of the operands changing signs every iteration, 
1000 iterations:


Benchmark  Mode  Cnt ScoreError  Units
MulHiTest.mulHiTest1   (aph) avgt3  1570.587 ± 16.602  ns/op
MulHiTest.mulHiTest2   (adinn)   avgt3  2237.637 ±  4.740  ns/op

In any case, note that with this optimization the unsigned mulHi is in the 
nanosecond range, so Good Enough. IMO.

-

PR: https://git.openjdk.java.net/jdk/pull/4644

Re: RFR: 8188044: We need Math.unsignedMultiplyHigh [v2]

2021-07-02 Thread Raffaello Giulietti


... or even as a one liner, like in the test

	return Math.multiplyHigh(x, y) + ((x >> (Long.SIZE - 1)) & y) + ((y >> 
(Long.SIZE - 1)) & x);




On 2021-07-02 13:08, Andrew Dinn wrote:

On Fri, 2 Jul 2021 09:39:46 GMT, Andrew Haley  wrote:


src/java.base/share/classes/java/lang/Math.java line 1211:


1209: long z1 = t >>> 32;
1210:
1211: return x1 * y1 + z1 + (z0 >>> 32);


Suggestion:

 long result = Math.multiplyHigh(x, y);
 if (x < 0) result += y;
 if (y < 0) result += x;
 return result;


This is just subtracting the 2's-complement offset. I guess the idea, longer 
term, is that this be an intrinsic anyway, but if you do `unsignedMultiplyHigh` 
this way you'll utilize the existing `multiplyHigh` intrinsic on all platforms 
that have it.


You can also do that branchlessly which might prove better

  long result = Math.multiplyHigh(x, y);
  result += (y & (x >> 63));
  result += (x & (y >> 63));
  return result;

-

PR: https://git.openjdk.java.net/jdk/pull/4644

Re: Comment on JDK-826722: SoftReference not cleared on OutOfMemoryError: Requested array size exceeds VM limit

2021-06-04 Thread Raffaello Giulietti


Hi Mandy,

the OOME thrown for VM limits reasons is not related to any purported 
heap exhaustion but to the VM refusing to allocate an array of size 
Integer.MAX_VALUE or Integer.MAX_VALUE - 1, *even* if there's plenty of 
space.


For example, with 8 GiB of heap and a size of Integer.MAX_VALUE - 2 the 
small program runs without fuss:


java -XX:+UseG1GC -Xms8g -Xmx8g -cp ... Softly 2147483645


But when the argument is increased by 1 to Integer.MAX_VALUE - 1

java -XX:+UseG1GC -Xms8g -Xmx8g -cp ... Softly 2147483646

you immediately get:

Exception in thread "main" java.lang.AssertionError: non-null referent
at Softly.main(Softly.java:26)
Caused by: java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at Softly.main(Softly.java:15)


In other words, OOME is "abused" in such cases. A VM limit error should 
really throw another kind of error, not OOME, because contrary to its 
name there's not necessarily a lack of memory space, as shown here.



To parallel current behavior, thus, the spec should be amended as 
proposed "... an OutOfMemoryError caused by Java heap space exhaustion." 
or a similar wording.
Alternatively, to maintain the current spec untouched, the VM should 
throw another kind of error for VM limits. Not sure if this has any 
adverse impact on existing code in the wild.



Greetings
Raffaello




On 2021-06-04 21:16, Mandy Chung wrote:
I'm not sure if the spec should be updated.  JDK-8267222 needs the GC 
team to evaluate.


I have added my comment in this JBS issue.

The SoftReference spec has the guarantee:
    “All soft references to softly-reachable objects are guaranteed to 
have been cleared before the virtual machine throws an OutOfMemoryError.”


This is a reasonable guarantee expected by design in response to memory 
demand.


For the OOME thrown due to "requested array size exceeds VM limit", it 
seems that this is a fast-path throwing OOME without really going 
through the object allocation request (where reference processing will 
be performed in GC cycle).


The question to the GC team is whether VM implementation can and should 
support this soft reference guarantee. Note that the soft reference 
objects are cleared as specified, the large object allocation exceeding 
VM limit would fail any way. If the implementation is feasible, I'm 
inclined to clear the soft reference objects when OOME is thrown as 
specified even the object allocation request is known to fail.


Mandy

On 6/3/21 11:57 AM, Raffaello Giulietti wrote:

Hi,

upon reading [1] I tried a similar scenario, but where OOME are caused 
by "Java heap space" exhaustion rather than by VM limits.



import java.lang.ref.SoftReference;
import java.text.DecimalFormat;
import java.util.ArrayList;

public class Softly {

    public static void main(String[] args) {
    var size = Integer.parseInt(args[0]);
    var format = new DecimalFormat("#,###");
    var news = 0;
    var ref = new SoftReference<>(new ArrayList<>());
    for (;;) {
    byte[] b = null;
    try {
    b = new byte[size];
    ++news;
    ref.get().add(b);
    } catch (NullPointerException __) {
    System.out.format("totSize = %20s, allocations = 
%d\n", format.format((long) news * size), news);

    ref = new SoftReference<>(new ArrayList<>());
    ref.get().add(b);
    } catch (OutOfMemoryError e) {
    if (ref.refersTo(null)) {
    throw new AssertionError("allocations = 
%d".formatted((news)), e);

    }
    throw new AssertionError("non-null referent", e);
    }
    }
    }

}


E.g.,
java -XX:+UseG1GC -Xms1g -Xmx1g -cp ... Softly 8


Depending on the collector and how tight the heap is, I sometimes 
observe a "Java heap space" OOME but then the referent of ref is null. 
I never observed a OOME with a non-null referent for ref. Hence, in 
scenarios where OOME are caused by heap exhaustion, soft refs seem to 
work as advertised.


Tried on AdoptOpenJDK-16.0.1+9 with SerialGC, ParallelGC, G1GC, ZGC 
and ShenandoahGC with either -Xms1g/-Xmx1g or -Xms2g/-Xmx2g (small 
heaps) and various byte[] sizes.


Thus, the current wording in SoftReference's javadoc:

"All soft references to softly-reachable objects are guaranteed to 
have been cleared before the virtual machine throws an OutOfMemoryError."


could be amended to read:

"All soft references to softly-reachable objects are guaranteed to 
have been cleared before the virtual machine throws an 
OutOfMemoryError caused by Java heap space exhaustion."



Greetings
Raffaello



[1] https://bugs.openjdk.java.net/browse/JDK-8267222

Comment on JDK-826722: SoftReference not cleared on OutOfMemoryError: Requested array size exceeds VM limit

2021-06-03 Thread Raffaello Giulietti


Hi,

upon reading [1] I tried a similar scenario, but where OOME are caused 
by "Java heap space" exhaustion rather than by VM limits.



import java.lang.ref.SoftReference;
import java.text.DecimalFormat;
import java.util.ArrayList;

public class Softly {

public static void main(String[] args) {
var size = Integer.parseInt(args[0]);
var format = new DecimalFormat("#,###");
var news = 0;
var ref = new SoftReference<>(new ArrayList<>());
for (;;) {
byte[] b = null;
try {
b = new byte[size];
++news;
ref.get().add(b);
} catch (NullPointerException __) {
System.out.format("totSize = %20s, allocations = %d\n", 
format.format((long) news * size), news);

ref = new SoftReference<>(new ArrayList<>());
ref.get().add(b);
} catch (OutOfMemoryError e) {
if (ref.refersTo(null)) {
throw new AssertionError("allocations = 
%d".formatted((news)), e);

}
throw new AssertionError("non-null referent", e);
}
}
}

}


E.g.,
java -XX:+UseG1GC -Xms1g -Xmx1g -cp ... Softly 8


Depending on the collector and how tight the heap is, I sometimes 
observe a "Java heap space" OOME but then the referent of ref is null. I 
never observed a OOME with a non-null referent for ref. Hence, in 
scenarios where OOME are caused by heap exhaustion, soft refs seem to 
work as advertised.


Tried on AdoptOpenJDK-16.0.1+9 with SerialGC, ParallelGC, G1GC, ZGC and 
ShenandoahGC with either -Xms1g/-Xmx1g or -Xms2g/-Xmx2g (small heaps) 
and various byte[] sizes.


Thus, the current wording in SoftReference's javadoc:

"All soft references to softly-reachable objects are guaranteed to have 
been cleared before the virtual machine throws an OutOfMemoryError."


could be amended to read:

"All soft references to softly-reachable objects are guaranteed to have 
been cleared before the virtual machine throws an OutOfMemoryError 
caused by Java heap space exhaustion."



Greetings
Raffaello



[1] https://bugs.openjdk.java.net/browse/JDK-8267222

Integrated: 8268077: java.util.List missing from Collections Framework Overview

2021-06-03 Thread Raffaello Giulietti

On Tue, 1 Jun 2021 19:51:39 GMT, Raffaello Giulietti 
 wrote:

> Trivial changes to this non-normative document.

This pull request has now been integrated.

Changeset: 5405f983
Author:    Raffaello Giulietti 
Committer: Stuart Marks 
URL:   
https://git.openjdk.java.net/jdk/commit/5405f983db7d359bb65c42366541104c5e9ef7c3
Stats: 7 lines in 1 file changed: 2 ins; 0 del; 5 mod

8268077: java.util.List missing from Collections Framework Overview

Reviewed-by: smarks

-

PR: https://git.openjdk.java.net/jdk/pull/4289

RFR: 8268077: java.util.List missing from Collections Framework Overview

2021-06-01 Thread Raffaello Giulietti

Trivial changes to this non-normative document.

-

Commit messages:
 - 8268077: java.util.List missing from Collections Framework Overview

Changes: https://git.openjdk.java.net/jdk/pull/4289/files
 Webrev: https://webrevs.openjdk.java.net/?repo=jdk=4289=00
  Issue: https://bugs.openjdk.java.net/browse/JDK-8268077
  Stats: 7 lines in 1 file changed: 2 ins; 0 del; 5 mod
  Patch: https://git.openjdk.java.net/jdk/pull/4289.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/4289/head:pull/4289

PR: https://git.openjdk.java.net/jdk/pull/4289

Re: java.util.List missing from "Collections Framework Overview" javadoc

2021-06-01 Thread Raffaello Giulietti


Hi,

can anybody take a look at this?
I would do it myself but I don't have Author status to add an issue to 
the JBS.


Should be a trivial change.

Thanks
Raffaello


On 2021-05-30 18:03, Raffaello Giulietti wrote:

Hello,

seems like java.util.List is missing from the list in the "Collection 
Interfaces" section in
https://docs.oracle.com/en/java/javase/16/docs/api/java.base/java/util/doc-files/coll-overview.html 



Should be easy to fix and doesn't seem to require a CSR as the document 
is non-normative.


Greetings
Raffaello

java.util.List missing from "Collections Framework Overview" javadoc

2021-05-30 Thread Raffaello Giulietti


Hello,

seems like java.util.List is missing from the list in the "Collection 
Interfaces" section in

https://docs.oracle.com/en/java/javase/16/docs/api/java.base/java/util/doc-files/coll-overview.html

Should be easy to fix and doesn't seem to require a CSR as the document 
is non-normative.


Greetings
Raffaello

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

2021-05-25 Thread Raffaello Giulietti

On Fri, 16 Apr 2021 11:30:32 GMT, Raffaello Giulietti 
 wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> Raffaello Giulietti has updated the pull request incrementally with one 
> additional commit since the last revision:
> 
>   4511638: Double.toString(double) sometimes produces incorrect results

Hi,

a reminder to keep PR [1] alive and to invite interested reviewers to 
comment it.

The corresponding CSR is in [2]


Greetings
Raffaello

---

[1] https://github.com/openjdk/jdk/pull/3402
[2] https://bugs.openjdk.java.net/browse/JDK-8202555

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Reminder RFR: 4511638: Double.toString(double) sometimes produces incorrect results

2021-05-25 Thread Raffaello Giulietti


Hi,

a reminder to keep PR [1] alive and to invite interested reviewers to 
comment it.


The corresponding CSR is in [2]


Greetings
Raffaello

---

[1] https://github.com/openjdk/jdk/pull/3402
[2] https://bugs.openjdk.java.net/browse/JDK-8202555

Re: New java.util.Strings class

2021-05-22 Thread Raffaello Giulietti


Hi Alberto,

there's an understandable tendency to wish to add convenience/utility 
methods like the ones you are proposing. Often, however, the perceived 
benefits are not assessed accurately. In other words, convenience 
methods in core libraries must be of real general interest.


Coming to your class, it exposes the constant EMPTY_STRING for "". Now, 
this is 12 characters (or even more if you don't use static imports) 
against 2 for nothing in exchange: "" is very readable, is not a magic 
constant that would deserve a global name, is interned and comes exactly 
to the point. Finding/replacing all occurrences of "" in a code base, if 
so needed, is easy even with the most elementary tools.


Some methods are there just to come around null strings. Now, a properly 
written application rarely uses null strings and if that happens it is 
mostly because of logical errors. The proposed methods hide them instead 
of pointing at their cause by throwing, so would even be harmful if used 
there. Thus, the general usefulness of such methods is probably more 
limited than intended by the proposal.


Other methods are focused on empty strings or strings of whitespaces. 
Agreed, some business logic would have to treat them specially and could 
benefit, but what's so outstanding with them to deserve API points in a 
core lib? In addition, some methods invoke String::strip() just to test 
whether a string is whitespaces.


This is not to say that your methods are not useful, only that they are 
probably not *that* useful to be included as a core lib.


A great post about the tension between the desire to add convenience 
methods to the core libs and try to refrain from doing so:

https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-May/077103.html


Greetings
Raffaello


On 2021-05-21 11:11, Alberto Otero Rodríguez wrote:

Hi,

I have made other changes to the Strings class I proposed in my previous 
messages.

The changes are:

   *   Added the new methods compareTo and compareToIgnoreCase
   *   Changed WhiteSpace for Whitespace

You can see the new code here:
https://github.com/Aliuken/Java-Strings/blob/main/Strings.java

With those changes, the annotations suggested in the previous message should 
change to:
- @NonNullNorWhitespace
- @NonNullNorWhitespaceElse(defaultValue)
- @NonNullNorWhitespaceElseGet(class::supplierMethod)

Regards,

Alberto Otero Rodríguez.

De: Alberto Otero Rodríguez 
Enviado: miércoles, 19 de mayo de 2021 11:36
Para: core-libs-dev@openjdk.java.net 
Asunto: RE: New java.util.Strings class

Hi,

I have made some changes to the Strings class I proposed in my previous message.

The changes are:

   *   Use str.isEmpty() instead of EMPTY_STRING.equals(str)
   *   Create methods using str.strip() to check a String is not Whitespace

You can see the new code here:
https://github.com/Aliuken/Java-Strings/blob/main/Strings.java

With those changes, the following annotations could also be created for method 
arguments and return types:
- @NonNullNorWhiteSpace
- @NonNullNorWhiteSpaceElse(defaultValue)
- @NonNullNorWhiteSpaceElseGet(class::supplierMethod)

I didn't have any response to the previous message.

Please, take this one in consideration.

Regards,

Alberto Otero Rodríguez.


De: Alberto Otero Rodríguez
Enviado: lunes, 17 de mayo de 2021 14:58
Para: core-libs-dev@openjdk.java.net 
Asunto: New java.util.Strings class

Hi, members of the core-libs developers of OpenJDK.

I think Java needs a Strings class similar to the java.util.Objects class of 
Java 16 to be used in method arguments, return types and Stream filters.

I have coded it myself here based on the Objects implementation of Java 16 
(please have a look):
https://github.com/Aliuken/Java-Strings/blob/main/Strings.java

In fact, I think it would be useful also adding annotations for method 
arguments and return types such as:
- @NonNull
- @NonNullElse(defaultValue)
- @NonNullElseGet(class::supplierMethod)
- @NonNullNorEmpty
- @NonNullNorEmptyElse(defaultValue)
- @NonNullNorEmptyElseGet(class::supplierMethod)

With that kind of annotations, you could assume thinks like:
- an argument or return type cannot have value null, but an Exception
- an argument or return type cannot have value null, but a default value

What do you think?

Waiting for your response.

PS: I am sending this email repeated. I have sended it yesterday with my other 
email account (alber8...@gmail.com), but I wasn't a member of this mailing 
list. Now I have become a member with this other email account.

Regards,

Alberto Otero Rodríguez.

Re: Fast and cheap (Double|Float)::toString Java algorithm

2021-05-04 Thread Raffaello Giulietti


Hi Suminda,

as I explained back in February, I already experimented blending 
Schubfach with a variant of Grisu back in 2018. Contrary to my 
expectations, I observed noticeable performance drops wrt pure 
Schubfach. I didn't explore any further, but I think that the more 
complex control logic required by the blend makes it harder for a JIT 
compiler to optimize.


Thus, I prefer to wait until my own contribution has been reviewed and 
integrated (if at all) before adding complexity that might not be 
beneficial in Java.


(Junekey measured that a blend *is* beneficial in C++, though. But it's 
somehow hard to compare a static AOT compilation, as done in C++, and a 
JIT compilation, as done on the JVM.)



Greetings
Raffaello



On 2021-05-04 18:14, Suminda Sirinath Salpitikorala Dharmasena wrote:

Hello,

I hope everything is well with you.

Due to other commitments this work has stalled.

I was planning to implement:
- fast to string conversion
- fast string parsing
- fast formatting
- fast search
- fast sort
- fast templating
- fast buffers

The code I have done so far is here: https://github.com/sirinath/moby 
<https://github.com/sirinath/moby>


Wondering if someone can take this forward until I have more time in my 
hand.


Suminda

On Sat, 6 Feb 2021 at 09:50, Suminda Sirinath Salpitikorala Dharmasena 
mailto:sirinath19...@gmail.com>> wrote:


Hello,

I am working on a port of DragonBox to Java. If there is interest in
contributing this is most welcome.

Suminda

On Fri, 5 Feb 2021 at 21:51, Raffaello Giulietti
mailto:raffaello.giulie...@gmail.com>> wrote:

Hello,

as a reminder, a Java implementation of Schubfach [1] intended to
replace the current slow and expensive OpenJDK
(Double|Float)::toString
algorithm has been discussed, presented and contributed to this
mailing
list in several posts. The last implementation is available in
pre-Skara
webrev form, as referenced in [2]. On my laptop hardware, the
speedup
factor is 17.7x wrt OpenJDK.



Recently, Alexander Bolz translated Schubfach to C++ for the
purpose of
comparing the performance of several bin2dec floating-point numbers
algorithms [3].

In the meantime, Junekey Jeon implemented and perfected his own
Dragonbox in C++, a new algorithm based on Schubfach [4]. From
[5]: "In
addition to the core idea of Schubfach, Dragonbox utilizes some
Grisu-like ideas to minimize the number of expensive 128-bit ×
64-bit
multiplications, at the cost of having more branches and
divisions-by-constants."

In the C++ ecosystem, Schubfach has been the fastest known
algorithm
before being gently pushed aside by Dragonbox, as can be seen in
the
graphs in [3].



While developing Schubfach back in 2018, I experimented myself with
blending core Schubfach with my own earlier algorithm similar to
Grisu.
However, probably due to uncontrollable JIT compilation behavior, I
could not observe any benefit. On the contrary, I measured
performance
drops probably because of the added complexity, which is public
enemy
nr. 1 for JIT compilers. Therefore, I opted for the simpler current
design that seemed more suitable for (the then 2018 version of) C2.



I hope this can somehow revive this community's interest and
confidence
toward Schubfach to definitely supplant the current expensive
(Double|Float)::toString algorithm.


Greetings
Raffaello



[1]
https://drive.google.com/open?id=1luHhyQF9zKlM8yJ1nebU0OgVYhfC6CBN
<https://drive.google.com/open?id=1luHhyQF9zKlM8yJ1nebU0OgVYhfC6CBN>
[2]

https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065297.html

<https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-March/065297.html>
[3] https://github.com/abolz/Drachennest
<https://github.com/abolz/Drachennest>
[4] https://github.com/jk-jeon/dragonbox
<https://github.com/jk-jeon/dragonbox>
[5]

https://github.com/jk-jeon/dragonbox/blob/master/other_files/Dragonbox.pdf

<https://github.com/jk-jeon/dragonbox/blob/master/other_files/Dragonbox.pdf>

Integrated: 8264514: HexFormat implementation tweaks

2021-04-22 Thread Raffaello Giulietti

On Wed, 7 Apr 2021 18:49:36 GMT, Raffaello Giulietti 
 wrote:

> (Changed to new branch in personal fork)
> 
> Please review these small tweaks.
> For background information see 
> [1](https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-March/075822.html)
> 
> Greetings
> Raffaello

This pull request has now been integrated.

Changeset: fa82d475
Author:Raffaello Giulietti 
Committer: Roger Riggs 
URL:   https://git.openjdk.java.net/jdk/commit/fa82d475
Stats: 67 lines in 1 file changed: 10 ins; 26 del; 31 mod

8264514: HexFormat implementation tweaks

Reviewed-by: rriggs

-

PR: https://git.openjdk.java.net/jdk/pull/3381

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

2021-04-19 Thread Raffaello Giulietti


Hi Andrey,

thanks for your careful reading.

I'll keep a note and collect yours with changes coming from other 
reviewers before committing a larger batch of small changes. I would 
like to avoid wasting a lot of energy right now just to rebuild 
everything and to run the automatic tests on GitHub :-)



Greetings
Raffaello


On 2021-04-18 23:19, Andrey Turbanov wrote:

On Fri, 16 Apr 2021 11:30:32 GMT, Raffaello Giulietti 
 wrote:


Hello,

here's a PR for a patch submitted on March 2020 
[1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was a 
thing.

The patch has been edited to adhere to OpenJDK code conventions about 
multi-line (block) comments. Nothing in the code proper has changed, except for 
the addition of redundant but clarifying parentheses in some expressions.


Greetings
Raffaello


Raffaello Giulietti has updated the pull request incrementally with one 
additional commit since the last revision:

   4511638: Double.toString(double) sometimes produces incorrect results


src/java.base/share/classes/jdk/internal/math/DoubleToDecimal.java line 69:


67:
68: /* 10^(E_MIN - 1) <= MIN_VALUE < 10^E_MIN */
69: static final int E_MIN = -323;


It seems that `E_MIN`/`E_MAX`/`K_MIN`/`K_MAX`  are not used in production code.
I think it worth to move them to tests.

src/java.base/share/classes/jdk/internal/math/DoubleToDecimal.java line 117:


115:  * where there are H digits d
116:  */
117: public final int MAX_CHARS = H + 7;


Can it be made `static` ?

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: 8252827: Caching Integer.toString just like Integer.valueOf

2021-04-17 Thread Raffaello Giulietti

Hi,

in view of Integer becoming a primitive class [1], the IntegerCache is 
probably going to disappear.

For a small, fixed range like the one you are proposing [-1, 16], 
there's no real need for a separate cache class. You could have a switch 
in the implementation of toString(), with the string literals then being 
part of the constant pool of the class. Not free beer, but supported by 
the VM since day 0.

It's only when the range is open that you'd need a cache similar to 
IntegerCache.

My 2 cents as well :-)
Raffaello

[1] https://openjdk.java.net/jeps/402

On 2021-04-17 11:18, Laurent Bourgès wrote:

Hi,

I read the JBS bug and I interpret it as:
- IntegerCache provides Integer instances for [-128, 127] by default
- Having Integer.toString(int) could behave the same or at least cache 
most probable values like [-1 to 16] or using the IntegerCache range.

It looks trivial and potentially could benefits to jdk itself to reduce 
memory garbage : is Integer.toString(int) widely used in the jdk codebase ?

Finally it can be alternatively implemented in application's code.

My 2 cents,
Laurent

Le sam. 17 avr. 2021 à 11:06, Raffaello Giulietti 
mailto:raffaello.giulie...@gmail.com>> a 
écrit :

On 2021-04-17 07:07, David Holmes wrote:
 > On 17/04/2021 4:54 am, Raffaello Giulietti wrote:
 >> I guess the reporter meant to limit the cache range similarly to
the
 >> one used for valueOf().
 >>
 >> I have no clue about the benefit/cost ratio for the proposed String
 >> cache. It really depends on usage, workload, etc. One can easily
 >> imagine both extreme scenarios but it's hard to tell how the
average
 >> one would look.
 >>
 >> My post is only about either solving the issue by implementing the
 >> cache, which I can contribute to; or closing it because of lack of
 >> real-world need or interest.
 >
 > Caching for the sake of caching is not an objective in itself.
Unless
 > the caching can be shown to solve a real problem, and the
strategy for
 > managing the cache is well-defined, then I would just close the
 > enhancement request. (Historically whether an issue we don't have
any
 > firm plans to address is just left open "forever" or closed, depends
 > very much on who does the bug triaging in that area. :) )
 >
 > Cheers,
 > David
 >

Indeed, the impression is that many of the issues are probably open
because it's unclear whether they should be addressed with some
implementation or spec effort or whether they should be closed.
Triaging
is certainly a harder job than it appears at first sight ;-)

It would be useful to have a kind of "suspended" or "limbo" resolution
state on the JBS for issues like this one, so people searching for more
compelling open ones would not encounter them.

Personally, I would close this issue without a "fix"; or "suspend" it.

Greetings
Raffaello

 >>
 >> Greetings
 >> Raffaello
 >>
 >>
 >> On 2021-04-16 20:36, Roger Riggs wrote:
 >>> Hi,
 >>>
 >>> Is there any way to quantify the savings?
     >>> And what technique can be applied to limit the size of the cache.
 >>> The size of the small integer cache is somewhat arbitrary.
 >>>
 >>> Regards, Roger
 >>>
 >>> On 4/16/21 12:48 PM, Raffaello Giulietti wrote:
 >>>> Hello,
 >>>>
 >>>> does the enhancement proposed in [1] make sense, both today
and when
 >>>> wrappers will be migrated to primitive classes?
 >>>> If so, it should be rather simple to add it and I could
prepare a PR.
 >>>> If not, the issue might perhaps be closed.
 >>>>
 >>>>
 >>>> Greetings
 >>>> Raffaello
 >>>>
 >>>> 
 >>>>
 >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8252827
<https://bugs.openjdk.java.net/browse/JDK-8252827>
 >>>

Re: 8252827: Caching Integer.toString just like Integer.valueOf

2021-04-17 Thread Raffaello Giulietti





On 2021-04-17 07:07, David Holmes wrote:

On 17/04/2021 4:54 am, Raffaello Giulietti wrote:
I guess the reporter meant to limit the cache range similarly to the 
one used for valueOf().


I have no clue about the benefit/cost ratio for the proposed String 
cache. It really depends on usage, workload, etc. One can easily 
imagine both extreme scenarios but it's hard to tell how the average 
one would look.


My post is only about either solving the issue by implementing the 
cache, which I can contribute to; or closing it because of lack of 
real-world need or interest.


Caching for the sake of caching is not an objective in itself. Unless 
the caching can be shown to solve a real problem, and the strategy for 
managing the cache is well-defined, then I would just close the 
enhancement request. (Historically whether an issue we don't have any 
firm plans to address is just left open "forever" or closed, depends 
very much on who does the bug triaging in that area. :) )


Cheers,
David




Indeed, the impression is that many of the issues are probably open 
because it's unclear whether they should be addressed with some 
implementation or spec effort or whether they should be closed. Triaging 
is certainly a harder job than it appears at first sight ;-)


It would be useful to have a kind of "suspended" or "limbo" resolution 
state on the JBS for issues like this one, so people searching for more 
compelling open ones would not encounter them.


Personally, I would close this issue without a "fix"; or "suspend" it.


Greetings
Raffaello





Greetings
Raffaello


On 2021-04-16 20:36, Roger Riggs wrote:

Hi,

Is there any way to quantify the savings?
And what technique can be applied to limit the size of the cache.
The size of the small integer cache is somewhat arbitrary.

Regards, Roger

On 4/16/21 12:48 PM, Raffaello Giulietti wrote:

Hello,

does the enhancement proposed in [1] make sense, both today and when 
wrappers will be migrated to primitive classes?

If so, it should be rather simple to add it and I could prepare a PR.
If not, the issue might perhaps be closed.


Greetings
Raffaello



[1] https://bugs.openjdk.java.net/browse/JDK-8252827

Re: 8252827: Caching Integer.toString just like Integer.valueOf

I guess the reporter meant to limit the cache range similarly to the one 
used for valueOf().


I have no clue about the benefit/cost ratio for the proposed String 
cache. It really depends on usage, workload, etc. One can easily imagine 
both extreme scenarios but it's hard to tell how the average one would look.


My post is only about either solving the issue by implementing the 
cache, which I can contribute to; or closing it because of lack of 
real-world need or interest.



Greetings
Raffaello


On 2021-04-16 20:36, Roger Riggs wrote:

Hi,

Is there any way to quantify the savings?
And what technique can be applied to limit the size of the cache.
The size of the small integer cache is somewhat arbitrary.

Regards, Roger

On 4/16/21 12:48 PM, Raffaello Giulietti wrote:

Hello,

does the enhancement proposed in [1] make sense, both today and when 
wrappers will be migrated to primitive classes?

If so, it should be rather simple to add it and I could prepare a PR.
If not, the issue might perhaps be closed.


Greetings
Raffaello



[1] https://bugs.openjdk.java.net/browse/JDK-8252827

8252827: Caching Integer.toString just like Integer.valueOf


Hello,

does the enhancement proposed in [1] make sense, both today and when 
wrappers will be migrated to primitive classes?

If so, it should be rather simple to add it and I could prepare a PR.
If not, the issue might perhaps be closed.


Greetings
Raffaello



[1] https://bugs.openjdk.java.net/browse/JDK-8252827

8246334: Casting ‘double to int’ and ‘long to int’ produce different results


Hello,

the reporter of [1] seems to understand that the current behavior is 
well specified (since ever) and correctly implemented. Also, it's 
unclear how s/he would like to enhance it.


Shouldn't [1] be closed?


Greetings
Raffaello



[1] https://bugs.openjdk.java.net/browse/JDK-8246334

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results [v2]

> Hello,
> 
> here's a PR for a patch submitted on March 2020 
> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was a 
> thing.
> 
> The patch has been edited to adhere to OpenJDK code conventions about 
> multi-line (block) comments. Nothing in the code proper has changed, except 
> for the addition of redundant but clarifying parentheses in some expressions.
> 
> 
> Greetings
> Raffaello

Raffaello Giulietti has updated the pull request incrementally with one 
additional commit since the last revision:

  4511638: Double.toString(double) sometimes produces incorrect results

-

Changes:
  - all: https://git.openjdk.java.net/jdk/pull/3402/files
  - new: https://git.openjdk.java.net/jdk/pull/3402/files/cc10a92d..22092f0c

Webrevs:
 - full: https://webrevs.openjdk.java.net/?repo=jdk=3402=01
 - incr: https://webrevs.openjdk.java.net/?repo=jdk=3402=00-01

  Stats: 26 lines in 1 file changed: 10 ins; 7 del; 9 mod
  Patch: https://git.openjdk.java.net/jdk/pull/3402.diff
  Fetch: git fetch https://git.openjdk.java.net/jdk pull/3402/head:pull/3402

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results

On Thu, 8 Apr 2021 21:12:21 GMT, Raffaello Giulietti 
 wrote:

> Hello,
> 
> here's a PR for a patch submitted on March 2020 
> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was a 
> thing.
> 
> The patch has been edited to adhere to OpenJDK code conventions about 
> multi-line (block) comments. Nothing in the code proper has changed, except 
> for the addition of redundant but clarifying parentheses in some expressions.
> 
> 
> Greetings
> Raffaello

Modified
test/langtools/tools/javac/sym/ElementStructureTest.java
as suggested by @lahodaj

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results

On Fri, 16 Apr 2021 08:22:41 GMT, Jan Lahoda  wrote:

>> Hello,
>> 
>> here's a PR for a patch submitted on March 2020 
>> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was 
>> a thing.
>> 
>> The patch has been edited to adhere to OpenJDK code conventions about 
>> multi-line (block) comments. Nothing in the code proper has changed, except 
>> for the addition of redundant but clarifying parentheses in some expressions.
>> 
>> 
>> Greetings
>> Raffaello
>
> @ rgiulietti, I thought you'd simply add the test change to your patch. But I 
> can start a PR for it, if you prefer.

@lahodaj Oh, my understanding was that yours is a permanent change worth of 
integrating anyway. But yes, I can add your change to my changeset.

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results

On Thu, 8 Apr 2021 21:12:21 GMT, Raffaello Giulietti 
 wrote:

> Hello,
> 
> here's a PR for a patch submitted on March 2020 
> [1](https://cr.openjdk.java.net/~bpb/4511638/webrev.04/) when Mercurial was a 
> thing.
> 
> The patch has been edited to adhere to OpenJDK code conventions about 
> multi-line (block) comments. Nothing in the code proper has changed, except 
> for the addition of redundant but clarifying parentheses in some expressions.
> 
> 
> Greetings
> Raffaello

Fine, thanks!

Will your change be integrated soon on master?
What am I supposed to do then, rebasing my branch with the updated master?

(BTW, you could make use of instanceof pattern matching if you don't need 
backward compat ;-) )

-

PR: https://git.openjdk.java.net/jdk/pull/3402

Re: RFR: 4511638: Double.toString(double) sometimes produces incorrect results