Re: RFR: JDK-8194750, Console.readPassword does not save/restore tty settings

[Martin Buchholz]

 Emacs users thank you for working on this.  To repro, try emacs -q, M-x
shell and run your manual test in there.  You'll see stty -a reports -echo.

This is trickier than I expected, since you have to manage saving/restoring
around each call to readPassword AND have an exit hook to restore in case
the user never gets around to responding to the prompt.

I see in the old code echo returns a boolean that you would think could be
used to restore.  You want to save/restore around each readPassword AND
around the entire java program.  It still seems right to restore after
every readpassword, using the returned value from echo (which the suggested
fix ignores!).  But also, the very first time readPassword is called, you
create an exit hook to restore the value known at that time.  So you need
echo0 as well ... or do you ... maybe you just need to keep track of a
single state variable for you to do first-time setup.

+return tio.c_lflag & ECHO;

Pedantically, functions returning jboolean should return only JNI_TRUE and
JNI_FALSE.  I would do:

return (tio.c_lflag & ECHO) != 0;

although there's the even more pedantic

return ((tio.c_lflag & ECHO) != 0) ? JNI_TRUE : JNI_FALSE;

typo: chosole




On Fri, Apr 13, 2018 at 2:10 PM, Xueming Shen 
wrote:

> Hi,
>
> Please help review the change for JDK-8194750.
>
> issue: https://bugs.openjdk.java.net/browse/JDK-8194750
> webrev: http://cr.openjdk.java.net/~sherman/8194750/webrev
>
> *fix has been manually verified. No new auto-regression test added.
>
> Thanks,
> Sherman
>

RFR: JDK-8194750, Console.readPassword does not save/restore tty settings

[Xueming Shen]

Hi,

Please help review the change for JDK-8194750.

issue: https://bugs.openjdk.java.net/browse/JDK-8194750
webrev: http://cr.openjdk.java.net/~sherman/8194750/webrev

*fix has been manually verified. No new auto-regression test added.

Thanks,
Sherman

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Alan Bateman]

On 13/04/2018 15:14, Adam Farley8 wrote:

Hi Alan, Peter,

I see that native memory is tracked in java.nio.Bits, but that only 
includes what the user thinks they are allocating.


When the VM adds extra memory to the allocation amount this extra bit 
is not represented in the Bits total.
A cursory glance shows, minimum, that we round the requested memory 
quantity up to the heap word size in
the Unsafe.allocateMemory code, and something to do with 
nmt_header_size in os:malloc() (os.cpp) too.

Is the align_up(sz, HeapWordSize) really causing an issue?

We could change Bits to align with HotSpot. The BufferPoolMXBean API 
allows the capacity and memory usage to differ (when we originally added 
this, direct buffers were page aligned) so doing this would mean it more 
accurately reflects the memory allocated to direct buffers.


-Alan

Re: [11] RFR: 8201507: Generate alias entries in j.t.f.ZoneName from tzdb at build time

[Roger Riggs]

Hi Naoto,

Looks fine.

Regards, Roger

On 4/12/18 8:34 PM, Naoto Sato wrote:

Hi,

Please review the fix to the subject issue. While fixing 8189784 [1], 
I noticed that not only CLDR zones but also tzdb link entries are also 
hard coded. So I further modified j.t.f.ZoneName to generate tzdb 
entries at the build time.


The issue: https://bugs.openjdk.java.net/browse/JDK-8201507
Fix: http://cr.openjdk.java.net/~naoto/8201507/webrev.00/

Like the last time, including build-dev for makefile modification review.

Naoto

[1] 
http://mail.openjdk.java.net/pipermail/i18n-dev/2018-April/002492.html

Re: [11] RFR: 8201507: Generate alias entries in j.t.f.ZoneName from tzdb at build time

[Erik Joelsson]

Build changes look good.

/Erik


On 2018-04-12 17:34, Naoto Sato wrote:

Hi,

Please review the fix to the subject issue. While fixing 8189784 [1], 
I noticed that not only CLDR zones but also tzdb link entries are also 
hard coded. So I further modified j.t.f.ZoneName to generate tzdb 
entries at the build time.


The issue: https://bugs.openjdk.java.net/browse/JDK-8201507
Fix: http://cr.openjdk.java.net/~naoto/8201507/webrev.00/

Like the last time, including build-dev for makefile modification review.

Naoto

[1] 
http://mail.openjdk.java.net/pipermail/i18n-dev/2018-April/002492.html

Re: Missing many locales support on AIX platform

[Bhaktavatsal R Maram]

Hi Volker,

Thank you. I will take help of my colleagues and host webrev with all the 
default charsets that are missing for AIX. 
 
- Bhaktavatsal Reddy

-Volker Simonis  wrote: -
To: Bhaktavatsal R Maram 
From: Volker Simonis 
Date: 04/13/2018 08:51PM
Cc: Alan Bateman , Java Core Libs 
, Tim Ellison , 
ppc-aix-port-...@openjdk.java.net
Subject: Re: Missing many locales support on AIX platform

Hi Bhaktavatsal Reddy,

thanks for addressing this long standing issue. I've opened "8201540:
[AIX] Extend the set of supported charsets in java.base" [1] to track
this issue.

As I wrote in the bug report, this problem is the consequence of an
emergency fix (JDK-8081332) I did back in 2015 to fix the build on AIX
after the integration of the modularity support change (see
discussion: [2]). At that time I only added the minimal set of
charsets which were required to fix the build.

It would be great if you can get in touch with your IBM colleagues to
find out which are the default extended charsets supported by IBM J9
on AIX. I think we should try to use a similar set here in our OpenJDK
port which is also used by the OpenJ9 for building OpenJ9 on AIX.

Also, your IBM colleagues can help you to host a webrev which will
make the further review of these changes much easier. I've but Tim
from IBM on CC which should have an overview of all the IBM people
involved in the OpenJDK project. Besides that I know at least the
following people who might help you:

Michihiro Horie: ho...@jp.ibm.com
Hiroshi H Horii: ho...@jp.ibm.com
Gustavo Romero: grom...@linux.vnet.ibm.com
Matthew Brandyberyy: mbra...@linux.vnet.ibm.com

Thank you and best regards,
Volker

[1] 
https://urldefense.proofpoint.com/v2/url?u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8201540=DwIFaQ=jf_iaSHvJObTbx-siA1ZOg=KUVGEwJiRVpNtQ9wUhGP6BKqzSTV1OWX31WWPdQMmqg=cieXGANE8bD3liEC2gJzl5hfZorR2qIfggL6U9t-Et8=hC4gcA6uomYgY26uR74VelobSIK1ReDsjRZGmI46UBY=
[2] 
https://urldefense.proofpoint.com/v2/url?u=http-3A__mail.openjdk.java.net_pipermail_2d-2Ddev_2015-2DMay_005431.html=DwIFaQ=jf_iaSHvJObTbx-siA1ZOg=KUVGEwJiRVpNtQ9wUhGP6BKqzSTV1OWX31WWPdQMmqg=cieXGANE8bD3liEC2gJzl5hfZorR2qIfggL6U9t-Et8=qfYLaVVn7tapNDhv5bY6yE72nhngaWqte4wtRhQ3wB8=

On Fri, Apr 13, 2018 at 2:42 PM, Bhaktavatsal R Maram
 wrote:
> Hi Alan,
>
> Thank you for your response. I'm happy that my patch was attached. But, I 
> don't see attachment. So, I inlined patch which contain diffs from 2 
> changesets in mail text. If a Jira bug is opened for this issue, probably I 
> can attach complete and consolidated patch there.
>
> At high level, I'm adding following charsets to standard charset in 
> java.base. For this, I have to change IBM943C and IBM942C from source to 
> template to handle java package and aliases. It is also required to add 
> codepage 932 as alias for IBM942C because both are one and same.
>
> Big5, Big5_HKSCS, GB18030, IBM942, IBM942C, IBM943, IBM943C, IBM950, IBM970, 
> IBM1124, TIS_620
>
>
> These are default charsets for some of locales supported by Operating System 
> (AIX). Since these are not available in standard charset, JDK can't be used 
> in those locale even if they are available in jdk.charset module (java 
> -version fails).
>
> I've followed some of the discussions around this in community and understand 
> that default charset of a locale should be made available in java.charset 
> module by using stdcs-* mechanism. On Linux, they were added to java.base in 
> similar way. As it is missing for AIX, I've added them to enable JDK support 
> for more locales.
>
>
> Thanks,
> Bhaktavatsal Reddy
>
> -Alan Bateman  wrote: -
> To: Bhaktavatsal R Maram , core-libs-dev@openjdk.java.net
> From: Alan Bateman 
> Date: 04/13/2018 03:52PM
> Subject: Re: Missing many locales support on AIX platform
>
> On 13/04/2018 10:35, Bhaktavatsal R Maram wrote:
>> Given that patch is big, I am sending patch as attachment again after 
>> changing some mail settings. Hopefully, it will make to community this time.
> Your patch was attached.
>
> If I read it correctly, you've switched IBM943C to a template but there
> aren't additional aliases so this part is effectively a no-op, is that
> right? For IBM932C, you've moved it to be template and added several
> aliases.
>
> The rest is AIX specific and I hope the SAP engineers that maintain the
> AIX port can help you. It may be that you are testing with locales that
> aren't supported configurations for the AIX port in OpenJDK. As a
> general point, we try to keep as many of the exotic and multibyte
> charsets out of java.base. They are of course still available to
> applications via the API and the jdk.charsets service provider module.
>
> -Alan
>
>

Re: InetAddress.getByName/getAllByName for empty host string

[Jaikiran Pai]

Hi Chris,

Thank you creating that JIRA.

If the fix involves just updating the javadoc, is this something that 
youwould like me to contribute as a patch? I have a signed and approved 
OCA, but I will need a sponsor if I do come up with the patch.


-Jaikiran


On 13/04/18 8:41 PM, Chris Hegarty wrote:

Hi Jaikiran,

On 13/04/18 15:29, Jaikiran Pai wrote:
The javadoc of InetAddress.getByName(host) and getAllByName(host) 
states that:


If the host is null then an InetAddress representing an address of 
the loopback interface is returned.


For non-null values the javadoc explains what the implementation 
does. However, there seems to be no mention of what it does with an 
empty string (length == 0) value for the host param. Right now, the 
implementation seems to treat an empty value the same way it treats 
the host == null case and returns an InetAddress representing the 
loopback address.


Consider the following example:

public class InetAddressTest {
 public static void main(final String[] args) throws Exception {
 System.out.println("InetAddress.getByName() for empty string 
returns " + java.net.InetAddress.getByName(""));
 System.out.println("InetAddress.getAllByName() for empty 
string returns "
 + 
java.util.Arrays.toString(java.net.InetAddress.getAllByName("")));


 }
}

This outputs:

InetAddress.getByName() for empty string returns localhost/127.0.0.1
InetAddress.getAllByName() for empty string returns 
[localhost/127.0.0.1]



Is it intentional for these APIs to behave this way for empty string?


Yes.


If so, should the javadoc be updated to explicitly state this behaviour?


Yeah, probably.

The following JIRA issue has been filed to track this:
  https://bugs.openjdk.java.net/browse/JDK-8201545

-Chris.

Re: Missing many locales support on AIX platform

[Volker Simonis]

Hi Bhaktavatsal Reddy,

thanks for addressing this long standing issue. I've opened "8201540:
[AIX] Extend the set of supported charsets in java.base" [1] to track
this issue.

As I wrote in the bug report, this problem is the consequence of an
emergency fix (JDK-8081332) I did back in 2015 to fix the build on AIX
after the integration of the modularity support change (see
discussion: [2]). At that time I only added the minimal set of
charsets which were required to fix the build.

It would be great if you can get in touch with your IBM colleagues to
find out which are the default extended charsets supported by IBM J9
on AIX. I think we should try to use a similar set here in our OpenJDK
port which is also used by the OpenJ9 for building OpenJ9 on AIX.

Also, your IBM colleagues can help you to host a webrev which will
make the further review of these changes much easier. I've but Tim
from IBM on CC which should have an overview of all the IBM people
involved in the OpenJDK project. Besides that I know at least the
following people who might help you:

Michihiro Horie: ho...@jp.ibm.com
Hiroshi H Horii: ho...@jp.ibm.com
Gustavo Romero: grom...@linux.vnet.ibm.com
Matthew Brandyberyy: mbra...@linux.vnet.ibm.com

Thank you and best regards,
Volker

[1] https://bugs.openjdk.java.net/browse/JDK-8201540
[2] http://mail.openjdk.java.net/pipermail/2d-dev/2015-May/005431.html

On Fri, Apr 13, 2018 at 2:42 PM, Bhaktavatsal R Maram
 wrote:
> Hi Alan,
>
> Thank you for your response. I'm happy that my patch was attached. But, I 
> don't see attachment. So, I inlined patch which contain diffs from 2 
> changesets in mail text. If a Jira bug is opened for this issue, probably I 
> can attach complete and consolidated patch there.
>
> At high level, I'm adding following charsets to standard charset in 
> java.base. For this, I have to change IBM943C and IBM942C from source to 
> template to handle java package and aliases. It is also required to add 
> codepage 932 as alias for IBM942C because both are one and same.
>
> Big5, Big5_HKSCS, GB18030, IBM942, IBM942C, IBM943, IBM943C, IBM950, IBM970, 
> IBM1124, TIS_620
>
>
> These are default charsets for some of locales supported by Operating System 
> (AIX). Since these are not available in standard charset, JDK can't be used 
> in those locale even if they are available in jdk.charset module (java 
> -version fails).
>
> I've followed some of the discussions around this in community and understand 
> that default charset of a locale should be made available in java.charset 
> module by using stdcs-* mechanism. On Linux, they were added to java.base in 
> similar way. As it is missing for AIX, I've added them to enable JDK support 
> for more locales.
>
>
> Thanks,
> Bhaktavatsal Reddy
>
> -Alan Bateman  wrote: -
> To: Bhaktavatsal R Maram , core-libs-dev@openjdk.java.net
> From: Alan Bateman 
> Date: 04/13/2018 03:52PM
> Subject: Re: Missing many locales support on AIX platform
>
> On 13/04/2018 10:35, Bhaktavatsal R Maram wrote:
>> Given that patch is big, I am sending patch as attachment again after 
>> changing some mail settings. Hopefully, it will make to community this time.
> Your patch was attached.
>
> If I read it correctly, you've switched IBM943C to a template but there
> aren't additional aliases so this part is effectively a no-op, is that
> right? For IBM932C, you've moved it to be template and added several
> aliases.
>
> The rest is AIX specific and I hope the SAP engineers that maintain the
> AIX port can help you. It may be that you are testing with locales that
> aren't supported configurations for the AIX port in OpenJDK. As a
> general point, we try to keep as many of the exotic and multibyte
> charsets out of java.base. They are of course still available to
> applications via the API and the jdk.charsets service provider module.
>
> -Alan
>
>

Re: InetAddress.getByName/getAllByName for empty host string

[Chris Hegarty]

Hi Jaikiran,

On 13/04/18 15:29, Jaikiran Pai wrote:
The javadoc of InetAddress.getByName(host) and getAllByName(host) states 
that:


If the host is null then an InetAddress representing an address of the 
loopback interface is returned.


For non-null values the javadoc explains what the implementation does. 
However, there seems to be no mention of what it does with an empty 
string (length == 0) value for the host param. Right now, the 
implementation seems to treat an empty value the same way it treats the 
host == null case and returns an InetAddress representing the loopback 
address.


Consider the following example:

public class InetAddressTest {
 public static void main(final String[] args) throws Exception {
 System.out.println("InetAddress.getByName() for empty string 
returns " + java.net.InetAddress.getByName(""));
 System.out.println("InetAddress.getAllByName() for empty string 
returns "
 + 
java.util.Arrays.toString(java.net.InetAddress.getAllByName("")));


 }
}

This outputs:

InetAddress.getByName() for empty string returns localhost/127.0.0.1
InetAddress.getAllByName() for empty string returns [localhost/127.0.0.1]


Is it intentional for these APIs to behave this way for empty string?


Yes.

If 
so, should the javadoc be updated to explicitly state this behaviour?


Yeah, probably.

The following JIRA issue has been filed to track this:
  https://bugs.openjdk.java.net/browse/JDK-8201545

-Chris.

InetAddress.getByName/getAllByName for empty host string

[Jaikiran Pai]

The javadoc of InetAddress.getByName(host) and getAllByName(host) states 
that:


If the host is null then an InetAddress representing an address of the 
loopback interface is returned.


For non-null values the javadoc explains what the implementation does. 
However, there seems to be no mention of what it does with an empty 
string (length == 0) value for the host param. Right now, the 
implementation seems to treat an empty value the same way it treats the 
host == null case and returns an InetAddress representing the loopback 
address.


Consider the following example:

public class InetAddressTest {
    public static void main(final String[] args) throws Exception {
        System.out.println("InetAddress.getByName() for empty string 
returns " + java.net.InetAddress.getByName(""));
        System.out.println("InetAddress.getAllByName() for empty string 
returns "
            + 
java.util.Arrays.toString(java.net.InetAddress.getAllByName("")));


    }
}

This outputs:

InetAddress.getByName() for empty string returns localhost/127.0.0.1
InetAddress.getAllByName() for empty string returns [localhost/127.0.0.1]


Is it intentional for these APIs to behave this way for empty string? If 
so, should the javadoc be updated to explicitly state this behaviour?



-Jaikiran

Re: [PATCH] RFR Bug-pending: Enable Hotspot to Track Native Memory Usage for Direct Byte Buffers

[Adam Farley8]

Hi Alan, Peter,

I see that native memory is tracked in java.nio.Bits, but that only 
includes what the user thinks they are allocating.

When the VM adds extra memory to the allocation amount this extra bit is 
not represented in the Bits total. 
A cursory glance shows, minimum, that we round the requested memory 
quantity up to the heap word size in 
the Unsafe.allocateMemory code, and something to do with nmt_header_size 
in os:malloc() (os.cpp) too.

Here's a set of solutions for the problem of "what you see isn't what you 
get" for DBBs (3 is my favourite, but I 
suspect 2 is the most likely to be accepted):

1) Use the code below to call DBB-specific Unsafe methods, allowing us to 
track the quantity of native
memory reserved for DBBs at the point of allocation.

Pros: 
- Doesn't require changing the Bits class.
- Allows us to easily devise a list of DBB native memory allocation 
locations.
- Fits with existing OpenJ9 code, which tracks DBB memory usage *after* 
native code has added to the total 
memory requested, resulting in more accurate totals than Bits.

Cons: 
- Requires some work on the Hotspot side to make it happen if Hotspot 
users want the true benefit.
OR
- Requires a commit that only benefits Java if you're running on the 
OpenJ9 VM.

2) Modify the Bits code to interrogate the VM via an extra method in 
Unsafe, in order to determine the
true quantity of native memory that is being allocated. 

E.g. User requests 10 bits, VM says it needs +2, add 12 to cumulative 
total, return "true". 
User later says to free 10 bits, VM says it needs +2, so we subtract 12 
from the total.

Note: For consistency, the VM and Bits should use the same code when 
figuring out how much space 
will be added to the request by the VM.

Pros: 
- Much smaller change than (1)
- Makes Bits much more accurate.
- Retains the Bits interface, and doesn't require a change to Unsafe.

Cons: 
- Requires us to add a native method to Bits, or somewhere visible to 
Bits.

3) Modify the Bits code to use an internal array for memory reservations, 
returns an index. The user
can use this index to:
- Return the amount of memory they requested.
- Return the amount of memory the VM will actually reserve (Bits will 
retrieve this from the VM via a 
native method).
- Set the address the VM gave them for this reservation. 
- Free the memory reserved.

Note: The existing internal totals, along with Shared Secrets, remain.

Pros: 
- *Much* more accurate memory tracking for DBBs.
- Easier debugging for DBB overruns, as we know where all the DBBs are.
- Prevents the user trying to free too much or too little memory for an 
allocated DBB.

Cons: 
- Many changes to Bits.
- Native code changes.
- Probably best to restructure the memory allocation code to share the 
same logic
(abstracting out the size+x gubbins so Bits' new native code logic remains 
up to date).


Any thoughts?

- Adam




>> Hi Adam,
>>
>> Did you know that native memory is already tracked on the Java side for
>> direct ByteBuffers? See class java.nio.Bits. Could you make use of it?
>>
> Right, these are the fields that are exposed at runtime via 
> BufferPoolMXBean. A SA based tool could read from a core file. I can't 
> tell if this is enough for Adam, it may be that the his tool reveals 
> more details on the buffers in the pools.
> 
> -Alan


P.S. Code:

diff --git 
a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template 
b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
--- a/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
+++ b/src/java.base/share/classes/java/nio/Direct-X-Buffer.java.template
@@ -85,7 +85,7 @@
 // Paranoia
 return;
 }
-UNSAFE.freeMemory(address);
+UNSAFE.freeDBBMemory(address);
 address = 0;
 Bits.unreserveMemory(size, capacity);
 }
@@ -118,7 +118,7 @@
 
 long base = 0;
 try {
-base = UNSAFE.allocateMemory(size);
+base = UNSAFE.allocateDBBMemory(size);
 } catch (OutOfMemoryError x) {
 Bits.unreserveMemory(size, cap);
 throw x;
diff --git a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java 
b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
--- a/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
+++ b/src/java.base/share/classes/jdk/internal/misc/Unsafe.java
@@ -632,6 +632,26 @@
 }
 
 /**
+ * Allocates a new block of native memory for DirectByteBuffers, of 
the
+ * given size in bytes.  The contents of the memory are 
uninitialized;
+ * they will generally be garbage.  The resulting native pointer will
+ * never be zero, and will be aligned for all value types.  Dispose 
of
+ * this memory by calling {@link #freeDBBMemory} or resize it with
+ * {@link #reallocateDBBMemory}.
+ *
+ * @throws RuntimeException if the size is negative or too large 
+ *

Re: Missing many locales support on AIX platform

[Roger Riggs]

Hi,

The question has come up before:   [1]

Most OpenJDK mailing lists,  ..., allow these attachment types:

  text/plain
  text/x-diff
  text/x-patch
  message/rfc822

Attachments with the following filename extensions are dropped:

  exe
  bat
  cmd
  com
  pif
  scr
  vbs
  cpl

Regards, Roger

[1] 
http://mail.openjdk.java.net/pipermail/code-tools-dev/2018-March/000378.html





On 4/13/18 5:37 AM, Bhaktavatsal R Maram wrote:

No luck! :(

How can I attach patch? Any suggestions?

Thanks,
Bhaktavatsal Reddy

   


-Bhaktavatsal R Maram/India/IBM wrote: -
To: core-libs-dev@openjdk.java.net
From: Bhaktavatsal R Maram/India/IBM
Date: 04/13/2018 03:05PM
Subject: Re: Missing many locales support on AIX platform

Given that patch is big, I am sending patch as attachment again after changing 
some mail settings. Hopefully, it will make to community this time.

Thanks,
Bhaktavatsal Reddy
  

   


-Bhaktavatsal R Maram/India/IBM wrote: -
To: core-libs-dev@openjdk.java.net
From: Bhaktavatsal R Maram/India/IBM
Date: 04/12/2018 11:47PM
Subject: Re: Missing many locales support on AIX platform

Please review this patch for this issue. I have added IBM-943C as well to 
standard charset along with IBM-943.

# HG changeset patch
# User bhamaram
# Date 1523510846 -19800
#  Thu Apr 12 10:57:26 2018 +0530
# Node ID a76dd37b1c6b47e7ac99f279c4ba76832dd9736f
# Parent  129d60b5dac71872f3317d94c724304100bc2048
java.lang.IllegalArgumentException from java -version on AIX with different 
locales

diff -r 129d60b5dac7 -r a76dd37b1c6b make/data/charsetmapping/charsets
--- a/make/data/charsetmapping/charsets Thu Apr 12 09:03:46 2018 -0400
+++ b/make/data/charsetmapping/charsets Thu Apr 12 10:57:26 2018 +0530
@@ -933,11 +933,16 @@
  
  charset x-IBM942C IBM942C

  package sun.nio.cs.ext
-typesource
+typetemplate
  alias   cp942C   # JDK historical
  alias   ibm942C
  alias   ibm-942C
  alias   942C
+alias   cp932
+alias   ibm932
+alias   ibm-932
+alias   932
+alias   x-ibm932
  
  charset x-IBM943 IBM943

  package sun.nio.cs.ext
diff -r 129d60b5dac7 -r a76dd37b1c6b make/data/charsetmapping/stdcs-aix
--- a/make/data/charsetmapping/stdcs-aixThu Apr 12 09:03:46 2018 -0400
+++ b/make/data/charsetmapping/stdcs-aixThu Apr 12 10:57:26 2018 +0530
@@ -1,6 +1,16 @@
  #
  #   generate these charsets into sun.nio.cs
  #
+Big5
+Big5_HKSCS
  EUC_CN
  EUC_KR
  GBK
+GB18030
+IBM942
+IBM942C
+IBM943
+IBM950
+IBM970
+IBM1124
+TIS_620
diff -r 129d60b5dac7 -r a76dd37b1c6b 
src/jdk.charsets/share/classes/sun/nio/cs/ext/IBM942C.java
--- a/src/jdk.charsets/share/classes/sun/nio/cs/ext/IBM942C.javaThu Apr 
12 09:03:46 2018 -0400
+++ /dev/null   Thu Jan 01 00:00:00 1970 +
@@ -1,93 +0,0 @@
-/*
- * Copyright (c) 2003, 2004, Oracle and/or its affiliates. All rights reserved.
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This code is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License version 2 only, as
- * published by the Free Software Foundation.  Oracle designates this
- * particular file as subject to the "Classpath" exception as provided
- * by Oracle in the LICENSE file that accompanied this code.
- *
- * This code is distributed in the hope that it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
- * version 2 for more details (a copy is included in the LICENSE file that
- * accompanied this code).
- *
- * You should have received a copy of the GNU General Public License version
- * 2 along with this work; if not, write to the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
- *
- * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
- * or visit www.oracle.com if you need additional information or have any
- * questions.
- */
-
-package sun.nio.cs.ext;
-
-import java.nio.charset.Charset;
-import java.nio.charset.CharsetDecoder;
-import java.nio.charset.CharsetEncoder;
-import java.util.Arrays;
-import sun.nio.cs.DoubleByte;
-import sun.nio.cs.HistoricallyNamedCharset;
-import static sun.nio.cs.CharsetMapping.*;
-
-public class IBM942C extends Charset implements HistoricallyNamedCharset
-{
-public IBM942C() {
-super("x-IBM942C", ExtendedCharsets.aliasesFor("x-IBM942C"));
-}
-
-public String historicalName() {
-return "Cp942C";
-}
-
-public boolean contains(Charset cs) {
-return ((cs.name().equals("US-ASCII"))
-|| (cs instanceof IBM942C));
-}
-
-public CharsetDecoder newDecoder() {
-return new DoubleByte.Decoder(this,
-  IBM942.b2c,
-  b2cSB,
-  0x40,
-

Re: Missing many locales support on AIX platform

[Bhaktavatsal R Maram]

Hi Alan,

Thank you for your response. I'm happy that my patch was attached. But, I don't 
see attachment. So, I inlined patch which contain diffs from 2 changesets in 
mail text. If a Jira bug is opened for this issue, probably I can attach 
complete and consolidated patch there.

At high level, I'm adding following charsets to standard charset in java.base. 
For this, I have to change IBM943C and IBM942C from source to template to 
handle java package and aliases. It is also required to add codepage 932 as 
alias for IBM942C because both are one and same.

Big5, Big5_HKSCS, GB18030, IBM942, IBM942C, IBM943, IBM943C, IBM950, IBM970, 
IBM1124, TIS_620


These are default charsets for some of locales supported by Operating System 
(AIX). Since these are not available in standard charset, JDK can't be used in 
those locale even if they are available in jdk.charset module (java -version 
fails).

I've followed some of the discussions around this in community and understand 
that default charset of a locale should be made available in java.charset 
module by using stdcs-* mechanism. On Linux, they were added to java.base in 
similar way. As it is missing for AIX, I've added them to enable JDK support 
for more locales.
 

Thanks,
Bhaktavatsal Reddy

-Alan Bateman  wrote: -
To: Bhaktavatsal R Maram , core-libs-dev@openjdk.java.net
From: Alan Bateman 
Date: 04/13/2018 03:52PM
Subject: Re: Missing many locales support on AIX platform

On 13/04/2018 10:35, Bhaktavatsal R Maram wrote:
> Given that patch is big, I am sending patch as attachment again after 
> changing some mail settings. Hopefully, it will make to community this time.
Your patch was attached.

If I read it correctly, you've switched IBM943C to a template but there 
aren't additional aliases so this part is effectively a no-op, is that 
right? For IBM932C, you've moved it to be template and added several 
aliases.

The rest is AIX specific and I hope the SAP engineers that maintain the 
AIX port can help you. It may be that you are testing with locales that 
aren't supported configurations for the AIX port in OpenJDK. As a 
general point, we try to keep as many of the exotic and multibyte 
charsets out of java.base. They are of course still available to 
applications via the API and the jdk.charsets service provider module.

-Alan

Re: RFR: 8201274: Launch Single-File Source-Code Programs

[Maurizio Cimadamore]

One small followup question - the test SourceLauncherTest is not 
covering any shebang cases - is that deliberate? I see that those seem 
to be covered in the launcher test too, but I wonder if we should have 
tests for clearly broken stuff, such as


'#'

'#!'

'#!\n'

Another small question - I see that the shebang process essentially 
replaces the first line separator with \n - that is probably ok given 
that the file is processed internally after that - but I wonder if you 
have thought about pros and cons of preserving the original line separator.


Cheers
Maurizio


On 12/04/18 21:46, Maurizio Cimadamore wrote:
Looks great - some initial comments (I can't really comment on the 
launcher changes):


* This logic is efficient:

int magic = (in.read() << 8) + in.read();
boolean shebang = magic == (('#' << 8) + '!');

but convoluted to read; perhaps could be improved slightly by making 
'#' << 8) + '!' a static constant, and comparing against that.


* I note that the reading logic in general could be simplified, e.g. 
using Files.lines(Path) - but I assume that you wrote the code as is 
for performance reasons (e.g. to avoid creating too many string 
objects) ?


* I see that both the file manager and the class loader reasonably 
share the same context: a Map. I would make this more 
explicit, by having a Context class, whose state is the map, and then 
have the context provide two methods:


getClassLoader()
getFileManager()

This way the sharing will be automatic, no need to extract one field 
from one place and pass it over to the other place.


* Big whohoo for being able to use the enhanced diagnostic framework 
with a couple of tweaks on the makefile - I hope that would have been 
the case when I put in the support, but since we have never done it - 
wasn't 100% sure it would work ;-)



Overall I like it quite a lot and I think you went for a really clean 
design - kudos!


Maurizio



On 12/04/18 21:15, Jonathan Gibbons wrote:

Please review an initial implementation for the feature described in
JEP 330: Launch Single-File Source-Code Programs.

The work is described in the JEP and CSR, and falls into various parts:

  * The part to handle the new command-line options is in the native
    Java launcher code.
  * The part to invoke the compiler and subsequently execute the code
    found in the source file is in a new class in the jdk.compiler 
module.

  * There are some minor Makefile changes, to add support for a new
    resource file.

There are no changes to javac itself.

JEP: http://openjdk.java.net/jeps/330
JBS: https://bugs.openjdk.java.net/browse/JDK-8201274
CSR: https://bugs.openjdk.java.net/browse/JDK-8201275
Webrev: http://cr.openjdk.java.net/~jjg/8201274/webrev.00/

-- Jon

Re: Missing many locales support on AIX platform

[Alan Bateman]

On 13/04/2018 10:35, Bhaktavatsal R Maram wrote:

Given that patch is big, I am sending patch as attachment again after changing 
some mail settings. Hopefully, it will make to community this time.

Your patch was attached.

If I read it correctly, you've switched IBM943C to a template but there 
aren't additional aliases so this part is effectively a no-op, is that 
right? For IBM932C, you've moved it to be template and added several 
aliases.


The rest is AIX specific and I hope the SAP engineers that maintain the 
AIX port can help you. It may be that you are testing with locales that 
aren't supported configurations for the AIX port in OpenJDK. As a 
general point, we try to keep as many of the exotic and multibyte 
charsets out of java.base. They are of course still available to 
applications via the API and the jdk.charsets service provider module.


-Alan

Re: Missing many locales support on AIX platform

[Bhaktavatsal R Maram]

No luck! :(

How can I attach patch? Any suggestions?

Thanks,
Bhaktavatsal Reddy

  

-Bhaktavatsal R Maram/India/IBM wrote: -
To: core-libs-dev@openjdk.java.net
From: Bhaktavatsal R Maram/India/IBM
Date: 04/13/2018 03:05PM
Subject: Re: Missing many locales support on AIX platform

Given that patch is big, I am sending patch as attachment again after changing 
some mail settings. Hopefully, it will make to community this time.

Thanks,
Bhaktavatsal Reddy
 

  

-Bhaktavatsal R Maram/India/IBM wrote: -
To: core-libs-dev@openjdk.java.net
From: Bhaktavatsal R Maram/India/IBM
Date: 04/12/2018 11:47PM
Subject: Re: Missing many locales support on AIX platform

Please review this patch for this issue. I have added IBM-943C as well to 
standard charset along with IBM-943.

# HG changeset patch
# User bhamaram
# Date 1523510846 -19800
#  Thu Apr 12 10:57:26 2018 +0530
# Node ID a76dd37b1c6b47e7ac99f279c4ba76832dd9736f
# Parent  129d60b5dac71872f3317d94c724304100bc2048
java.lang.IllegalArgumentException from java -version on AIX with different 
locales

diff -r 129d60b5dac7 -r a76dd37b1c6b make/data/charsetmapping/charsets
--- a/make/data/charsetmapping/charsets Thu Apr 12 09:03:46 2018 -0400
+++ b/make/data/charsetmapping/charsets Thu Apr 12 10:57:26 2018 +0530
@@ -933,11 +933,16 @@
 
 charset x-IBM942C IBM942C
 package sun.nio.cs.ext
-typesource
+typetemplate
 alias   cp942C   # JDK historical
 alias   ibm942C
 alias   ibm-942C
 alias   942C
+alias   cp932
+alias   ibm932
+alias   ibm-932
+alias   932
+alias   x-ibm932
 
 charset x-IBM943 IBM943
 package sun.nio.cs.ext
diff -r 129d60b5dac7 -r a76dd37b1c6b make/data/charsetmapping/stdcs-aix
--- a/make/data/charsetmapping/stdcs-aixThu Apr 12 09:03:46 2018 -0400
+++ b/make/data/charsetmapping/stdcs-aixThu Apr 12 10:57:26 2018 +0530
@@ -1,6 +1,16 @@
 #
 #   generate these charsets into sun.nio.cs
 #
+Big5
+Big5_HKSCS
 EUC_CN
 EUC_KR
 GBK
+GB18030
+IBM942
+IBM942C
+IBM943
+IBM950
+IBM970
+IBM1124
+TIS_620
diff -r 129d60b5dac7 -r a76dd37b1c6b 
src/jdk.charsets/share/classes/sun/nio/cs/ext/IBM942C.java
--- a/src/jdk.charsets/share/classes/sun/nio/cs/ext/IBM942C.javaThu Apr 
12 09:03:46 2018 -0400
+++ /dev/null   Thu Jan 01 00:00:00 1970 +
@@ -1,93 +0,0 @@
-/*
- * Copyright (c) 2003, 2004, Oracle and/or its affiliates. All rights reserved.
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This code is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License version 2 only, as
- * published by the Free Software Foundation.  Oracle designates this
- * particular file as subject to the "Classpath" exception as provided
- * by Oracle in the LICENSE file that accompanied this code.
- *
- * This code is distributed in the hope that it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
- * version 2 for more details (a copy is included in the LICENSE file that
- * accompanied this code).
- *
- * You should have received a copy of the GNU General Public License version
- * 2 along with this work; if not, write to the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
- *
- * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
- * or visit www.oracle.com if you need additional information or have any
- * questions.
- */
-
-package sun.nio.cs.ext;
-
-import java.nio.charset.Charset;
-import java.nio.charset.CharsetDecoder;
-import java.nio.charset.CharsetEncoder;
-import java.util.Arrays;
-import sun.nio.cs.DoubleByte;
-import sun.nio.cs.HistoricallyNamedCharset;
-import static sun.nio.cs.CharsetMapping.*;
-
-public class IBM942C extends Charset implements HistoricallyNamedCharset
-{
-public IBM942C() {
-super("x-IBM942C", ExtendedCharsets.aliasesFor("x-IBM942C"));
-}
-
-public String historicalName() {
-return "Cp942C";
-}
-
-public boolean contains(Charset cs) {
-return ((cs.name().equals("US-ASCII"))
-|| (cs instanceof IBM942C));
-}
-
-public CharsetDecoder newDecoder() {
-return new DoubleByte.Decoder(this,
-  IBM942.b2c,
-  b2cSB,
-  0x40,
-  0xfc);
-}
-
-public CharsetEncoder newEncoder() {
-return new DoubleByte.Encoder(this, c2b, c2bIndex);
-}
-
-final static char[] b2cSB;
-final static char[] c2b;
-final static char[] c2bIndex;
-
-static {
-IBM942.initb2c();
-
-// the mappings need udpate are
-//u+001a  <-> 0x1a
-//u+001c  <-> 0x1c
-//u+005c  <-> 0x5c
-//u+007e  <-> 0x7e
-

Re: Missing many locales support on AIX platform

[Bhaktavatsal R Maram]

Given that patch is big, I am sending patch as attachment again after changing 
some mail settings. Hopefully, it will make to community this time.

Thanks,
Bhaktavatsal Reddy
 

  

-Bhaktavatsal R Maram/India/IBM wrote: -
To: core-libs-dev@openjdk.java.net
From: Bhaktavatsal R Maram/India/IBM
Date: 04/12/2018 11:47PM
Subject: Re: Missing many locales support on AIX platform

Please review this patch for this issue. I have added IBM-943C as well to 
standard charset along with IBM-943.

# HG changeset patch
# User bhamaram
# Date 1523510846 -19800
#  Thu Apr 12 10:57:26 2018 +0530
# Node ID a76dd37b1c6b47e7ac99f279c4ba76832dd9736f
# Parent  129d60b5dac71872f3317d94c724304100bc2048
java.lang.IllegalArgumentException from java -version on AIX with different 
locales

diff -r 129d60b5dac7 -r a76dd37b1c6b make/data/charsetmapping/charsets
--- a/make/data/charsetmapping/charsets Thu Apr 12 09:03:46 2018 -0400
+++ b/make/data/charsetmapping/charsets Thu Apr 12 10:57:26 2018 +0530
@@ -933,11 +933,16 @@
 
 charset x-IBM942C IBM942C
 package sun.nio.cs.ext
-typesource
+typetemplate
 alias   cp942C   # JDK historical
 alias   ibm942C
 alias   ibm-942C
 alias   942C
+alias   cp932
+alias   ibm932
+alias   ibm-932
+alias   932
+alias   x-ibm932
 
 charset x-IBM943 IBM943
 package sun.nio.cs.ext
diff -r 129d60b5dac7 -r a76dd37b1c6b make/data/charsetmapping/stdcs-aix
--- a/make/data/charsetmapping/stdcs-aixThu Apr 12 09:03:46 2018 -0400
+++ b/make/data/charsetmapping/stdcs-aixThu Apr 12 10:57:26 2018 +0530
@@ -1,6 +1,16 @@
 #
 #   generate these charsets into sun.nio.cs
 #
+Big5
+Big5_HKSCS
 EUC_CN
 EUC_KR
 GBK
+GB18030
+IBM942
+IBM942C
+IBM943
+IBM950
+IBM970
+IBM1124
+TIS_620
diff -r 129d60b5dac7 -r a76dd37b1c6b 
src/jdk.charsets/share/classes/sun/nio/cs/ext/IBM942C.java
--- a/src/jdk.charsets/share/classes/sun/nio/cs/ext/IBM942C.javaThu Apr 
12 09:03:46 2018 -0400
+++ /dev/null   Thu Jan 01 00:00:00 1970 +
@@ -1,93 +0,0 @@
-/*
- * Copyright (c) 2003, 2004, Oracle and/or its affiliates. All rights reserved.
- * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
- *
- * This code is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License version 2 only, as
- * published by the Free Software Foundation.  Oracle designates this
- * particular file as subject to the "Classpath" exception as provided
- * by Oracle in the LICENSE file that accompanied this code.
- *
- * This code is distributed in the hope that it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
- * version 2 for more details (a copy is included in the LICENSE file that
- * accompanied this code).
- *
- * You should have received a copy of the GNU General Public License version
- * 2 along with this work; if not, write to the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
- *
- * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
- * or visit www.oracle.com if you need additional information or have any
- * questions.
- */
-
-package sun.nio.cs.ext;
-
-import java.nio.charset.Charset;
-import java.nio.charset.CharsetDecoder;
-import java.nio.charset.CharsetEncoder;
-import java.util.Arrays;
-import sun.nio.cs.DoubleByte;
-import sun.nio.cs.HistoricallyNamedCharset;
-import static sun.nio.cs.CharsetMapping.*;
-
-public class IBM942C extends Charset implements HistoricallyNamedCharset
-{
-public IBM942C() {
-super("x-IBM942C", ExtendedCharsets.aliasesFor("x-IBM942C"));
-}
-
-public String historicalName() {
-return "Cp942C";
-}
-
-public boolean contains(Charset cs) {
-return ((cs.name().equals("US-ASCII"))
-|| (cs instanceof IBM942C));
-}
-
-public CharsetDecoder newDecoder() {
-return new DoubleByte.Decoder(this,
-  IBM942.b2c,
-  b2cSB,
-  0x40,
-  0xfc);
-}
-
-public CharsetEncoder newEncoder() {
-return new DoubleByte.Encoder(this, c2b, c2bIndex);
-}
-
-final static char[] b2cSB;
-final static char[] c2b;
-final static char[] c2bIndex;
-
-static {
-IBM942.initb2c();
-
-// the mappings need udpate are
-//u+001a  <-> 0x1a
-//u+001c  <-> 0x1c
-//u+005c  <-> 0x5c
-//u+007e  <-> 0x7e
-//u+007f  <-> 0x7f
-
-b2cSB = Arrays.copyOf(IBM942.b2cSB, IBM942.b2cSB.length);
-b2cSB[0x1a] = 0x1a;
-b2cSB[0x1c] = 0x1c;
-b2cSB[0x5c] = 0x5c;
-b2cSB[0x7e] = 0x7e;
-b2cSB[0x7f] = 0x7f;
-
-IBM942.initc2b();
-c2b = 

Re: Clean-room implementation of Double::toString(double) and Float::toString(float)

[Thomas Stüfe]

On Sat, Mar 31, 2018 at 12:17 AM, Brian Burkhalter
 wrote:
> Hi Raffaello,
>
> On Mar 30, 2018, at 2:57 PM, raffaello.giulie...@gmail.com wrote:
>
 […]

 The new code also has a better specification than the current one, while
 being mostly compatible. Indeed, the current specification leaves room
 for interpretation and thus cannot ensure that an implementation
 produces consistent and unique results from one release to the next. The
 newer spec ensures a unique result.
>>>
>>> Any specification change would need to go through the Compatibility and
>>> Specification Review process. [3]
>>>
>>
>> OK, as you will see, as soon as the code will be uploaded, the only
>> thing that formally affects output is the "1.0E23" versus "9.99E22"
>> issue. Everything else is worded in such a way to remain compatible but
>> is simply a little bit more rigorous.
>
> Sounds good.
>
>> My wording was misleading: I already got the confirmation that my OCA
>> application has been accepted, so I'm formally ready to contribute.
>
> That’s good as it gives more time.
>
>>> Per the JDK 11 schedule [5] there could well be sufficient time to run
>>> this submission through the review processes. I suggest, once your OCA
>>> has been processed, to proceed by posting your proposed changes for
>>> review on this mailing list. Note that in general attachments are
>>> scrubbed, so the patch would need either to be included inline or
>>> published as a webrev [6].
>>>
>>
>> OK, I'll take a look on how the mechanics works.
>>
>> I'm usually on Windows. Are there technical issues with it as far as
>> Webrev is concerned? I mean, I could setup a Linux VM in VirtualBox if
>> this simplifies my life, but I'd prefer continuing my main work in Win.
>
> As seen in Jon’s posting there are some attachment types which will work. As 
> to webrev, I think it should work on Windows at least in cygwin but I’ve not 
> used it there myself. If it’s just a matter of creating a webrev I could do 
> that on your behalf based on your supplied patch.
>

Just to chime in here, producing webrevs on windows is easy:

- in cygwin, install Korn shell (ksh)
- and, of course, mercurial

the rest is standard:
- clone the code-tools/webrev repo:
http://hg.openjdk.java.net/code-tools/webrev/
- run webrev: ksh /webrev.ksh -o 

Best Regards, Thomas

> Thanks,
>
> Brian

Re: RFR: 8201274: Launch Single-File Source-Code Programs

[Jan Lahoda]

The javac part looks OK to me.

A nit comment, in:
launcher/Main.java/MemoryFileManager#createInMemoryClassFile, there is:
return new FilterOutputStream(new ByteArrayOutputStream()) { ...

It could I think be written as:
return new ByteArrayOutputStream() {
@Override
public void close() throws IOException {
super.close();
byte[] bytes = toByteArray();
map.put(className, bytes);
}
};

(I.e. without the cast to ByteArrayOutputStream.)

Jan

On 12.4.2018 22:15, Jonathan Gibbons wrote:

Please review an initial implementation for the feature described in
JEP 330: Launch Single-File Source-Code Programs.

The work is described in the JEP and CSR, and falls into various parts:

  * The part to handle the new command-line options is in the native
Java launcher code.
  * The part to invoke the compiler and subsequently execute the code
found in the source file is in a new class in the jdk.compiler module.
  * There are some minor Makefile changes, to add support for a new
resource file.

There are no changes to javac itself.

JEP: http://openjdk.java.net/jeps/330
JBS: https://bugs.openjdk.java.net/browse/JDK-8201274
CSR: https://bugs.openjdk.java.net/browse/JDK-8201275
Webrev: http://cr.openjdk.java.net/~jjg/8201274/webrev.00/

-- Jon