Re: [concurrency-interest] RFR: 8065804: JEP 171: Clarifications/corrections for fence intrinsics

[Hans Boehm]

If we look at using purely store fences and purely load fences in the
"initialized flag" example as in this discussion, I think it's worth
distinguishing too possible scenarios:

1) We guarantee some form of dependency-based ordering, as most real
computer architectures do.  This probably invalidates the example from my
committee paper that's under discussion here.  The problem is, as always,
that we don't know how to make this precise at the programming language
level.  It's the compiler's job to break certain dependencies, like the
dependency of the store to x on the load of y in x = 0 * y.  Many people
are thinking about this problem, both to deal with "out-of-thin-air" issues
correctly in various memory models, and to design a version of C++'s
memory_order_consume that's more usable.  If we had a way to guarantee some
well-defined notion of dependency-based ordering, then at least some of the
examples here would need to be revisited.

2) We don't guarantee that dependencies imply any sort of ordering.  Then I
think the weird example under discussion here stands.  There is officially
nothing to prevent the load of x.a in thread 1 from being reordered with
the store to x_init.

But there may actually be better examples as to why the store-store
ordering in the initializing thread is not always enough.  Consider:

Thread 1:
x.a = 1;
if (x.a != 1) world_is_broken = true;
StoreStore fence;
x_init = true;
...
if (world_is_broken) die();

Thread 2:
if (x_init) {
full fence;
x.a++;
}

I think there is nothing to prevent the read of x.a in Thread 1 from seeing
the incremented value, at least if (1) the compiler promotes
world_is_broken to a register, and  (2) at the assembly level the store to
x_init is not dependent on the load of x.a.  (1) seems quite plausible, and
(2) seems very reasonable if the architecture has a conditional move
instruction or the like.  (For Itanium, (2) holds even for the naive
compilation.)

This is not a particularly likely scenario, but I have no idea how would
concoct programming rules that would guarantee to prevent this kind of
weirdness.  The first two statements of Thread 1 might appear inside an
"initialize a" library routine that knows nothing about concurrency.

Hans


On Wed, Dec 17, 2014 at 10:54 AM, Martin Buchholz 
wrote:

> On Wed, Dec 17, 2014 at 1:28 AM, Peter Levart 
> wrote:
> > On 12/17/2014 03:28 AM, David Holmes wrote:
> >>
> >> On 17/12/2014 10:06 AM, Martin Buchholz wrote:
> >> Hans allows for the nonsensical, in my view, possibility that the load
> of
> >> x.a can happen after the x_init=true store and yet somehow be subject
> to the
> >> ++ and the ensuing store that has to come before the x_init = true.
> >
> > Perhaps, he is speaking about why it is dangerous to replace BOTH release
> > with just store-store AND acquire with just load-load?
>
> I'm pretty sure he's talking about weakening EITHER.
>
> """Clearly, and unsurprisingly, it is unsafe to replace the
> load_acquire with a version that restricts only load ordering in this
> case. That would allow the store to x in thread 2 to become visible
> before the initialization of x by thread 1 is complete, possibly
> losing the update, or corrupting the state of x during initialization.
>
> More interestingly, it is also generally unsafe to restrict the
> release ordering constraint in thread 1 to only stores."""
>
> (What's "clear and unsurprising" to Hans may not be to the rest of us)
>

Re: Explicit Serialization API and Security

[Peter Firmstone]

Not quite, the constructor signature is the same for super and child classes.

ReadSerial is a container for each serialized Object in an ObjectInputStream 
that provides and controlls access to serial fields, identified by name, type 
and calling class, such that each class has it's own parameters contained 
within ReadSerial.

Serial parameters, might be better terminology than serial fields.  Serial 
parameters, retrieved from ReadSerial are compatible with serial fields in the 
serialization specification, providing a compatible upgrade path, and enabling 
all classes belonging to an Object to obtain parameters during construction, 
whilst allowing implementations to be independant of an object's serial form.

When ReadSerial doesn't contain a parameter, it will return null.

// pseudocode
public abstract class ReadSerial {

protected ReadSerial(){
// check permission
}

// perform caller class dependant permission and type check
public  T getParameter( Class type, String name );

}


Peter.

Sent from my Nokia N900.

- Original message -
> So, if I understand this correctly, the way this would get used is:
> 
> class BaseFoo implements Serializable {
>           private final int x;
> 
>           public BaseFoo(ReadSerial rs) {
>                   this(rs.getInt("x"));
>           }
> 
>           public BaseFoo(int x) {
>                   this.x = x;
>           }
> }
> 
> Right?
> 
> What happens with subclasses?   I think then I need an extra RS arg in my 
> constructors:
> 
> class SubFoo extends BaseFoo {
>           private final int y;
> 
>           public SubFoo(ReadSerial rs) {
>                   this(rs.getInt("y"));
>           }
> 
>           public BaseFoo(ReadSerial rs, int y) {
>                   super(rs);
>                   this.y = y;
>           }
> }
> 
> Is this what you envision?
> 
> 
> 
> 
> 
> On 12/27/2014 8:03 PM, Peter Firmstone wrote:
> > Is there any interest in developing an explicit API for Serialization?:
> > 
> > 1. Use a public constructor signature with a single argument,
> > ReadSerialParameters (read only, writable only by the
> > serialization framework) to recreate objects, subclasses (when
> > permitted) call this first from their own constructor, they have
> > an identical constructor signature.   ReadSerialParameters that are
> > null may contain a circular reference and will be available after
> > construction, see #3 below.
> > 2. Use a factory method (defined by an interface) with one parameter,
> > WriteSerialParameters (write only, readable only by the
> > serialization framework), this method can be overridden by
> > subclasses (when permitted)
> > 3. For circular links, a public method (defined by an interface) that
> > accepts one argument, ReadSerialParameters, this method is called
> > after the constructor completes, subclasses overriding this should
> > call the superclass method.   If this method is not called, an
> > implementation, if known to possibly contain circular links,
> > should check it has been fully initialized in each object method
> > called.
> > 4. Retains compatibility with current serialization stream format.
> > 5. Each serial field has a name, calling class and object reference,
> > similar to explicitly declaring "private static final
> > ObjectStreamField[] serialPersistentFields ".
> > 
> > Benefits:
> > 
> > 1. An object's internal form is not publicised.
> > 2. Each class in an object's heirarchy can use a static method to
> > check invarients and throw an exception, prior to
> > java.lang.Object's constructor being called, preventing
> > construction and avoiding finalizer attacks.
> > 3. Final field friendly.
> > 4. Compatible with existing serial form.
> > 5. Flexible serial form evolution.
> > 6. All methods are public and explicitly defined.
> > 7. All class ProtectionDomain's exist in the current execution
> > context, allowing an object to throw a SecurityException before
> > construction.
> > 8. Less susceptible to deserialization attacks.
> > 
> > Problems:
> > 
> > 1. Implementations cannot be package private or private.   Implicit
> > serialization publicises internal form, any thoughts?
> > 
> > Recommendations:
> > 
> > 1. Create a security check in the serialization framework for
> > implicit serialization, allowing administrators to reduce their
> > deserialization attack surface.
> > 2. For improved security, disallow classes implementing explicit
> > serialization from having static state and static initializer
> > blocks, only allow static methods, this would require complier and
> > verifier changes.
> > 3. Alternative to #2, allow final static fields, but don't allow
> > static initializer blocks or mutable static fields, similar to
> > interfaces.
> > 
> > Penny for your thoughts?
> > 
> > Regards,
> > 
> > Peter Firmstone.

Re: RFR 8066397 Remove network-related seed initialization code in ThreadLocal/SplittableRandom

[Bradford Wetmore]

To the actual proposal:

http://cr.openjdk.java.net/~plevart/jdk9-dev/SystemRandom/webrev.03/

Overall, I'm ok with what's proposed.  This is more straightforward to
parse/understand than trying to adjust NativeSeedGenerator to
create/call directly (e.g. UNIX:  new
NativeSeedGenerator("/dev/urandom") or Windows:  new
NativeSeedGenerator()).  But I'd still like to understand why you
moved away from this.

One concern is that you're duplicating native libraries in java.base,
and it would be the third JDK library overall with this type of call.
There's one in libjava (for java.base/WinCAPISeedGenerator for
sun.security.provider.NativeSeedGenerator) and sunmscapi (for
jdk.crypto.mscapi/SunMSCAPI/sun.security.mscapi).  Would it work to
tweak the WinCAPISeedGenerator so you don't have to create a new dll
for java.base?


The SystemRandom JNI bindings for Windows are located in:

 java.base/windows/native/libjava/SystemRandomImpl_md.c

...so as I understand they are also part of libjava. No new DLL here.


True.  My thought should have been about having very similar code 
duplicated in libjava.


I'm ok with this, though it's not really clear if/when MS will drop 
support for ADVAPI32!RtlGenRandom.  This always makes me nervous because 
whatever is put in will likely never change until some MS change breaks it.


I'm not familiar with what Alan/Mandy/company have in mind down the 
road, but I haven't heard of libjava splitting.



What are the fallbacks for SystemRandomImpl if /dev/urandom or the
rtlGenRandomFN/CryptGenRandom aren't available?  Is that something
you'll bake into TLR or will you do it here?

>

I think it's better to leave it to consumers (TLR/SplittableRandom) as
they know what's good-enough for them. The API allows for arbitrary
number of bytes to be generated and I don't have an easy means of
generating more than 8 "random" bytes just from System.nanoTime() and
System.currentTimeMillis() short of using SecureRandom as a fall-back.


webrev.03 only has new code, no changes yet to TLR/SplittableRandom, so 
I'm not yet quite following where TLR/SR will be changing.  Also, what 
is proposed for platforms that aren't Unix/Windows?  Should there be a 
generic fallback mechanism like ThreadedSeedGenerator?  (Note, I'm not 
suggesting using it, it's rather...SSLLLOOO...)



The problem is also how to make access to this functionality for
different consumers that are located in different packages (java.util,
java.util.concurrent) and make it somehow usable also for external
access. There is a desire to use this also from stand-alone builds of
java.util.concurrent utilities. That's why my initial approach for
SystemRandom used a public API in java.util.

The approach used with sun.misc.Unsafe is probably not going to work for
user code in JDK9 with modules, as sun.misc will not be globally
exported. Are any non J2SE packages going to be globally exported? I see
jdk and jdk.net are already mentioned as such globally exported packages
in modules.xml...


That's a good question for Alan/Mandy/company.


Martin wrote:


https://bugs.openjdk.java.net/browse/JDK-8047769


If you've been following this bug, I've figured why the NativePRNG$*
classes are initing and thus opening the /dev/random,urandom.  This
definitely needs some adjustment.


Something like the following could be used in NativePRNG and
URLSeedGenerator:

http://cr.openjdk.java.net/~plevart/misc/FileInputStreamPool/FileInputStreamPool.java


(See the other active thread in core-libs-dev.)

Brad

Re: RFR: JDK-8047769 SecureRandom should be more frugal with file descriptors

[Bradford Wetmore]

Just to followup, I've analyzed the whole PIT run.  The second one's 
call stack is identical to:


JDK-8067344: Adjust 
java/lang/invoke/LFCaching/LFGarbageCollectedTest.java for recent 
changes in java.lang.invoke


So, really the only problem is the use of Asserts in your test case.

Brad


Looks like you have a committer status, will you be pushing this?


I can, yes. As soon as we clear-out the remaining questions, right?


Yes.  The comments below are minor and shouldn't need another review
cycle.  I have started a JPRT job for you, I'll run it through "core"
target which will give us:

jdk_lang, jdk_math, jdk_util, jdk_io, jdk_net, jdk_nio, jdk_security*,
jdk_rmi, jdk_text, jdk_time, jdk_other, core_tools.

Anything else?  I'm off tomorrow, I should have full results Wed.

Here are the preliminary results for the jobs that have finished.
jdk.testlibrary.Asserts is causing compilation errors, additional
comments below:

/opt/jprt/T/P1/003505.brwetmor/s/jdk/test/sun/security/provider/FileInputStreamPool/FileInputStreamPoolTest.java:33:
error: package jdk.testlibrary does not exist
import static jdk.testlibrary.Asserts.*;
  ^
/opt/jprt/T/P1/003505.brwetmor/s/jdk/test/sun/security/provider/FileInputStreamPool/FileInputStreamPoolTest.java:52:
error: cannot find symbol
 assertEquals(bytes.length, nread, "short read");
 ^
   symbol:   method assertEquals(int,int,String)
   location: class FileInputStreamPoolTest
/opt/jprt/T/P1/003505.brwetmor/s/jdk/test/sun/security/provider/FileInputStreamPool/FileInputStreamPoolTest.java:53:
error: cannot find symbol
 assertTrue(Arrays.equals(readBytes, bytes),
 ^
   symbol:   method assertTrue(boolean,String)
   location: class FileInputStreamPoolTest
3 errors

TEST RESULT: Failed. Compilation failed: Compilation failed

I'm also getting failures in the following test.  I unfortunately have
to leave now, so don't have time to look at this.  But it did mention
"seed" so I'm mentioning it here.

TEST: java/lang/invoke/LFCaching/LFGarbageCollectedTest.java

ACTION: main -- Failed. Execution failed: `main' threw exception:
java.lang.Error: 36 of 39 test cases FAILED! Rerun the test with the
same "-Dseed=" option as in the log file!
REASON: User specified action: run main/othervm LFGarbageCollectedTest
TIME:   213.406 seconds
messages:
command: main LFGarbageCollectedTest
reason: User specified action: run main/othervm LFGarbageCollectedTest
elapsed time (seconds): 213.406
STDOUT:
-Dseed=6102311124531075592
-DtestLimit=2000
Number of iterations according to -DtestLimit is 153 (1989 cases)
Code cache size is 251658240 bytes
Non-profiled code cache size is 123058253 bytes
Number of iterations limited by code cache size is 84 (1092 cases)
Number of iterations limited by non-profiled code cache size is 41 (533
cases)
Number of iterations is set to 41 (533 cases)
Not enough time to continue execution. Interrupted.
STDERR:
Iteration 0:
Tested LF caching feature with MethodHandles.foldArguments method.
java.lang.AssertionError: Error: Lambda form is not garbage collected
 at LFGarbageCollectedTest.doTest(LFGarbageCollectedTest.java:81)
 at
LambdaFormTestCase$TestRun.doIteration(LambdaFormTestCase.java:139)
 at LambdaFormTestCase$$Lambda$2/5042013.call(Unknown Source)
 at
jdk.testlibrary.TimeLimitedRunner.call(TimeLimitedRunner.java:71)
 at LambdaFormTestCase.runTests(LambdaFormTestCase.java:201)
 at LFGarbageCollectedTest.main(LFGarbageCollectedTest.java:105)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

Re: Lower overhead String encoding/decoding

[Xueming Shen]

On 12/30/14 3:25 PM, Richard Warburton wrote:

Hi,

Thanks for looking at this patch and agreeing to sponsor - I've
pushed fixes for these issues at:

http://cr.openjdk.java.net/~rwarburton/string-patch-webrev-10/



Hope everyone has had a happy holiday period. Just following up on 
this issue to make sure that you have all the information needed and 
I've not been blocking the RFE process. Got a bit concerned due to 
radio silence.


regards,

  Richard Warburton

http://insightfullogic.com
@RichardWarburto 

Hi Richard,

My apology for the delay, things are little slow during this period of 
the year:-)
I will sponsor the rfe and prepare for the CCC (internally). We may need 
go through

the api and the implementation one more time.

Regards,
-Sherman

Re: RFR: 5050783: Throwable convenience method: String getStackTraceString()

[Claes Redestad]

Hi,

seems like a reasonable convenience method. :-)

Process stuff: OCA/JSPA signing aside, the patch needs to be submitted 
in its
entirety to some part of the OpenJDK infrastructure before it can be 
accepted.
Attaching the patch in an e-mail to this mailing list is acceptable for 
a small
patch like this when lacking upload privileges to cr.openjdk.java.net. A 
sponsor

could help you with the uploading etc.

This request adds a new public method, which means some internal approval
will be necessary. This is something you'll need help with from an Oracle
sponsor. I could volunteer to initiate the requests internally.

Code feedback:

The test relies on the message generated when provoking a division by
zero. I'm not sure this is a good idea nor the place to enforce this
message format with a test. Perhaps just throwing a RuntimeException
with some specified message and test for that?

May I suggest to also add a test to ensure the method behaves well for an
exception with fillInStackTrace overridden to produce stackless exceptions?

Just a thought... I noticed PrintWriter, PrintStream and StringBuilder all
inherit from Appendable, so the PrintStreamOrWriter/StackTracePrinter 
classes
could be replaced with a single StackTraceAppendable taking an 
Appendable. One
static class instead of 1 abstract and 3 concrete static inner classes 
could work
out to be a good deal, but there's some odd mechanics in 
BufferedWriter/PrintWriter
to use the value of the line.separator property at object creation time 
which
would be hard to support without exposing the lineSeparator fields to 
Throwable.

Perhaps a future consideration.

Performance characteristics might change ever so slightly either way, so it
would be nice with some microbenchmarks to quantify.

/Claes

On 2014-12-31 11:51, Ivan St. Ivanov wrote:

Hello,

I am Ivan from the Bulgarian Java User Group. As part of our Adopt OpenJDK
efforts, we organized hackathon beginning of this month in Sofia. There we
worked on the following JIRA issue:

https://bugs.openjdk.java.net/browse/JDK-5050783

Unfortunately none of us has author privileges, so the issue is still
unassigned. But, nevertheless, our JUG has already signed JSPA and some of
our members (including myself) have signed OCA. And we informed in a
special email this mailing list about our plans prior to the meeting.

Our hackathon was already documented by our JUG member Mihail Stoynov:

http://mihail.stoynov.com/2014/12/12/building-openjdk-and-submitting-a-solution-on-a-feature-request-with-the-bulgarian-java-user-group/

We used the days after the meeting for offline polishing our contribution
and adding some more unit tests. Here is the result of our work:

http://dmitryalexandrov.net/~bgjug/5050783/webrev.00/

We would be very happy if you are able to review it, share your feedback
and guide us through the contribution process as this is our first attempt.

Thanks and have a very successful year!
Ivan

Re: RFR(S): 8067471: Use private static final char[0] for empty Strings

[Lev Priima]

Thanks Ivan!

I've updated: http://cr.openjdk.java.net/~lpriima/8067471/webrev.05/.

Best Regards,
Lev

RFR: 5050783: Throwable convenience method: String getStackTraceString()

[Ivan St. Ivanov]

Hello,

I am Ivan from the Bulgarian Java User Group. As part of our Adopt OpenJDK
efforts, we organized hackathon beginning of this month in Sofia. There we
worked on the following JIRA issue:

https://bugs.openjdk.java.net/browse/JDK-5050783

Unfortunately none of us has author privileges, so the issue is still
unassigned. But, nevertheless, our JUG has already signed JSPA and some of
our members (including myself) have signed OCA. And we informed in a
special email this mailing list about our plans prior to the meeting.

Our hackathon was already documented by our JUG member Mihail Stoynov:

http://mihail.stoynov.com/2014/12/12/building-openjdk-and-submitting-a-solution-on-a-feature-request-with-the-bulgarian-java-user-group/

We used the days after the meeting for offline polishing our contribution
and adding some more unit tests. Here is the result of our work:

http://dmitryalexandrov.net/~bgjug/5050783/webrev.00/

We would be very happy if you are able to review it, share your feedback
and guide us through the contribution process as this is our first attempt.

Thanks and have a very successful year!
Ivan