Re: User model stacking: current status

[Remi Forax]

> From: "Brian Goetz" 
> To: "daniel smith" 
> Cc: "valhalla-spec-experts" 
> Sent: Tuesday, June 14, 2022 1:04:39 AM
> Subject: Re: User model stacking: current status

> I've done a little more shaking of this tree. It involves keeping the notion
> that the non-identity buckets differ only in the treatment of their val
> projection, but makes a further normalization that enables the buckets to
> mostly collapse away.

> "value class X" means:

> - Instances are identity-free
> - There are two types, X.ref (reference, nullable) and X.val (direct,
> non-nullable)
> - Reference types are atomic, as always
> - X is an alias for X.ref

> Now, what is the essence of B2? B2 means not "I hate zeros", but "I don't like
> that uninitialized variables are initialized to zero." It doesn't mean the 
> .val
> projection is meaningless, it means that we don't trust arbitrary clients with
> it. So, we can make a slight adjustment:

> - The .val type is always there, but for "B2" classes, it is *inaccessible
> outside the nest*, as per ordinary accessibility.

> This means that within the nest, code that understands the restrictions can,
> say, create `new X.val[7]` and expose it as an `X[]`, as long as it doesn't 
> let
> the zero escape. This gives B2 classes a lot more latitude to use the .val 
> type
> in safe ways. Basically: if you don't trust people with the .val type, don't
> let the val type escape.
I don't trust myself with a B2.val. 
The val type for B2 should not exist at all, otherwise any libraries using the 
reflection can do getClass() on a X.val[] (even typed as a X[]). 

> There's a bikeshed to paint, but it might look something like:

> value class B2 {
> private class val { }
> }

> or, flipping the default:

> value class B3a {
> public class val { }
> }

> So B2 is really a B3a whose value projection is encapsulated.
and here you lost me, .ref and .val are supposed to be projection types not 
classes, at runtime there is only one class. 

> The other bucket, B3n, I think can live with a modifier:

> non-atomic value class B3n { }

> While these are all the same buckets as before, this feels much more like "one
> new bucket" (the `non-atomic` modifier is like `volatile` on a field; we don't
> think of this as creating a different bucket of fields.)
yes ! 

> Summary:

> class B1 { }
> value class B2 { private class val { } }
> value class B3a { }
> non-atomic value class B3n { }

> Value class here is clearly the star of the show; all value classes are 
> treated
> uniformly (ref-default, have a val); some value classes encapsulate the val
> type; some value classes further relax the integrity requirements of instances
> on the heap, to get better flattening and performance, when their semantics
> don't require it.

> It's an orthogonal choice whether the default is "val is private" and "val is
> public".
It makes B2.val a reality, but B3 has no sane default value otherwise it's a 
B3, so B2.val should not exist. 

regards, 
Rémi 

> On 6/3/2022 3:14 PM, Brian Goetz wrote:

>> Continuing to shake this tree.

>> I'm glad we went through the exploration of "flattenable B3.ref"; while I 
>> think
>> we probably could address the challenges of tearing across the null channel /
>> data channels boundary, I'm pretty willing to let this one go. Similarly I'm
>> glad we went through the "atomicity orthogonal to buckets" exploration, and 
>> am
>> ready to let that one go too.

>> What I'm not willing to let go of us making atomicity explicit in the model. 
>> Not
>> only is piggybacking non-atomicity on something like val-ness too subtle and
>> surprising, but non-atomicity seems like it is a property that the class 
>> author
>> needs to ask for. Flatness is an important benefit, but only when it doesn't
>> get in the way of safety.

>> Recall that we have three different representation techniques:

>> - no-flat -- use a pointer
>> - low-flat -- for sufficiently small (depending on size of atomic 
>> instructions
>> provided by the hardware) values, pack multiple fields into a single,
>> atomically accessed unit.
>> - full-flat -- flatten the layout, access individual individual fields 
>> directly,
>> may allow tearing.

>> The "low-flat" bucket got some attention recently when we discovered that 
>> there
>> are usable 128-bit atomics on Intel (based on a recent revision of the chip
>> spec), but this is not a slam-dunk; it requires some serious compiler heroics
>> to pack multiple values into single accesses. But there may be targets of
>> opportunity here for single-field values (like Optional) or final fields. And
>> we can always fall back to no-flat whenever the VM feels like it.

>> One of the questions that has been raised is how similar B3.ref is to B2,
>> specifically with respect to atomicity. We've gone back and forth on this.

>> Having shaken the tree quite a bit, what feels like the low energy state to 
>> me
>> right now is:

>> - The ref type of all on-identity classes are treated 

Re: User model stacking: current status

[Remi Forax]

> From: "Brian Goetz" 
> To: "daniel smith" 
> Cc: "valhalla-spec-experts" 
> Sent: Friday, June 3, 2022 9:14:39 PM
> Subject: Re: User model stacking: current status

> Continuing to shake this tree.

> I'm glad we went through the exploration of "flattenable B3.ref"; while I 
> think
> we probably could address the challenges of tearing across the null channel /
> data channels boundary, I'm pretty willing to let this one go. Similarly I'm
> glad we went through the "atomicity orthogonal to buckets" exploration, and am
> ready to let that one go too.

> What I'm not willing to let go of us making atomicity explicit in the model. 
> Not
> only is piggybacking non-atomicity on something like val-ness too subtle and
> surprising, but non-atomicity seems like it is a property that the class 
> author
> needs to ask for. Flatness is an important benefit, but only when it doesn't
> get in the way of safety.

> Recall that we have three different representation techniques:

> - no-flat -- use a pointer
> - low-flat -- for sufficiently small (depending on size of atomic instructions
> provided by the hardware) values, pack multiple fields into a single,
> atomically accessed unit.
> - full-flat -- flatten the layout, access individual individual fields 
> directly,
> may allow tearing.

> The "low-flat" bucket got some attention recently when we discovered that 
> there
> are usable 128-bit atomics on Intel (based on a recent revision of the chip
> spec), but this is not a slam-dunk; it requires some serious compiler heroics
> to pack multiple values into single accesses. But there may be targets of
> opportunity here for single-field values (like Optional) or final fields. And
> we can always fall back to no-flat whenever the VM feels like it.

> One of the questions that has been raised is how similar B3.ref is to B2,
> specifically with respect to atomicity. We've gone back and forth on this.

> Having shaken the tree quite a bit, what feels like the low energy state to me
> right now is:

> - The ref type of all on-identity classes are treated uniformly; B3.ref and
> B2.ref are translated the same, treated the same, have the same atomicity, the
> same nullity, etc.
> - The only difference across the spectrum of non-identity classes is the
> treatment of the val type. For B2, this means the val type is *illegal*; for
> B3, this means it is atomic; for B3n, it is non-atomic (which in practice will
> mean more flatness.)
> - (controversial) For all types, the ref type is the default. This means that
> some current value-based classes can migrate not only to B2, but to B3 or B3n.
> (And that we could migrate to B2 today and further to B3 tomorrow.)

> While this is technically four flavors, I don't think it needs to feel that
> complex. I'll pick some obviously silly modifiers for exposition:

> - class B1 { }
> - zero-hostile value class B2 { }
> - value class B3 { }
> - tearing-happy value class B3n { }

> In other words: one new concept ("value class"), with two sub-modifiers
> (zero-hostile, and tearing-happy) which affect the behavior of the val type
> (forbidden for B2, loosened for B3n.)

> For heap flattening, what this gets us is:

> - B1 -- no-flat
> - B2, B3.ref, B3n.ref -- low-flat atomic (with null channel)
> - B3 -- low-flat (atomic, no null channel)
> - B3n -- full-flat (non-atomic, no null channel)

> This is a slight departure from earlier tree-shakings with respect to tearing.
> In particular, refs do not tear at all, so programs that use all refs will
> never see tearing (but it is still possible to get a torn value using .val and
> then box that into a ref.)

> If you turn this around, the declaration-site decision tree becomes:

> - Do I need identity (mutability, subclassing, aliasing)? Then B1.
> - Are uninitialized values unacceptable? Then B2.
> - Am I willing to tolerate tearing to enable more flattening? Then B3n.
> - Otherwise, B3.

> And the use-site decision tree becomes:

> - For B1, B2 -- no choices to make.
> - Do I need nullity? Then .ref
> - Do I need atomicity, and the class doesn't already provide it? Then .ref
> - Otherwise, can use .val

> The main downside of making ref the default is that people will grumble about
> having to say .val at the use site all the time. And they will! And it does
> feel a little odd that you have to opt into val-ness at both the declaration
> and use sites. But it unlocks a lot of things (see Kevin's list for more):

> - The default name is the safest version.
> - Every unadorned name works the same way; it's always a reference type. You
> don't need to maintain a mental database around "which kind of name is this".
> - Migration from B1 -> B2 -> B3 is possible. This is huge (and more than we 
> had
> hoped for when we started this game.)

> (The one thing to still worry about is that while refs can't tear, you can 
> still
> observe a torn value through a ref, if someone tore it and then boxed it. I
> don't see how we defend against this, but 

Re: Anonymous value classes

[Remi Forax]

- Original Message -
> From: "daniel smith" 
> To: "valhalla-spec-experts" 
> Sent: Friday, June 3, 2022 6:18:42 PM
> Subject: Re: Anonymous value classes

>> On Jun 3, 2022, at 10:15 AM, Dan Smith  wrote:
>> 
>> Our javac prototype has long included support for a 'value' keyword after 
>> 'new'
>> to indicate that an anonymous class is a value class
> 
> (I see Remi brought this up in the list in July 2018, which is probably what
> inspired the prototype implementation. There wasn't really any followup
> discussion.)

I need it when trying to implement List.of()/Map.of() with few elements using 
value classes.

Rémi

Re: Anonymous value classes

[Remi Forax]

> From: "Brian Goetz" 
> To: "daniel smith" , "valhalla-spec-experts"
> 
> Sent: Friday, June 3, 2022 6:21:26 PM
> Subject: Re: Anonymous value classes

> There is no chance to get any calling-convention optimization here, since the
> concrete class name will not show up in any method descriptor (or preload
> attribute). There is no chance to get any heap flattening here, since the
> concrete class name will not show up in any field descriptor or `newarray`
> operand.
Nope, anonymous classes are anonymous only for Java not in the bytecode, by 
example 

var box = new Object() { }; 
Supplier supplier = () -> box; 
var list = Arrays.asList(box); 

is translated to 

Code: 
0: new #7 // class AnonymousClassNameLeaking$1 
3: dup 
4: invokespecial #9 // Method AnonymousClassNameLeaking$1."":()V 
7: astore_1 

8: aload_1 
9: invokedynamic #10, 0 // InvokeDynamic 
#0:get:(LAnonymousClassNameLeaking$1;)Ljava/util/function/Supplier; 
14: astore_2 

15: iconst_1 
16: anewarray #7 // class AnonymousClassNameLeaking$1 
19: dup 
20: iconst_0 
21: aload_1 
22: aastore 
23: invokestatic #14 // Method 
java/util/Arrays.asList:([Ljava/lang/Object;)Ljava/util/List; 
26: astore_3 

Here, the anonymous class name appears as parameter of invokedynamic, at 
runtime the field (box is captured) of the lambda proxy is also typed 
LAnonymousClassNameLeaking$1; and any varargs will create an array of the 
anonymous class. 

Rémi 

> On 6/3/2022 12:15 PM, Dan Smith wrote:

>> Our javac prototype has long included support for a 'value' keyword after 
>> 'new'
>> to indicate that an anonymous class is a value class:

>> Runnable r = new value Runnable() {
>> public void run() { x.foo(); }
>> };

>> Is this something we'd like to preserve as a language feature?

>> Arguments for:

>> - Allows the semantics of "I don't need identity" to be conveyed (often true 
>> for
>> anonymous classes).

>> - Gives the JVM more information for optimization. If we don't need a heap
>> object, evaluating the expression may be a no-op.

>> Arguments against:

>> - Opens a Pandora's box of syntax: what other keywords can go there? 
>> 'identity'?
>> 'primitive'? 'static'? 'record'?

>> - Because there's no named type, there are significantly fewer opportunities 
>> for
>> optimization—you're probably going to end up with a heap object anyway.

>> - Value classes are primarily focused on simple data-carrying use cases, but 
>> any
>> data being carried by an anonymous class is usually incidental. A new 
>> language
>> feature would draw a lot of attention to this out-of-the-mainstream use case.

>> - In the simplest cases, you can use a lambda instead, and there the API
>> implementation has freedom to implement lambdas with value classes if it 
>> turns
>> out to be useful.

>> - The workaround—declare a local class instead—is reasonably straightforward 
>> for
>> the scenarios where there's a real performance bottleneck that 'value' can 
>> help
>> with.

Re: User model stacking: current status

[Remi Forax]

> From: "Brian Goetz" 
> To: "daniel smith" 
> Cc: "valhalla-spec-experts" 
> Sent: Sunday, May 8, 2022 6:32:09 PM
> Subject: Re: User model stacking: current status

> To track the progress of the spiral:

> - We originally came up with the B2/B3 division to carve off B2 as the "safe
> subset", where you get less flattening but nulls and more integrity. This
> provided a safe migration target for existing VBCs, as well as a reasonable
> target for creating new VBCs that want to be mostly class-like but enjoy some
> additional optimization (and shed accidental identity for safety reasons.)

> - When we put all the flesh on the bones of B2/B3, there were some undesirable
> consequences, such as (a) tearing was too subtle, and (b) both the semantics
> and cost model differences between B2/B3 were going to be hard to explain (and
> in some cases, users have bad choices between semantics and performance.)

> - A few weeks ago, we decided to more seriously consider separating atomicity
> out as an explicit thing on its own. This had the benefit of putting semantics
> first, and offered a clearer cost model: you could give up identity but keep
> null-default and integrity (B2), further give up nulls to get some more 
> density
> (B3.val), and further further give up atomicity to get more flatness
> (non-atomic B3.) This was honest, but led people to complain "great, now there
> are four buckets."

> - We explored making non-atomicity a cross-cutting concern, so there are two 
> new
> buckets (VBC and primitive-like), either of which can choose their atomicity
> constraints, and then within the primitive-like bucket, the .val and .ref
> projections differ only with respect to the consequences of nullity. This felt
> cleaner (more orthogonal), but the notion of a non-atomic B2 itself is kind of
> weird.

> So where this brings us is back to something that might feel like the
> four-bucket approach in the third bullet above, but with two big differences:
> atomicity is an explicit property of a class, rather than a property of
> reference-ness, and a B3.ref is not necessarily the same as a B2. This
> recognizes that the main distinction between B2 or B3 is *whether a class can
> tolerate its zero value.*

> More explicitly:

> - B1 remains unchanged

> - B2 is for "ordinary" value-based classes. Always atomic, always nullable,
> always reference; the only difference with B1 is that it has shed its 
> identity,
> enabling routine stack-based flattening, and perhaps some heap flattening
> depending on VM sophistication and heroics. B2 is a good target for migrating
> many existing value-based classes.

> - B3 means that a class can tolerate its zero (uninitialized) value, and
> therefore gives rise to two types, which we'll call B3.ref and B3.val. The
> former is a reference type and is therefore nullable and null-default; the
> latter is a direct/immediate/value type whose default is zero.

> - B3 classes can further be marked non-atomic; this unlocks greater flattening
> in the heap at the cost of tearing under race, and is suitable for classes
> without cross-field invariants. Non-atomicity accrues equally to B3.ref and
> B3.val; a non-atomic B3.ref still tears (and therefore might expose its zero
> under race, as per friday's discussions.)

> Syntactically (reminder: NOT an invitation to discuss syntax at this point),
> this might look like:

> class B1 { } // identity, reference, atomic

> value-based class B2 { } // non-identity, reference, atomic

> value class B3 { } // non-identity, .ref and .val, both atomic

> non-atomic value class B3 { } // similar to B3, but both are non-atomic

> So, two new (but related) class modifiers, of which one has an additional
> modifier. (The spelling of all of these can be discussed after the user model
> is entirely nailed down.)

> So, there's a monotonic sequence of "give stuff up, get other stuff":

> - B2 gives up identity relative to B1, gains some flattening
> - B3 optionally gives up null-defaultness relative to B2, yielding two types,
> one of which sheds some footprint
> - non-atomic B3 gives up atomicity relative to B3, gaining more flatness, for
> both type projections
There is also something we should talk, using non-atomic value classes does not 
mean automatically better performance. 
It's something i've discovered trying to implement HashMap (more Map.of() in 
fact) using value classes. 
Updating a value class in the heap requires more writes, more memory traffic 
than just updating pointers so depending on the algorithm, you may see 
performance degradation compared to a pointer based implementation. 

So even if we provide non-atomic B3, performance can be worst than using atomic 
B3, sadly gaining more flatness does not necessarily translate into better 
performance. 

Rémi 

Side Note: using more pointers mean more pressure to the GC, but it's very hard 
to quantify that pressure so maybe overall the system is more performant but 
given that JMH tests 

Re: User model: terminology

[Remi Forax]

> From: "Brian Goetz" 
> To: "valhalla-spec-experts" 
> Sent: Wednesday, May 4, 2022 5:05:24 PM
> Subject: User model: terminology

> Let's talk about terminology. (This is getting dangerously close to a
> call-for-bikeshed, so let's exercise restraint.)

> Currently, we have primitives and classes/references, where primitives have
> box/wrapper reference companions. The original goal of Bucket 3 was to model
> primitive/box pairs. We have tentatively been calling these "primitives", but
> there are good arguments why we should not overload this term.

> We have tentatively assigned the phrase "value class" to all identity-free
> classes, but it is also possible we can use value to describe what we've been
> calling primitives, and use something else (identity-free, non-identity) to
> describe the bigger family.

> So, in our search for how to stack the user model, we should bear in mind that
> names that have been tentatively assigned to one thing might be a better fit
> for something else (e.g., the "new primitives"). We are looking for:

> - A term for all non-identity classes. (Previously, all classes had identity.)

I've used the term "immediate", immediate object vs reference object. 

> - A term for what we've been calling atomicity: that instances cannot appear 
> to
> be torn, even when published under race. (Previously, all classes had this
> property.)

As you said, the default should be non-tearable. 
I believe that we should use a term that indicates that the object is composed 
of several values, a term like "compound", "composite" or perhaps "aggregate". 
I think i prefer compound due to its Latin root. 

The other solution is instead of saying that it's non-terable by default, is to 
force users to always use a keyword to indicate the "atomiciy" state, 
(non-)splitable, (non-)secable (secable is more or less the latin equivalent of 
the greek atomic). 

> - A term for those non-identity classes which do not _require_ a reference.
> These must have a valid zero, and give rise to two types, what we've been
> calling the "ref" and "val" projections.

I like "zero-default" (as opposite of null-default) but mostly because it's a 
valid hyphenated keyword. 

> - A term for what we've been calling the "ref" and "val" projections.

Technically, what we called the ref projection is now a nullable projection, we 
are adding null into the set of possible values. 

> Let's start with _terms_, not _declaration syntax_.

Rémi 

Re: Null channels (was: User model stacking)

[Remi Forax]

- Original Message -
> From: "Brian Goetz" 

[...]

> 
> What I'm trying to do here is decomplect flattening from nullity. Right
> now, we have an unfortunate interaction which both makes certain
> combinations impossible, and makes the user model harder to reason about.
> 
> Identity-freedom unlocks flattening in the stack (calling convention.)
> The lesson of that exercise (which was somewhat surprising, but good) is
> that nullity is mostly a non-issue here -- we can treat the nullity
> information as just being an extra state component when scalarizing,
> with some straightforward fixups when we adapt between direct and
> indirect representations.  This is great, because we're not asking users
> to choose between nullability and flattening; users pick the combination
> of { identity, nullability } they want, and they get the best flattening
> we can give:
> 
>     case (identity, _) -> 1; // no flattening
>     case (non-identity, non-nullable) -> nFields;  // scalarize fields
>     case (non-identity, nullable) -> nFields + 1;  // scalarize fields
> with extra null channel
> 
> Asking for nullability on top of non-identity means only that there is a
> little more "footprint" in the calling convention, but not a qualitative
> difference.  That's good.
> 
> In the heap, it is a different story.  What unlocks flattening in the
> heap (in addition to identity-freedom) is some permission for
> _non-atomicity_ of loads and stores.  For sufficiently simple classes
> (e.g., one int field) this is a non-issue, but because loads and stores
> of references must be atomic (at least, according to the current JMM),
> references to wide values (B2 and B3.ref) cannot be flattened as much as
> B3.val.  There are various tricks we can do (e.g., stuffing two 32 bit
> fields into a 64 bit atomic) to increase the number of classes that can
> get good flattening, but it hits a wall much faster than "primitives".
> 
> What I'd like is for the flattening story on the heap and the stack to
> be as similar as possible.  Imagine, for a moment, that tearing was not
> an issue.  Then where we would be in the heap is the same story as
> above: no flattening for identity classes, scalarization in the heap for
> non-nullable values, and scalarization with an extra boolean field
> (maybe, same set of potential optimizations as on the stack) for
> nullable values.  This is very desirable, because it is so much easier
> to reason about:
> 
>  - non-identity unlocks scalarization on the stack
>  - non-atomicity unlocks flattening in the heap
>  - in both, ref-ness / nullity means maybe an extra byte of footprint
> compared to the baseline
> 
> (with additional opportunistic optimizations that let us get more
> flattening / better footprint in various special cases, such as very
> small values.)

yes, choosing (non-)identity x (non-)nullability x (non-)atomicity at 
declaration site makes the performance model easier to understand.
At declaration site, there are still nullability x atomicity with .ref and 
volatile respectively.

I agree with John that being able to declare array items volatile is missing 
but i believe it's an Array 2.0 feature.

Once we get universal generics, what we win is that not only ArrayList is 
compact on heap but ArrayList too. 

Rémi

Re: [External] : Re: User model stacking

[Remi Forax]

> From: "Brian Goetz" 
> To: "Dan Heidinga" 
> Cc: "valhalla-spec-experts" 
> Sent: Thursday, April 28, 2022 1:15:08 AM
> Subject: Re: [External] : Re: User model stacking

> Let me try and put some more color on the bike shed (but, again, let’s focus 
> on
> model, not syntax, for now.)

> We have two axes of variation we want to express with non-identity classes:
> atomicity constraints, and whether there is an additional zero-default
> companion type. These can be mostly orthogonal; you can have either, neither,
> or both. We've been previously assuming that "primitiveness" lumps this all
> together; primitives get more flattening, primitives can be
> non-nullable/zero-default, primitives means the good name goes to the "val"
> type. Primitive-ness implicitly flips the "safety vs performance" priority,
> which has been bothering us because primitives also code like a class. So we
> were trying to claw back some atomicity for primitives.

> But also, we're a little unhappy with B2 because B2 comes with _more_ 
> atomicity
> than is necessarily needed; a B2 with no invariants still gets less flattening
> than a B3. That's a little sad. And also that it seems like a gratuitous
> difference, which makes the user model more complicated. So we’re suggesting
> restacking towards:

> - Value classes are those without identity
> - Value classes can be atomic or non-atomic, the default is atomic (safe by
> default)
> - Value classes can further opt into having a "val" projection (name TBD, val 
> is
> probably not it)
> - Val projections are non-nullable, zero-default — this is the only difference
> - Both the ref and val projections inherit the atomicity constraints of the
> class, making atomicity mostly orthogonal to ref/val/zero/null

Now that the model is clearer, let's try to discuss about the val projection. 

Once we have universal generics, we will have an issue with value type with 
zero-default, there are a lot of API in the JDK that explicitly specify that 
they return/pass null as parameter, 
by example, Map.get(), for those call, we need a way to say that the type is 
not T but T | null. 
The current proposal is to use T.ref for that. 

Now, Kevin and Brian thinks that for zero-default value type, in the language, 
Complex.val should be used instead of Complex. 
Lets see how it goes 
1/ There is a difference between Foo and Foo.ref for generics, Foo is a class 
while Foo.ref is a type. 
The idea of using Complex.val means that the relationship is reversed, 
Complex is the type and Complex.val is the class. 
If we ride with that horse, it means that in universal generics, we should not 
use T but T.val apart when we want T.val | null that can be spelled T. 

2/ Because Complex.val is a class and Complex is a type, we have a weird 
dis-symmetry, 
User will declare a class Complex, but to create a Complex, they will have to 
use new Complex.val(). 
As a user this is weird. 

3/ This may change but currently, Foo.class exists but Foo.ref.class is not 
allowed, you have to use a method to get the projection, 
something like Foo.class.getMeTheProjectionPlease(). 
With .val being the default, it means that Complex.val.class exists while 
Complex.class does not. 
Same to get the default value, Complex.class.getDefaultValue() will not 
compile, it should be Complex.val.class.getDefaultValue(). 
Again weird. 

4/ It's a double opt-in, people have to opt-in at declaration site by asking 
for a zero-default value type but that is not enough, 
it only works if the type val is uses at use site. I don't know any feature in 
Java that requires a double opt-in. 

5/ It's easy to forget a ".val". To work, people will have to pepper .val 
everywhere and it will be easy to miss one occurrence. 
Depending on where the ".val" is missed, performance will suffer. This is 
something i see when beginners dab with generics 
for the first times, the compiler emits a warning because they miss one pair of 
<> somewhere in the code. 
To avoid missing , compilers/IDEs will try to rely on null analysis to emit a 
warning when Complex.val should be used instead of Complex. 
Again, the relationship is seen in the wrong direction, with .ref, you get 
performance by default, the compiler does not compile 
if you try to return/store null so by design the compiler helps you to write 
the right code. 

6/ Zero-default value type does not imply non-atomic anymore, 
so a zero-default value type is not more dangerous that a null-default value 
type anymore. 

regards, 
Rémi 

Re: User model stacking

[Remi Forax]

- Original Message -
> From: "Dan Heidinga" 
> To: "Brian Goetz" 
> Cc: "valhalla-spec-experts" 
> Sent: Wednesday, April 27, 2022 8:51:15 PM
> Subject: Re: User model stacking

> I'm trying to understand how this refactoring fits the VM physics.
> 
> In particular, __non-atomic & __zero-ok fit together at the VM level
> because the VM's natural state for non-atomic (flattened) data is zero
> filled.  When those two items are decoupled, I'm unclear on what the
> VM would offer in that case.  Thoughts?

__non-atomic but ! __zero-ok means you have an additional bit that indicate if 
it's null or not.
from the VM POV, fields are still zero filled but you have a way to encode null.

> 
> How does "__non-atomic __non-id class B2a { }" fit with the "no new
> nulls" requirements?

For me, i may be wrong, the "no new nulls" requirements is from the POV of the 
language / user, not from the POV of the VM.
The VM may have several encodings of null internally. 

> 
> --Dan

Rémi

> 
> On Wed, Apr 27, 2022 at 12:45 PM Brian Goetz  wrote:
>>
>> Here’s some considerations for stacking the user model.  (Again, please let’s
>> resist the temptation to jump to the answer and then defend it.)
>>
>> We have a stacking today which says:
>>
>>  - B1 is ordinary identity classes, giving rise to a single reference type
>>  - B2 are identity-free classes, giving rise to a single reference type
>>  - B3 are flattenable identity-free classes, giving rise to both a reference
>>  (L/ref) and primitive (Q/val) type.
>>
>> This stacking has some pleasant aspects.  B2 differs from B1 by “only one 
>> bit”:
>> identity.  The constraints on B2 are those that come from the lack of 
>> identity
>> (mutability, extensibility, locking, etc.)  B2 references behave like the
>> object references we are familiar with; nullability, final field guarantees,
>> etc.  B3 further makes reference-ness optional; reference-free B3 values give
>> up the affordances of references: they are zero-default and tearable.  This
>> stacking is nice because it can framed as a sequence of “give up some X, get
>> some Y”.
>>
>> People keep asking “do we need B2, or could we get away with B1/B3”.  The 
>> main
>> reason for having this distinction is that some id-free classes have no
>> sensible default, and so want to use null as their default.  This is a
>> declaration-site property; B3 means that the zero value is reasonable, and 
>> use
>> sites can opt into / out of  zero-default / nullity.  We’d love to compress
>> away this bucket but forcing a zero on classes that can’t give it a 
>> reasonable
>> interpretation is problematic.  But perhaps we can reduce the visibility of
>> this in the model.
>>
>> The degrees of freedom we could conceivably offer are
>>
>> { identity or not, zero-capable or not, atomic or not } x { use-site,
>> declaration-site }
>>
>> In actuality, not all of these boxes make sense (disavowing the identity of 
>> an
>> ArrayList at the use site), and some have been disallowed by the stacking 
>> (some
>> characteristics have been lumped.)  Here’s another way to stack the
>> declaration:
>>
>>  - Some classes can disavow identity
>>  - Identity-free classes can further opt into zero-default (currently, B3,
>>  polarity chosen at use site)
>>  - Identity-free classes can further opt into tearability (currently, B3,
>>  polarity chosen at use site)
>>
>> It might seem the sensible move here is to further split B3 into B3a and B3b
>> (where all B3 support zero default, and a/b differ with regard to whether
>> immediate values are tearable).  But that may not be the ideal stacking,
>> because we want good flattening for B2 (and B3.ref) also.  Ideally, the
>> difference between B2 and B3.val is nullity only (Kevin’s antennae just went
>> up.)
>>
>> So another possible restacking is to say that atomicity is something that 
>> has to
>> be *opted out of* at the declaration site (and maybe also at the use site.)
>> With deliberately-wrong syntax:
>>
>> __non-id class B2 { }
>>
>> __non-atomic __non-id class B2a { }
>>
>> __zero-ok __non-id  class B3 { }
>>
>> __non-atomic __zero-ok  __non-id class B3a { }
>>
>> In this model, you can opt out of identity, and then you can further opt out 
>> of
>> atomicity and/or null-default.  This “pulls up” the atomicity/tearaiblity to 
>> a
>> property of the class (I’d prefer safe by default, with opt out), and makes
>> zero-*capability* an opt-in property of the class.  Then for those that have
>> opted into zero-capability, at the use site, you can select .ref (null) / 
>> .val
>> (zero).  Obviously these all need better spellings.  This model frames 
>> specific
>> capabilities as modifiers on the main bucket, so it could be considered 
>> either
>> a two bucket, or a four bucket model, depending on how you look.
>>
>> The author is in the best place to make the atomicity decision, since they 
>> know
>> the integrity constraints.  Single field classes, or classes with only 

Re: User model stacking

[Remi Forax]

- Original Message -
> From: "Brian Goetz" 
> To: "valhalla-spec-experts" 
> Sent: Wednesday, April 27, 2022 6:44:01 PM
> Subject: User model stacking

> Here’s some considerations for stacking the user model.  (Again, please let’s
> resist the temptation to jump to the answer and then defend it.)
> 
> We have a stacking today which says:
> 
> - B1 is ordinary identity classes, giving rise to a single reference type
> - B2 are identity-free classes, giving rise to a single reference type
> - B3 are flattenable identity-free classes, giving rise to both a reference
> (L/ref) and primitive (Q/val) type.
> 
> This stacking has some pleasant aspects.  B2 differs from B1 by “only one 
> bit”:
> identity.  The constraints on B2 are those that come from the lack of identity
> (mutability, extensibility, locking, etc.)  B2 references behave like the
> object references we are familiar with; nullability, final field guarantees,
> etc.  B3 further makes reference-ness optional; reference-free B3 values give
> up the affordances of references: they are zero-default and tearable.  This
> stacking is nice because it can framed as a sequence of “give up some X, get
> some Y”.
> 
> People keep asking “do we need B2, or could we get away with B1/B3”.  The main
> reason for having this distinction is that some id-free classes have no
> sensible default, and so want to use null as their default.  This is a
> declaration-site property; B3 means that the zero value is reasonable, and use
> sites can opt into / out of  zero-default / nullity.  We’d love to compress
> away this bucket but forcing a zero on classes that can’t give it a reasonable
> interpretation is problematic.  But perhaps we can reduce the visibility of
> this in the model.
> 
> The degrees of freedom we could conceivably offer are
> 
>{ identity or not, zero-capable or not, atomic or not } x { use-site,
>declaration-site }
> 
> In actuality, not all of these boxes make sense (disavowing the identity of an
> ArrayList at the use site), and some have been disallowed by the stacking 
> (some
> characteristics have been lumped.)  Here’s another way to stack the
> declaration:
> 
> - Some classes can disavow identity
> - Identity-free classes can further opt into zero-default (currently, B3,
> polarity chosen at use site)
> - Identity-free classes can further opt into tearability (currently, B3,
> polarity chosen at use site)
> 
> It might seem the sensible move here is to further split B3 into B3a and B3b
> (where all B3 support zero default, and a/b differ with regard to whether
> immediate values are tearable).  But that may not be the ideal stacking,
> because we want good flattening for B2 (and B3.ref) also.  Ideally, the
> difference between B2 and B3.val is nullity only (Kevin’s antennae just went
> up.)
> 
> So another possible restacking is to say that atomicity is something that has 
> to
> be *opted out of* at the declaration site (and maybe also at the use site.)
> With deliberately-wrong syntax:
> 
>__non-id class B2 { }
> 
>__non-atomic __non-id class B2a { }
> 
>__zero-ok __non-id  class B3 { }
> 
>__non-atomic __zero-ok  __non-id class B3a { }
> 
> In this model, you can opt out of identity, and then you can further opt out 
> of
> atomicity and/or null-default.  This “pulls up” the atomicity/tearaiblity to a
> property of the class (I’d prefer safe by default, with opt out), and makes
> zero-*capability* an opt-in property of the class.  Then for those that have
> opted into zero-capability, at the use site, you can select .ref (null) / .val
> (zero).  Obviously these all need better spellings.  This model frames 
> specific
> capabilities as modifiers on the main bucket, so it could be considered either
> a two bucket, or a four bucket model, depending on how you look.
> 
> The author is in the best place to make the atomicity decision, since they 
> know
> the integrity constraints.  Single field classes, or classes with only single
> field invariants (denominator != 0), do not need atomicity.  Classes with
> multi-field invariants do.
> 
> This differs from the previous stacking in that it moves the spotlight from
> _references_ and their properties, to the properties themselves.  It says to
> class writers: you should declare the ways in which you are willing to trade
> safety for performance; you can opt out of the requirement for references and
> nulls (saving some footprint) and atomicity (faster access).  It says to class
> *users*, you can pick the combination of characteristics, allowed by the
> author, that meet your needs (can always choose null default if you want, just
> use a ref.)
> 
> There are many choices here about “what are the defaults”.  More opting in at
> the declaration site might mean less need to opt in at the use site.  Or not.
> 
> (We are now in the stage which I call “shake the box”; we’ve named all the
> moving parts, and now we’re looking for the lowest-energy state we can get 
> 

Re: On tearing

[Remi Forax]

> From: "Brian Goetz" 
> To: "valhalla-spec-experts" 
> Sent: Wednesday, April 27, 2022 3:59:31 PM
> Subject: On tearing

> Several people have asked why I am so paranoid about tearing. This mail is 
> about
> tearing; there’ll be another about user model stacking and performance models.
> (Please, let’s try to resist the temptation to jump to “the answer”.)

> Many people are tempted to say “let it tear.” The argument for “let it tear” 
> is
> a natural-sounding one; after all, tearing only happens when someone else has
> made a mistake (data race). It is super-tempting to say “Well, they made a
> mistake, they get the consequences”.

> While there are conditions under which this would be a reasonable argument, I
> don’t think those conditions quite hold here, because from both the inside and
> the outside, B3 classes “code like a class.” Authors will feel free to use
> constructors to enforce invariants, and if the use site just looks like
> “Point”, clients will not be wanting to keep track of “is this one of those
> classes with, or without, integrity?” Add to this, tearing is already weird,
> and while it is currently allowed for longs and doubles, 99.% of Java
> developers have never actually seen it or had to think about it very 
> carefully,
> because implementations have had atomic loads and stores for decades.

> As our poster child, let’s take integer range:

> __B3 record IntRange(long low, int high) {

> public IntRange {
> if (low > high) throw;
> }
> }

> Here, the author has introduced an invariant which is enforced by the
> constructor. Clients would be surprised to find an IntRange in the wild that
> disobeys the invariant. Ranges have a reasonable zero value. This a an obvious
> candidate for B3.

> But, I can make this tear. Imagine a mutable field:

> /* mutable */ IntRange r;

> and two threads racing to write to r. One writes IntRange(5, 10); the other
> writes IntRange(2,4). If the writes are broken up into two writes, then a
> client could read IntRange(5, 4). Worse, unlike more traditional races which
> might be eventually consistent, this torn value will be observable forever.

> Why does this seem worse than a routine long tearing (which no one ever sees 
> and
> most users have never heard of)? Because by reading the code, it surely seems
> like the code is telling me that IntRange(5, 4) is impossible, and having one
> show up would be astonishing. Worse, a malicious user can create such a bad
> value (statistically) at will, and then inject that bad value into code that
> depends on the invariants holding.

> Not all values are at risk of such astonishment, though. Consider a class 
> like:

> __B3 record LongHolder(long x) { }

> Given that a LongHolder can contain any long value, users of LongHolder are 
> not
> expecting that the range is carefully controlled. There are no invariants for
> which breaking them would be astonishing; LongHolder(0x1234567887654321) is
> just as valid a value as LongHolder(3).

> There are two factors here: invariants and transparency. The above examples
> paint the ranges of invariants (from none at all, to invariants that constrain
> multiple fields). But there’s also transparency. The second example was
> unsurprising because the API allowed us to pass any long in, so we were not
> surprised to see big values coming out. But if the relationship between the
> representation and the construction API is more complicated, one could imagine
> thinking the constructor has deterred all the surprising values, and then 
> still
> see a surprising value. That longs might tear is less surprising because any
> combination of bits is a valid long, and there’s no way to exclude certain
> values when “constructing” a long.

> Separately, there are different considerations at the declaration and use 
> site.
> A user can always avoid tearing by avoiding data races, such as marking the
> field volatile (that’s the usual cure for longs and doubles.) But what we’re
> missing is something at the declaration site, where the author can say “I have
> integrity concerns” and constrain layout/access accordingly. We experimented
> with something earlier (“extends NonTearable”) in this area.

> Coming back to “why do we care so much”. PLT_Hulk summarized JCiP in one
> sentence:

> [ https://twitter.com/PLT_Hulk/status/509302821091831809 |
> https://twitter.com/PLT_Hulk/status/509302821091831809 ]

> If Java developers have learned one thing about concurrency, it is: “immutable
> objects are always thread-safe.” While we can equivocate about whether B3.val
> are objects or not, this distinction is more subtle than we can expect people
> to internalize. (If people internalized “Write immutable classes, they will
> always be thread-safe”, that would be pretty much the same thing.) We cannot
> deprive them of the most powerful and useful guideline for writing safe code.

> (To make another analogy: serialization lets objects appear to not obey
> invariants established 

Re: We need help to migrate from bucket 1 to 2; and, the == problem

[Remi Forax]

> From: "Kevin Bourrillion" 
> To: "valhalla-spec-experts" 
> Sent: Tuesday, April 26, 2022 5:12:54 AM
> Subject: We need help to migrate from bucket 1 to 2; and, the == problem

> So I want to make my class identityless. But -- whoops! -- I released it years
> ago and it has lots of usages. And though I've labeled it as "value-based", 
> any
> number of these callers are depending on its identity in some way or other.

> I'd like to put -- let's say an annotation on my class, like
> `@_FutureNonIdentityClass` or whatever, with the following effects:

> * I get a warning if I'm breaking any of the rules of identityless classes, 
> like
> if I have a non-final field.
> * Use sites get a warning if they do _anything_ identity-dependent with it 
> (==,
> identity hc, synchronization, ...?)

> This would leave me in a good position to add the real identity-forsaking
> keyword later (at which time the annotation becomes redundant and should cause
> a warning until it's removed).

> We can address all this in Error Prone, but I'm not sure it should be left to
> that, partly because a bunch of JDK value-based types need this same treatment
> themselves (apparently only the synchronization warning has been rolled out so
> far?).

> Could we get this supported in javac itself? The best thing would be to roll 
> it
> out in an even earlier release than bucket 2 types themselves... the sooner 
> the
> better (maybe we could help?).

> I think the annotation could be relegated to some one-off module so it doesn't
> pollute the beautiful jdk.base forever.

> ~~~

> One of the things this means is that people should stop using `==` on these
> value-based classes.

> And that is really really good, because what we are planning to do to `==` 
> is...
> really really bad. Don't misread me: if compatibility is sacrosanct then it is
> probably the least-bad thing we can do! But honestly, it's bad, because it's
> not a behavior that anyone ever *actually wants* -- unless they just happen to
> have no fields of reference types at all. But the fact that it does work in
> that case just makes the whole thing worse, because code like that will be a
> ticking time bomb waiting to do the wrong thing as soon as one reference-type
> field is added at any nested level below that point.

> What if we give users support for their migration path, so there *are no* 
> usages
> of `==` that need to remain compatible for these types? Then we could make 
> `==`
> *not do anything* at all for bucket-2 classes.

> This approach could save us from a lot of pain (longstanding pain and new 
> pain)
> for int and Integer and friends too.

> I think Java's historical priority of "compatibility at all costs" has been
> something of an illusion; it still leaves us high and dry when *we* want to
> adopt new features as *we* end up having to make incompatible changes to do 
> it.
> But if we always gave proper support to users' migration scenarios then we
> wouldn't always *need* the absolute compatibility at the language level.

>From my experience, there are == that are buried deeply masked as innocuous == 
>on Object. 
A simple example is assertSame() of JUnit, it is defined on Object but will 
call == on B2 if the arguments are B2. 

Any non-interprocedural analysis will miss them. Perhaps an agent that rewrite 
all acmp to use invokedynamic will be able to trap those runtime call to == and 
emit a warning with the corresponding stacktrace. 

Rémi 

B3 ref model

[Remi Forax]

Ok, maybe i've not understood correctly how B3 model works,
for me being a B3 is a runtime property, not a type property.

By example, if there is an Object but the VM knows the only possible type is a 
B3 and the value is not null then the VM is free to emit several stores, 
because it's a B3, so tearing can occur.

Said differently, B3 allows tearing, so B3.val and B3.ref allow tearing.

If i do not want tearing, then B3 has to be stored in a field volatile or i 
have to declare the class as a B2.

Did i get it right ?

Rémi

Re: [External] Foo / Foo.ref is a backward default; should be Foo.val / Foo

[Remi Forax]

> From: "Kevin Bourrillion" 
> To: "valhalla-spec-experts" 
> Sent: Monday, April 25, 2022 4:52:50 AM
> Subject: [External] Foo / Foo.ref is a backward default; should be Foo.val / 
> Foo

> Hi,

> The current plan for `primitive class Foo` -- to call the value type `Foo` and
> the reference type `Foo.ref` -- is causing a few problems that I think are
> unnecessary. I've felt for a while now that we are favoring the wrong default.
> We should let `Foo` be the reference type and require `Foo.val` (precise 
> syntax
> aside) for the value type.
> I started to list reasons and came up with more than expected.

If ref is the default for B3 then B3 is a worst B2, it's like saying let's 
transform all long to Long. 

> 1. The option with fewer hazards should usually be the default. Users won't 
> opt
> themselves into extra safety, but they will sometimes opt out of it. Here, the
> value type is the one that has attendant risks -- risk of a bad default value,
> risk of a bad torn value. We want using `Foo.val` to *feel like* cracking open
> the shell of a `Foo` object and using its innards directly. But if it's 
> spelled
> as plain `Foo` it won't "feel like" anything at all.

Users should use B2 by default, you did agree about that. 
if users want B3 we should give them B3, asking for B3.val is a kind of double 
opt-in. 

> 2. In the current plan a `Foo.ref` should be a well-behaved bucket 2 object. 
> But
> it sure looks like that `.ref` is specifically telling it NOT to be -- like
> it's saying "no, VM, *don't* optimize this to be a value even if you can!"
> That's of course not what we mean. With the change I'm proposing, `Foo.val`
> does make sense: it's just saying "hey runtime, while you already *might* have
> represented this as a value, now I'm demanding that you *definitely* do".
> That's a normal kind of a thing to do.

.ref should be rare in the end, it's mostly a stopgap measure because we do not 
have universal generics. 
Once we have universal generics, Foo.val make even less sense. 

> 3. This change would permit compatible migration of an id-less to primitive
> class. It's a no-op, and use sites are free to migrate to the value type if 
> and
> when ready. And if they already expose the type in their API, they are free to
> weigh the costs/benefits of foisting an incompatible change onto *their* 
> users.
> They have facilities like method deprecation to do it with. In the current
> plan, this all seems impossible; you would have to fix all your problematic
> call sites *atomically* with migrating the class.

B3 requires that the default value that makes sense and that bypassing the 
constructor is fine (because you can construct any values by "merging" existing 
values). 
Maybe we should disallow users to even write constructors to avoid to get them 
false hope. 

Anyway, those constraints mean that you will not be able to refactor most of 
the existing classes to a primitive classes because you are loosing 
encapsulation by doing that. 

> 4. It's much (much) easier on the mental model because *every (id-less) class
> works in the exact same way*. Some just *also* give you something extra, 
> that's
> all. This pulls no rugs out from under anyone, which is very very good.

No , B2 and B3 are different runtime models, even if B3 is ref by default. 
The idea of B3 being ref by default is infact dangerous exactly for the reason 
you explain, it looks like the two models are the same. 
The problem is that they are not. 

> 5. The two kinds of types have always been easily distinguishable to date. The
> current plan would change that. But they have important differences
> (nullability vs. the default value chief among them) just as Long and long do,
> and users will need to distinguish them. For example you can spot the 
> redundant
> check easily in `Foo.val foo = ...; / requireNonNull(foo);`.

You want a use site way to see if a type is a B3 as opposed to B1 and B2 that 
are both nullable. 
It's something that can be discussed separately. 

> 6. It's very nice when the *new syntax* corresponds directly to the *new 
> thing*.
> That is, until a casual developer *sees* `.val` for the first time, they won't
> have to worry about it.

But it's not true, compare 
Complex.val c = new Complex(1, 2); 
and 
Complex c = new Complex(1, 2); 

> 7. John seemed to like my last fruit analogy, so consider these two equivalent
> fruit stand signs:

> a) "for $1, get one apple OR one orange . . . with every orange purchased you
> must also take a free apple"
> b) "apples $1 . . . optional free orange with each purchase"

> Enough said I think :-)

> 8. The predefined primitives would need less magic. `int` simply acts like a
> type alias for `Integer.val`, simple as that. This actually shows that the
> whole feature will be easier to learn because it works very nearly how people
> already know primitives to work. Contrast with: we hack it so that what would
> normally be called `Integer` gets called `int` and 

Re: [External] Foo / Foo.ref is a backward default; should be Foo.val / Foo

[Remi Forax]

> From: "Brian Goetz" 
> To: "Dan Heidinga" 
> Cc: "Kevin Bourrillion" , "valhalla-spec-experts"
> 
> Sent: Monday, April 25, 2022 8:26:02 PM
> Subject: Re: [External] Foo / Foo.ref is a backward default; should be 
> Foo.val /
> Foo

>>> What I’m thinking here about migration is that existing APIs can say 
>>> “Optional”
>>> but field declarations can say Optional.val, getting additional footprint /
>>> flattening benefits, without perturbing the APIs (and with cheap 
>>> conversions.)

>> Aren't most of the migration cases (at least for existing VBC)
>> targeting B2? They need to keep the reference semantics due to
>> existing code using null and will still get optimized inside a
>> compiled body.

> Sort of. For existing uses, we’re stuck with compatibility with the L 
> protocol,
> certainly. But consider this example:

> class Foo {
> private Optional f;

> public Optional f() { return f; }
> }

> The API point has to stay LOptional, but the field could migrate further to
> QOptional, and there’s definitely value in that. With the current stake in the
> ground, we have no way to get there, but with Kevin’s proposal, we have the
> option to go further.

This seems very specific to Optional, for Optional storing null is always a 
mistake, but that's not true for other VBC, by example a deadline can be typed 
as an Instant with null meaning no deadline. 

Rémi 

128bits value type and VarHande.compareAndSet() Was: Objects vs. values, the continuation

[Remi Forax]

> From: "Brian Goetz" 
> To: "Dan Heidinga" 
> Cc: "Kevin Bourrillion" , "valhalla-spec-experts"
> 
> Sent: Monday, April 25, 2022 4:59:14 PM
> Subject: Re: Objects vs. values, the continuation

>>> The fact that these are "small" (at most 64 bits) is incidental, not 
>>> essential;
>>> introducing a new quadruple type would not destabilize our concept of a
>>> primitive value.

>> If we can tip the user's mental model so that they believe "small is
>> good" for B3 values, then we aid them in hitting the sweet space of
>> the design and help them avoid tearing issues. It doesn't change the
>> model but the more we can encourage the belief that B3 values should
>> be <= 64it the happier users will be with the results.

> I think its reasonable to say that “we can flatten 64 bits better than we can
> flatten 256, but go ahead and write the code you want, and we’ll do what we
> can.” Recent data suggests that we can get to 128 more quickly than we had
> initially expected, and (especially if we can drop the null footprint tax, as
> B3 does), you can do a lot in 128. Presumably in some future hardware
> generation this number will go up again, whether that is 5 or 10 years from
> now, we don’t know right now.

This is tangential but i write it here because i will forget again. 
There is an issue with representing B2 as a 128 bits value, while Intel and ARM 
both provides 128 atomic read/write if vectorized registers are used, they do 
not provide a CAS (or the equivament on ARM) on 128 bits. 

This is an issue for VarHandle because 
- one can create a VarHandle on any field (array cell), it does not have to be 
volatile (there is no way to declare the content of an array volatile) 
- VarHandle.compareAndSet() has to work. 

If we keep the exact same semantics for VarHandle, we can not use 128 bits for 
fields declared as B2 because a VarHandle may be constructed on it later. 

Rémi 

Re: Objects vs. values, the continuation

[Remi Forax]

> From: "Remi Forax" 
> To: "Kevin Bourrillion" 
> Cc: "valhalla-spec-experts" 
> Sent: Monday, April 25, 2022 12:23:32 AM
> Subject: Re: Objects vs. values, the continuation

>> From: "Kevin Bourrillion" 
>> To: "valhalla-spec-experts" 
>> Sent: Saturday, April 23, 2022 12:38:12 AM
>> Subject: Objects vs. values, the continuation

>> I'd like to remind everyone about this (self-important-sounding) document I
>> shared some months ago: [
>> https://docs.google.com/document/d/1J-a_K87P-R3TscD4uW2Qsbt5BlBR_7uX_BekwJ5BLSE/preview
>> | Data in Java programs: a basic conceptual model ]
>> I may have undersold it a bit last time. True, it's not the final word on the
>> only possible conceptual model anyone could ever form; however, it is at 
>> least
>> a very extensively thought-out and reviewed and self-consistent one. I've 
>> also
>> revised it a lot since you saw it (and it's still open for review). If 
>> nothing
>> else, at least when I make arguments on this list you don't have to wonder 
>> what
>> they are based on; I've laid it all out in black and white. And on that
>> subject...

>> The crux of that doc for Valhalla purposes is its clear separation between
>> objects and values as wholly disjoint concepts.

>> An object : has its own independent existence; is self-describing, thus can 
>> be
>> polymorphic; is always indirected / accessed via reference; is eligible to 
>> have
>> identity.

>> A value : has no independent existence; is container-described, thus is 
>> strictly
>> monomorphic; is always used directly / inline; cannot have identity. (Yes, 
>> I'm
>> glossing over that references are also values, here.)

>> What unifies objects and values (setting aside references) is that they are 
>> all
>> instances .

>> (First, to parrot Brian of a while ago: for a primitive type, the values are 
>> the
>> instances of the type; for a reference type, the values are references to the
>> instances of the type, those instances being objects.)

>> Some instances are of a type that came from a class, so they get to have 
>> members
>> . Some instances of are of a type that never used to have a class, but will 
>> now
>> (int, etc.) -- yay. And some are of array types, which act like halfway-fake
>> classes with a few halfway-fake members. Members for everybody, more or less!

>> Though we have at times said "the goal of Valhalla is to make everything an
>> object", I claim the unification we really want is for everything to be a 
>> class
>> instance. I think that gives us enough common ground to focus on when we 
>> don't
>> otherwise know which thing the thing is (e.g. with a type variable).

> As we discussed earlier, there are two approaches, one is to say that
> instance of class = object | value
> the other is to say that
> instance of class = object = reference object | immediate object

> I prefer the later to the former, because it does not goes against what people
> already think, said differently we add more vocabulary instead of trying to
> refine the existing vocabulary.

> I've done several talks on Valhalla,

(oops, part of that sentence went missing) 
I've done several talks on Valhalla, talking about value object or immediate 
object resonate more with the attendees than trying to redefine what an 
instance of a class is. 

Rémi 

Re: Objects vs. values, the continuation

[Remi Forax]

> From: "Brian Goetz" 
> To: "Kevin Bourrillion" 
> Cc: "valhalla-spec-experts" 
> Sent: Sunday, April 24, 2022 5:57:57 PM
> Subject: Re: Objects vs. values, the continuation

> Overall I find a lot to like about this presentation. I’m still a little iffy
> about whether we can redefine the letters o-b-j-e-c-t in this way, but that is
> largely a “syntax” reaction to your statements; the substance of the 
> statements
> sounds about right.

> I especially like this bit:

>> The way I think bucket 2 can and should be explained is: "in the programming
>> model it absolutely is an object. In the performance model , the VM can 
>> replace
>> it undetectably with a (compound) value. But that is behind the scenes; it's
>> still behaviorally an object and don't feel bad about calling it an object." 
>> To
>> me this is very simple and sensible.

> I think what is missing from our presentation — and likely key to succeeding —
> is how to describe “compound value” in a way that feels like a thing. Users
> mostly understand how primitives are different from objects, but “loose bag of
> primitives with limited integrity constraints” is a new and complex concept
> that I worry users will have a hard time keeping separate from their 
> conception
> of object. Once we start aggregating fields, the temptation is to say “that’s
> like an object” and then carry with it some incorrect assumptions of integrity
> (e.g., final field guarantees)

Having loose integrity is a property of primitive class but it's also a 
property of double and long. 
This is not how we describe long and double, at least not until we introduce 
the notion of concurrency. 

I think that having a default value / not being null is a property that is 
easier to understand and easier to grasp than the concept of integrity. 
Or maybe i'm not understanding what integrity really means. 

Rémi 

>> On Apr 22, 2022, at 6:38 PM, Kevin Bourrillion < [ mailto:kev...@google.com |
>> kev...@google.com ] > wrote:

>> I'd like to remind everyone about this (self-important-sounding) document I
>> shared some months ago: [
>> https://docs.google.com/document/d/1J-a_K87P-R3TscD4uW2Qsbt5BlBR_7uX_BekwJ5BLSE/preview
>> | Data in Java programs: a basic
>>  conceptual model ]

>> I may have undersold it a bit last time. True, it's not the final word on the
>> only possible conceptual model anyone could ever form; however, it is at 
>> least
>> a very extensively thought-out and reviewed and self-consistent one. I've 
>> also
>> revised it a lot since you saw it (and it's still open for review). If 
>> nothing
>> else, at least when I make arguments on this list you don't have to wonder 
>> what
>> they are based on; I've laid it all out in black and white. And on that
>> subject...

>> The crux of that doc for Valhalla purposes is its clear separation between
>> objects and values as wholly disjoint concepts.

>> An object : has its own independent existence; is self-describing, thus can 
>> be
>> polymorphic; is always indirected / accessed via reference; is eligible to 
>> have
>> identity.

>> A value : has no independent existence; is container-described, thus is 
>> strictly
>> monomorphic; is always used directly / inline; cannot have identity. (Yes, 
>> I'm
>> glossing over that references are also values, here.)

>> What unifies objects and values (setting aside references) is that they are 
>> all
>> instances .

>> (First, to parrot Brian of a while ago: for a primitive type, the values are 
>> the
>> instances of the type; for a reference type, the values are references to the
>> instances of the type, those instances being objects.)

>> Some instances are of a type that came from a class, so they get to have 
>> members
>> . Some instances of are of a type that never used to have a class, but will 
>> now
>> (int, etc.) -- yay. And some are of array types, which act like halfway-fake
>> classes with a few halfway-fake members. Members for everybody, more or less!

>> Though we have at times said "the goal of Valhalla is to make everything an
>> object", I claim the unification we really want is for everything to be a 
>> class
>> instance. I think that gives us enough common ground to focus on when we 
>> don't
>> otherwise know which thing the thing is (e.g. with a type variable).

>> One thing I like very much about this is that it fits perfectly with the fact
>> that Integer is a subtype of Object and int is not.

>> The way I think bucket 2 can and should be explained is: "in the programming
>> model it absolutely is an object. In the performance model , the VM can 
>> replace
>> it undetectably with a (compound) value. But that is behind the scenes; it's
>> still behaviorally an object and don't feel bad about calling it an object." 
>> To
>> me this is very simple and sensible.

>> If we instead want to say "the int value 5 is an object now too", then we 
>> have
>> some problems:

>> * I think it ruins those clean explanations just given
>> * we'd need 

Re: Objects vs. values, the continuation

[Remi Forax]

> From: "Kevin Bourrillion" 
> To: "valhalla-spec-experts" 
> Sent: Saturday, April 23, 2022 12:38:12 AM
> Subject: Objects vs. values, the continuation

> I'd like to remind everyone about this (self-important-sounding) document I
> shared some months ago: [
> https://docs.google.com/document/d/1J-a_K87P-R3TscD4uW2Qsbt5BlBR_7uX_BekwJ5BLSE/preview
> | Data in Java programs: a basic conceptual model ]
> I may have undersold it a bit last time. True, it's not the final word on the
> only possible conceptual model anyone could ever form; however, it is at least
> a very extensively thought-out and reviewed and self-consistent one. I've also
> revised it a lot since you saw it (and it's still open for review). If nothing
> else, at least when I make arguments on this list you don't have to wonder 
> what
> they are based on; I've laid it all out in black and white. And on that
> subject...

> The crux of that doc for Valhalla purposes is its clear separation between
> objects and values as wholly disjoint concepts.

> An object : has its own independent existence; is self-describing, thus can be
> polymorphic; is always indirected / accessed via reference; is eligible to 
> have
> identity.

> A value : has no independent existence; is container-described, thus is 
> strictly
> monomorphic; is always used directly / inline; cannot have identity. (Yes, I'm
> glossing over that references are also values, here.)

> What unifies objects and values (setting aside references) is that they are 
> all
> instances .

> (First, to parrot Brian of a while ago: for a primitive type, the values are 
> the
> instances of the type; for a reference type, the values are references to the
> instances of the type, those instances being objects.)

> Some instances are of a type that came from a class, so they get to have 
> members
> . Some instances of are of a type that never used to have a class, but will 
> now
> (int, etc.) -- yay. And some are of array types, which act like halfway-fake
> classes with a few halfway-fake members. Members for everybody, more or less!

> Though we have at times said "the goal of Valhalla is to make everything an
> object", I claim the unification we really want is for everything to be a 
> class
> instance. I think that gives us enough common ground to focus on when we don't
> otherwise know which thing the thing is (e.g. with a type variable).

As we discussed earlier, there are two approaches, one is to say that 
instance of class = object | value 
the other is to say that 
instance of class = object = reference object | immediate object 

I prefer the later to the former, because it does not goes against what people 
already think, said differently we add more vocabulary instead of trying to 
refine the existing vocabulary. 

I've done several talks on Valhalla, 

> One thing I like very much about this is that it fits perfectly with the fact
> that Integer is a subtype of Object and int is not.

I've not a clear idea how primitive will be retrofit to be "immediate class" 
but at some point, we will want ArrayList to be valid with E still bounded 
by Object, so the sentence "int is not a subtype of java.lang.Object" may be 
wrong depending how we retrofit existing primitive types. 

> The way I think bucket 2 can and should be explained is: "in the programming
> model it absolutely is an object. In the performance model , the VM can 
> replace
> it undetectably with a (compound) value. But that is behind the scenes; it's
> still behaviorally an object and don't feel bad about calling it an object." 
> To
> me this is very simple and sensible.

B2 are nullable immediate objects. 

> If we instead want to say "the int value 5 is an object now too", then we have
> some problems:

> * I think it ruins those clean explanations just given
> * we'd need to coin some new term to mean exactly what I've just proposed that
> "object" mean, and I have no idea what that would be (do you?)

> What are the latest thoughts on this?

I like "immediate" to characterize objects without a stable address in memory. 

Rémi 

Re: Object as a concrete class

[Remi Forax]

- Original Message -
> From: "Dan Heidinga" 
> To: "Brian Goetz" 
> Cc: "daniel smith" , "valhalla-spec-experts" 
> 
> Sent: Friday, April 1, 2022 3:50:20 PM
> Subject: Re: Object as a concrete class

> On Fri, Apr 1, 2022 at 9:29 AM Brian Goetz  wrote:
>>
>>
>> > assert new Object().hasIdentity();
>> > assert !new Point().hasIdentity();
>> >
>> > But the 'hasIdentity' method can contain arbitrary logic, and doesn't
>> > necessarily need to correlate with 'getClass().isIdentityClass()'.
>>
>> More precisely, we were being held hostage to the nature of interfaces;
>> by using `implements IdentityObject` as our measure of id-ness, we
>> forced legacy Object instances to be instances of some other class, so
>> that Object didn't need to implement the interface.
>>
>> > I don't see a useful way to generalize this to other "both kinds" classes 
>> > (for
>> > example, any class with an instance field must be an identity class or a 
>> > value
>> > class). But since we have to make special allowances for Object one way or
>> > another, it does seem plausible that we let 'new Object()' continue to 
>> > create
>> > direct instances of class Object, and then specify the following special 
>> > rules:
>> >
>> > - All concrete, non-value classes are implicitly identity classes *except 
>> > for
>> > Object*
>> >
>> > - The 'new' bytecode is allowed to be used with concrete identity classes 
>> > *and
>> > class Object*
>> >
>> > - Identity objects include instances of identity classes, arrays, *and 
>> > instances
>> > of Object*; 'hasIdentity' reflects this
>> >
>> > - [anything else?]
>> >
>> > There's some extra complexity here, but balanced against the cost of making
>> > every Java programmer adjust their model of what 'new Object()' means, and
>> > corresponding coding style refactorings, it seems like a win.
>> >
>> > Thoughts?
>>
>> Seems like a win.
>>
> 
> The alternative - which we've been exploring till now - is to have the
> VM do some magic to turn:
> 
> 0: new  #2  // class java/lang/Object
> 3: dup
> 4: invokespecial #1// Method java/lang/Object."":()V
> 
> into the `new` of a VM-specified Object subclass, and to allow
> invoking a super-class constructor on it rather than its own
> (remember, the instance is an Object subclass).
> 
> That's a lot of magic we can do away with through specification and
> some careful API design.  Seems like a win.

It is too magic for me, especially because
  new Object().getClass() != Object.class

> 
> The careful API design will be key as I can see a lot of corner cases
> related to `obj.getClass().isIdentityClass() != obj.hasIdentity()`.
> Do we have a sketch of what the apis for this would look like?  I'm
> assuming these are just for expository purposes as isIdentityClass()
> really needs to return a trinary value - {true, false, maybe}.

Whatever we choose, the VM doing a rewriting trick (**) or not,
the API is the same anyway and as you said it has to be carefully written.

We have several ways to frame the discrepancy around Object.class, one is to 
say that if Object acts as a class, it's an identity class,
if Object acts as a type (type of a parameter, a field, etc.) then it accepts 
both identity and value class.

I believe that most of the question about the Class as a type are better 
answered by using isAssignableFrom.
The case where someone needs to know if a type allows only identity class or 
only value class seems a weird question compared to, can i assign that value ?

So i think we may be fine by only implementing very few methods
 - Class.isValueClass() and
 - Class.isPrimitiveClass().

All other questions can be implemented in user code, by example
  type == Object.class || type.isInterface() || !type.isValueClass()


> 
> --Dan

Rémi

** and un-rewriting when the bytecode is asked through JVMTI ?

Re: Alternative to IdentityObject & ValueObject interfaces

[Remi Forax]

> From: "Brian Goetz" 
> To: "daniel smith" , "valhalla-spec-experts"
> 
> Sent: Thursday, March 24, 2022 1:46:44 PM
> Subject: Re: Alternative to IdentityObject & ValueObject interfaces

> On 3/23/2022 10:51 PM, Dan Smith wrote:

>>> On Mar 22, 2022, at 5:56 PM, Dan Smith < [ mailto:daniel.sm...@oracle.com |
>>> daniel.sm...@oracle.com ] > wrote:

>>> - Variable types: I don't see a good way to get the equivalent of an
>>> 'IdentityObject' type. It would involve tracking the 'identity' property
>>> through the whole type system, which seems like a huge burden for the
>>> occasional "I'm not sure you can lock on that" error message. So we'd 
>>> probably
>>> need to be okay letting that go. Fortunately, I'm not sure it's a great
>>> loss—lots of code today seems happy using 'Object' when it means, 
>>> informally,
>>> "object that I've created for the sole purpose of locking".

>>> - Type variable bounds: this one seems more achievable, by using the 
>>> 'value' and
>>> 'identity' keywords to indicate a new kind of bounds check ('>> extends Runnable>'). Again, it's added complexity, but it's more localized. 
>>> We
>>> should think more about the use cases, and decide if it passes the 
>>> cost/benefit
>>> analysis. If not, nothing else depends on this, so it could be dropped. (Or
>>> left to a future, more general feature?)

>> Per today's discussion, this part seems to be the central question: how much
>> value can we expect to get out of compile-time checking?

> This is indeed the question. There's both a "theory" and a "practice" aspect,
> too.

>> The type system is going to have three kinds of types:
>> - types that guarantee identity objects
>> - types that guarantee value objects
>> - types that include both kinds of objects

>> That third kind are a problem: we can specify checks with false positives
>> (programmer knows the operation is safe, compiler complains anyway) or false
>> negatives (operation isn't safe, but the compiler lets it go).

> Flowing {Value,Identity}Object property is likely to require shoring up
> intersection types too, since we can express Runnable as a type
> bound, but not as a denotable type. Var helps a little here but ultimately 
> this
> is a hole through which information will drain.

> The arguments you make here are compelling to me, that while it might work in
> theory, in practice there are too many holes:

> - Legacy code that already deals in Object / interfaces and is not going to
> change
> - Even in new code, I suspect people will continue to do so, because as you 
> say,
> it is tedious for marginal value
> - The lack of intersection types will make it worse
> - Because of the above, many of the errors would be more like warnings, making
> it even weaker

> All of this sounds like a recipe for "new complexity that almost no one will
> actually use."

I agree, 
so if we drop the idea of having identity vs value info into the type system, 
the follow-up question is "should we restrict inheritance or not ?" 

Classes are tagged with value or not, and for an abstract class or an interface 
by default they allow both value types or identity types as subtypes. 
Do we need more, i.e. be able to restrict subtypes of an abstract 
class/interface to be value types (or identity types) only ? 

Yesterday, Dan S. talk about a user being able to restrict a hierarchy to be 
identity classes only. This will not help already existing codes but may help 
new codes by instead of having IdentityObject in the JDK, let a user define his 
own interface that play the same role as IdentityObject but tailored to his 
problem ? Or do we consider that even that use case does not worth it's own 
weight ? 

Rémi 

Re: Alternative to IdentityObject & ValueObject interfaces

[Remi Forax]

- Original Message -
> From: "Maurizio Cimadamore" 
> To: "daniel smith" , "valhalla-spec-experts" 
> 
> Sent: Wednesday, March 23, 2022 11:23:26 AM
> Subject: Re: Alternative to IdentityObject & ValueObject interfaces

> On 22/03/2022 23:56, Dan Smith wrote:
>> Other abstract classes and interfaces are fine being neither (thus supporting
>> both kinds of subclasses).
> 
> I feel that for such a proposal to be really useful (but that's true for
> the interface-based approach as well IMHO), you need a way for the _use
> site_ to attach an identity vs. value annotation to types that can
> feature both polarities (Object, interfaces, value-compatible abstract
> classes).
> 
> It's perfectly fine to have identity vs. non-identity as a declaration
> property, for the cases whether that works. E.g. an ArrayList instance
> will always have identity. An instance of a `value class Point` will
> always be identity-less. Using modifiers vs. marker interfaces here is
> mostly an isomorphic move (and I agree that adding modifiers has less
> impact w.r.t. compatibility).
> 
> But it feels like both interfaces and decl-site modifiers fall short of
> having a consistent story for the "neither" case. I feel we'd like
> programmers to be able to say things like:
> 
> ```
> class Foo {
>    identity Object lock;
> 
>    void runAction(identity Runnable action) { ... }
> }
> ```
> 
> So, I believe the modifier idea has better potential than marker
> interfaces, because it scales at the use site in ways that marker
> interfaces can't (unless we allow intersection types in declarations).
> But of course I get that adding a new use-site modifier (like `final`)
> is also not to be taken lightly; aside from grammar conundrums, as you
> say it will have to be tracked by the type system.
> 
> Stepping back, you list 4 use cases:
> 
>> - Dynamic detection
>>
>> - Subclass restriction
>>
>> - Variable types
>>
>> - Type variable bounds
> IMHO, they are not equally important. And once you give up on "variable
> types" (as explained above, I believe this use case is not adequately
> covered by any proposal I've seen), then there's a question of how much
> incremental value the other use cases add. Dynamic detection can be
> added cheaply, fine. I also think that, especially in the context of
> universal generics, we do need a way to say: "this type variable is
> legacy/identity only" - but that can also be done quite cheaply. IMHO,
> restricting subclasses doesn't buy much, if you then don't have an
> adequate way to restrict type declarations at the use sites (for those
> things that cannot be restricted at the declaration), so I'd be also
> tempted to leave that use case alone as YAGNI (by teaching developers
> that synchronizing on Object and interface types is wrong, as we've been
> already trying to do).
> 
> P.S.
> 
> While writing this, a question came up: let's say I have a generic class
> like this:
> 
> ```
> class IdentityBox { ... }
> ```
> 
> Is IdentityBox a well-formed parameterized type? Based on your
> description I'm not sure: Runnable has the "neither" polarity, but T
> expects "identity". With marker interfaces this will just not work. With
> modifiers we could perhaps allow with unchecked warning?
> 
> I think it's important that the above type remains legal: I'd expect
> users to mark their type-variables as "identity" in cases where they
> just can't migrate the class implementation to support universal type
> variables. But if that decision results in a source incompatible change
> (w.r.t. existing parameterizations such as IdentityBox), then
> it doesn't look like a great story migration-wise.

Yes !

The neither types (Object, interfaces, abstract classes) act as an eraser of 
the identity|value bit if we do not support use site identity|value modifier, 
something like IdentityBox. And given that there is already 
existing codes in the wild that does not specify "identity" or "value" we need 
a kind of unsafe conversion/unsafe warning between the new world with use site 
type annotation and the old world with no type annotation.

As Brian said to Kevin, it's a problem very similar to the introduction of a 
null type annotation, it will be painful.

> 
> Maurizio

Rémi

Re: Alternative to IdentityObject & ValueObject interfaces

[Remi Forax]

Hi Brian, 
i've maybe have twisted mind but i read your email as a rebuttal of both 
IdentityObject/ValueObject and identity/value modifiers. 

As you said, an identity object and a value object are less dis-similar now 
that they were in the past: a value class now reuse the method equals and 
hashCode of j.l.Object instead of coming with it's own definition, 
a value class is now nullable.I agree with you that synchronized is not a real 
issue so as Dan H. said, the real remaining issue is weak refs. 

Now, if there is such a small differences between an identity object and a 
value object, do we really need to introduce a mechanism to separate them in 
term of typing ? 

Rémi 

> From: "Brian Goetz" 
> To: "daniel smith" , "valhalla-spec-experts"
> 
> Sent: Wednesday, March 23, 2022 1:01:20 PM
> Subject: Re: Alternative to IdentityObject & ValueObject interfaces

> Thanks Dan for putting the work in to provide a credible alternative.

> Let me add some background for how we came up with these things. At some point
> we asked ourselves, what if we had identity and value classes from day 1? How
> would that affect the object model? And we concluded at the time that we
> probably wouldn't want the identity-indeterminacy of Object, but instead would
> want something like

> abstract class Object
> class IdentityObject extends Object { }
> abstract class ValueObject extends Object { }

> So the {Identity,Value}Object interfaces seemed valuable pedagogically, in 
> that
> they make the object hierarchy reflect the language division. At the time, we
> imagined there might be methods that apply to all value objects, that could
> live in ValueObject.

> A separate factor is that we were taking operations that were previously total
> (locking, weak refs) and making them partial. This is scary! So we wanted a 
> way
> to make these expressible in the static type system.

> Unfortunately, the interfaces do not really deliver on either goal, because we
> can't turn back time. We still have to deal with `new Object()`, so we can't
> (yet) make Object abstract. Many signatures will not be changeable from
> "Object" to "IdentityObject" for reasons of compatibility, unless we make
> IdentityObject erase to Object (which has its own problems.) If people use it
> at all for type bounds, we'll see lots of uses of `Foo Bar>`, which will put more pressure on our weak support for
> intersection types. And dynamic errors will still happen, because too much of
> the world was built using signatures that don't express identity-ness. (Kevin
> will see a parallel to introducing nullness annotations; it might be fine if
> you build the world that way from scratch, but the transition is painful when
> you have to interpret an unadorned type as "of unspecified identity-ness.")

> Several years on, we're still leaning on the same few motivating examples --
> capturing things like "I might lock this" in the type system. That we haven't
> come up with more killer examples is notable. And I grow increasingly 
> skeptical
> of the value of the locking example, both because this is not how concurrent
> code is written, and because we *still* have to deal with the risk of dynamic
> errors because most of the world's code has not been (and will not be) written
> to use IdentityObject throughout.

> As Dan points out, the main thing we give up by backing off from these
> interfaces is the static typing; we don't get to use `IdentityObject` as a
> parameter type, return type, or type bound. And the only reason we've come up
> with so far to want that is a pretty lame one -- locking.

> From a language design perspective, I find that you declare a class with 
> `value
> class`, but you express the subclassing constraint with `extends
> IdentityObject`, to be pretty leaky.

> On 3/22/2022 7:56 PM, Dan Smith wrote:

>> In response to some encouragement from Remi, John, and others, I've decided 
>> to
>> take a closer look at how we might approach the categorization of value and
>> identity classes without relying on the IdentityObject and ValueObject
>> interfaces.

>> (For background, see the thread "The interfaces IdentityObject and 
>> ValueObject
>> must die" in January.)

>> These interfaces have found a number of different uses (enumerated below), 
>> while
>> mostly leaning on the existing functionality of interfaces, so there's a 
>> pretty
>> good complexity vs. benefit trade-off. But their use has some rough edges, 
>> and
>> inserting them everywhere has a nontrivial compatibility impact. Can we do
>> better?

>> Language proposal:

>> - A "value class" is any class whose instances are all value objects. An
>> "identity class" is any class whose instances are all identity objects.
>> Abstract classes can be value classes or identity classes, or neither.
>> Interfaces can be "value interfaces" or "identity interfaces", or neither.

>> - A class/interface can be designated a value class with the 'value' 
>> modifier.

>> value class Foo 

Re: Evolving instance creation

[Remi Forax]

- Original Message -
> From: "Dan Heidinga" 
> To: "daniel smith" 
> Cc: "valhalla-spec-experts" 
> Sent: Thursday, February 24, 2022 4:39:52 PM
> Subject: Re: Evolving instance creation

> Repeating what I said in the EG meeting:
> 
> * "new" carries the mental model of allocating space.  For identity
> objects, that's on the heap.  For values, that may just be stack space
> / registers.  But it indicates that some kind of allocation / demand
> for new storage has occurred.
> 
> * It's important that "new" returns a unique instance.  That invariant
> has existed since Java's inception and we should be careful about
> breaking it.  In the case of values, two identical values can't be
> differentiated so I think we're safe to say they are unique but
> indistinguishable as no user program can differentiate them.

Yes, it's more about == being different than "new" being different.

"new" always creates a new instance but in case of value types, == does not 
allow us see if the instance are different or not.

> 
> The rest of this is more of a language design question than a VM one.
> The `Foo()` (without a new) is a good starting point for a canonical
> factory model.  The challenge will be in expressing the difference
> between the factory method and the constructor as they need to be
> distinct items in the source (different invariants, different return
> values, etc)
> 
> --Dan

Rémi

> 
> On Tue, Feb 22, 2022 at 4:17 PM Dan Smith  wrote:
>>
>> One of the longstanding properties of class instance creation expressions 
>> ('new
>> Foo()') is that the instance being produced is unique—that is, not '==' to 
>> any
>> previously-created instance.
>>
>> Value classes will disrupt this invariant, because it's possible to "create" 
>> an
>> instance of a value class that already exists:
>>
>> new Point(1, 2) == new Point(1, 2) // always true
>>
>> A related, possibly-overlapping new Java feature idea (not concretely 
>> proposed,
>> but something the language might want in the future) is the declaration of
>> canonical factory methods in a class, which intentionally *don't* promise
>> unique instances (for example, they might implement interning). These 
>> factories
>> would be like constructors in that they wouldn't have a unique method name, 
>> but
>> otherwise would behave like ad hoc static factory methods—take some 
>> arguments,
>> use them to create/locate an appropriate instance, return it.
>>
>> I want to focus here on the usage of class instance creation expressions, and
>> how to approach changes to their semantics. This involves balancing the needs
>> of programmers who depend on the unique instance invariant with those who 
>> don't
>> care and would prefer fewer knobs/less complexity.
>>
>> Here are three approaches that I could imagine pursuing:
>>
>> (1) Value classes are a special case for 'new Foo()'
>>
>> This is the plan of record: the unique instance invariant continues to hold 
>> for
>> 'new Foo()' where Foo is an identity class, but if Foo is a value class, you
>> might get an existing instance.
>>
>> In bytecode, the translation of 'new Foo()' depends on the kind of class (as
>> determined at compile time). Identity class creation continues to be
>> implemented via 'new Foo; dup; invokespecial Foo.()V'. Value class
>> creation occurs via 'invokestatic Foo.()LFoo;' (method name
>> bikeshedding tk). There is no compatibility between the two (e.g., if an
>> identity class becomes a value class).
>>
>> In a way, it shouldn't be surprising that a value class doesn't guarantee 
>> unique
>> instances, because uniqueness is closely tied to identity. So special-casing
>> 'new Foo()' isn't that different from special-casing Object.equals'—in the
>> absence of identity, we'll do something reasonable, but not quite the same.
>>
>> Factories don't enter into this story at all. If we end up having unnamed
>> factories in the future, they will be declared and invoked with a separate
>> syntax, and will be declarable both by identity classes and value classes.
>> (Value class factories don't seem particularly compelling, but they could, 
>> say,
>> be used to smooth migration, like 'Integer.valueOf'.)
>>
>> Biggest concerns: for now, it can be surprising that 'new' doesn't always 
>> give
>> you a unique instance. In a future with factories, navigating between the 
>> 'new'
>> syntax and the factory invocation syntax may be burdensome, with style wars
>> about which approach is better.
>>
>> (2) 'new Foo()' as a general-purpose creation tool
>>
>> In this approach, 'new Foo()' is the use-site syntax for *both* factory and
>> constructor invocation. Factories and constructors live in the same overload
>> resolution "namespace", and all will be considered by the use site.
>>
>> In bytecode, the preferred translation of 'new Foo()' is 'invokestatic
>> Foo.()LFoo;'. Note that this is the case for both value classes *and
>> identity classes*. For compatibility, 'new/dup/' also needs to be
>> 

Re: Abstract class with fields implementing ValueObject

[Remi Forax]

- Original Message -
> From: "John Rose" 
> To: "Frederic Parain" 
> Cc: "valhalla-spec-experts" 
> Sent: Thursday, February 10, 2022 12:02:53 AM
> Subject: Re: Abstract class with fields implementing ValueObject

> That could be one of very many edge conditions in the JVMS that are not
> diagnosed directly by a validation, but that will eventually cause an error
> when the broken classfile is further used.
> 
> I don’t think there needs to be a special rule for this.  We don’t try to
> comprehensively diagnose all “impossible-to-use” classfiles.

or better we ditch ValueObject and IdentityObject ...

Rémi

> 
> On 9 Feb 2022, at 13:50, Frederic Parain wrote:
> 
>> There's a weird case that seems to be allowed by the Value Objects JVMS 
>> draft:
>>
>> An abstract class can declare non-static fields, which means it won't
>> have the ACC_PERMITS_VALUE flag set, but also declare that it implements
>> the ValueObject interface.
>>
>> The combination looks just wrong, because no class can subclass such class:
>>   - identity classes are not allowed because of the presence  of
>> the ValueObject interface
>>   - value classes are not allowed because of the absence of
>> ACC_PERMITS_VALUE
>>
>> I've looked for a rule that would prohibit such combination in the
>> JVMS draft but couldn't find one.
>>
>> Did I miss something?
>>
> > Fred

Re: EG meeting, 2022-02-09 [SoV-3: constructor questions]

[Remi Forax]

> From: "John Rose" 
> To: "daniel smith" 
> Cc: "valhalla-spec-experts" 
> Sent: Wednesday, February 9, 2022 7:32:07 PM
> Subject: Re: EG meeting, 2022-02-09 [SoV-3: constructor questions]

> On 8 Feb 2022, at 19:04, Dan Smith wrote:

>> "SoV-3: constructor questions": Dan asked about validation for  and 
>> 
>> methods. Answer: JVM doesn't care about  methods in abstract classes, 
>> the
>> rules about  methods still uncertain.
> On the question of JVM validation of  methods, I’m in favor of as few 
> rules
> as possible, ideally treating  as just another name. It’s super-power is
> not in its restrictions but in its conventionality: It’s the obvious choice 
> for
> constructor factory methods. But it is not necessarily limited to that use.

> Maximal limitation would be that a  method can only occur as the
> translation of a value-class constructor. Any evidence in the classfile that 
> it
> was not such a translation would be grounds for failing a validation check.
> We’d make as many such rules as we can think of.

> Arguments against:

>* Having a special method identifier in the JVMs without other restrictions
> would be a new thing, and hence suspicious.
>* Limiting the use of  as much as possible makes it clear, to higher 
> layers
>of the code (javac and reflection) what is going on in the class file, as a
> “reflection” of the source file.
>* Reflection of an irregular (non-source-conforming)  method has to be
> messy. (Is it really a constructor? Or is it just a method named  ?)

> Arguments in favor:

>* It is a new thing in the JVM for any descriptor to be constrained to 
> mention
>the same name as is the name of the constant pool item referred to by
> ClassFile.this_class item (JVMS 4.1). (It is suspicious.)
>* A maximal limitation would break hidden classes. (They must sometimes 
> return a
>supertype from their factories, since the HC is not always name-able in a
> descriptor. HCs only work because the previous point.)
>* A limitation might preclude a perhaps-desirable future translation 
> strategy
>that used  factories uniformly to translate new source code 
> expressions
> (identity or value objects, uniformly).
>* A limitation could remove a natural translation strategy for “canonical
>factory methods” in non-concrete types. This is a hypothetical language 
> feature
>for Java or some other language. (E.g., new List(a,b,c) instead of
>List.of(a,b,c) , removing the need of the user to remember whether the 
> word was
> of or make or build or some other designer choice.)
>* Most any limitation would preclude ad hoc use of  factories by
>translation strategies of other languages, such as Scala and Clojure, which
>surely have their own uses of JVM object life cycles. We want to be 
> friendly to
> non-Java languages.

> Compromise positions:

>* Require a  method to be ACC_STATIC but allow for any purpose (i.e., 
> any
> access and any descriptor).
>* Require a  method to return either the class named by this_class or 
> some
> super type (TBD how this should be checked).

> I would prefer the first compromise: It’s static but otherwise the JVM asks no
> questions.

> Regarding reflection, I think it would be OK to surface all of the  
> methods
> (of whatever signature) on the getConstructors list, even if they return
> “something odd”. Alternatively, to prevent a sharp edge we could have a new
> list Class::getFactories , and copy (not move) entries from that list onto
> getConstructors exactly when the return type matches the enclosing class. That
> is a more natural move for reflection (which operates on runtime types) than
> for class file structuring (which is more static).

> The reason I prefer to require static marking is that it would prevent the 
> funny
> name from appearing on the list of regular methods, via reflection.

I agree with static being the only requirement for the VM side. 

For reflection,i prefer the first alternative, surface the  method 
whatever the return type given that the return type is a declared type, not a 
runtime class. 
Dynamically typed languages like Groovy or JRuby may want to declare that the 
return type of  is java.lang.Object (it's the default type if no type is 
provided in Groovy), 
if reflection filter out those methods, it will be bad for the interropt 
between Java and those dynamically typed languages. 

Rémi 

Re: SoV-2: weak references

[Remi Forax]

- Original Message -
> From: "Dan Heidinga" 
> To: "Brian Goetz" 
> Cc: "valhalla-spec-experts" 
> Sent: Wednesday, February 9, 2022 5:50:29 PM
> Subject: Re: SoV-2: weak references

> One option is to look at what we can do to help users prepare for IAE
> when using value-based classes as keys to WHM.  Can we take an
> approach similar to JEP 390 [1] and provide JFR events that flag uses
> of value-based classes as keys?  It's not perfect but similar to JEP
> 390, it does help users to know if they need to do something to
> prepare for this.

I wonder if there is not a difference between a weak reference of a value class 
and a weak reference on a primitive class.
A weak reference on a primitive class makes no sense, a weak reference on a 
value class may make sense because a value class is handled by a pointer.

> 
> --Dan

Rémi

> 
> [1] http://openjdk.java.net/jeps/390
> 
> On Thu, Jan 20, 2022 at 1:54 PM Dan Heidinga  wrote:
>>
>> >
>> > It certainly seems that all the choices are bad.
>> >
>> > The "obvious" choice is to simply say that WeakReference makes no 
>> > sense,
>> > in that it sidesteps the hard semantics questions.
>>
>> It's an uncomfortable answer but it seems to provide the most
>> defensible (and thus understandable) semantics for users. Values don't
>> have an explicit lifetime and thus there is no way to tell when "this"
>> copy of a value goes out of scope and can be collected. The object
>> references (if any) held by the value are not a good proxy for its
>> lifecycle - they can have either shorter or much longer life spans -
>> and will make reasoning about when a WeakReference can be
>> collected difficult for experts, let alone most users.
>>
>> > My fear is that this will cause new failures, where existing libraries 
>> > that toss
>> > objects into WHMs to cache derived results, will start to experience new
>> > failures when the value types show up in the heap (after all,
>> > WeakReference::new takes Object.)
>>
>> This may be a case where the WeakReference constructor needs to be
>> updated to take an IdentityObject and the old constructor marked as
>> @Deprecated? Which doesn't solve the immediate problem but helps
>> justify adding a "fail-fast" check to all WeakReference constructors
>> so that they throw an IllegalArgumentException if the referent isn't
>> an IdentityObject.
>>
>> This won't avoid failures but it does make it very clear what went
>> wrong rather than introducing "strange", hard to diagnose failures.
>>
>> > And we'll have to have something to tell those users, because they 
>> > declared a
>> > WeakHashMap, and someone created a value subtype of User --
>> > which seems entirely valid.
>> >
>> > It is possible we could do something special with WHM, since it is layered 
>> > atop
>> > WR, but that still begs the question -- what?
>>
>> Starting from the conclusion that WeakReference is a
>> meaningless entity, what are the options here?
>>
>> 1) Make it impossible to use a Value as a key in a WeakHashMap.
>> ::put(key, value) & ::pulAll(Map m) will throw if a key is a Value
>> object.  ::containsKey(Object) will always be false if the Object is a
>> ValueObject.  This makes WeakHashMap unusable with Values.  The
>> semantics are clear but all existing uses of WeakHashMap will need to
>> be adapted to defensively check for Values and do something (tbd) to
>> avoid the exceptions.
>>
>> 2) Use strong references for Value keys in WeakHashMap.
>> Treat each Value object used as a key in WeakHashMap as a strong
>> reference.  This means Value keys will never be removed and will keep
>> their corresponding map value alive forever (or until explicitly
>> removed).  While this will allow WeakHashMaps to continue to be used
>> as Maps for Values, it will break the contract for WHM and may
>> introduce memory leaks into otherwise correct programs.
>>
>> 3) Pick some other object to act as the reference when using a Value
>> key in a WHM.
>> This is basically the solution we rejected for WeakReference
>> and all the same problems apply here.  It may allow existing code to
>> "keep working" when it first deals with Values but introduces strange
>> failure cases and difficult to reason about rules.  It avoids
>> exceptions but leaves the code doing the wrong thing with no way to
>> tell.
>>
>> Anyone see another option here?
>>
> > --Dan

Re: SoV-3: constructor questions

[Remi Forax]

- Original Message -
> From: "Dan Heidinga" 
> To: "daniel smith" 
> Cc: "valhalla-spec-experts" 
> Sent: Thursday, January 27, 2022 4:09:58 PM
> Subject: Re: SoV-3: constructor questions

> (Resending as I forgot to CC the list - Sorry for the duplicate email Dan)
> 
>> We can come up with a rule to specify, in the Java language, what an "empty
>> constructor" looks like. (Nothing syntactically in the body, for example.)
>>
>> It's harder for the JVM to specify what an "empty  method" looks like. 
>> It
>> must at least have an 'invokespecial' of its superclass's . Might do 
>> some
>> stacking of arguments. Etc. JVMs don't want to be in the business of
>> recognizing code patterns.
> 
> Ok.  This is due to a class having a single no-args `()V` method
> that has to do double duty - supply the super-call for identity
> subclasses and be ignored by value subclasses.  Seems reasonable.
> 
>> > 2) What is the rationale behind the return type restrictions on  
>> > methods?
> 
>> Treatment of  methods is still unresolved, so this (and the JEP) is just
>> describing one possible approach. I tried to reach a conclusion on this a few
>> months ago on this list, but we ended in an unresolved place. I'll try 
>> again...
>>
>> Anyway, in this incarnation: the rule is that the return type must be a type
>> that includes instances of the current class. So, in class Point, QPoint is
>> okay, LObject is okay, but LString is not.
> 
> I don't understand the point of this restriction.  Since
> Ljava/lang/Object; is acceptable (and has to be), I can use a ``
> method to return *any* class but the caller will need to downcast to
> use it.
> 
> class QPoint {
>  static Object () { return new String("string factory"); }
>  static void user() {
>String s = (String) QPoint.:()Ljava/lang/Object;
>  }
> }
> We've made the bytecode patterns ugly by requiring a checkcast - and
> maybe slightly harder to write a large set of factory methods due to
> needing different descriptors for each one - and gained what?  Javac
> may not generate this kind of code but it still seems odd to restrict
> it.

The reason John gave for allowing a  method to return a super type is for 
lambda proxies.

A lambda proxies is a hidden value class, i.e. a value class loaded by 
lookup.defineHiddenClass(),
given that a hidden class as no real name, the idea is to use Object or perhaps 
the functional interface as return type.
If the functional interface is used, there will be no weird cast in the 
bytecode.

[...]

> 
> --Dan

Rémi

> 
> On Wed, Jan 26, 2022 at 5:15 PM Dan Smith  wrote:
>>
>> > On Jan 26, 2022, at 2:18 PM, Dan Heidinga  wrote:
>> >
>> > After re-reading the State of Valhalla part 3 again [1], I have a
>> > couple of questions on constructor handling:
>> >
>> > 1) The rules for handling ACC_PERMITS_VALUE are different between
>> > SoV-2 and SoV-3 in that the language imposes constraints the VM
>> > doesn't check.  Is this deliberate?
>> >
>> > SoV-2 says:
>> >> The classes they can extend are restricted: Object or abstract classes 
>> >> with no
>> >> fields, empty no-arg constructor bodies, no other constructors, no 
>> >> instance
>> >> initializers, no synchronized methods, and whose superclasses all meet 
>> >> this
>> >> same set of conditions. (Number is an example of such a class.)
>> >
>> > while SoV-3 says:
>> >> Perhaps surprisingly, the JVM does not perform the following checks for
>> >> value-superclass candidates, although the source language compiler will
>> >> typically do so:
>> >>
>> >> It should have declared an empty no-argument constructor. (Or if it 
>> >> didn’t, then
>> >> the author has somehow consented to having all of the constructors being
>> >> skipped by the unnamed factory methods of value subclasses.)
>> >
>> > "Perhaps surprisingly" is right as I'm surprised =)  and not sure I
>> > follow why the VM wouldn't enforce the restriction.  Is it to avoid
>> > having to specify the attributes of that constructor?
>>
>> We can come up with a rule to specify, in the Java language, what an "empty
>> constructor" looks like. (Nothing syntactically in the body, for example.)
>>
>> It's harder for the JVM to specify what an "empty  method" looks like. 
>> It
>> must at least have an 'invokespecial' of its superclass's . Might do 
>> some
>> stacking of arguments. Etc. JVMs don't want to be in the business of
>> recognizing code patterns.
>>
>> So the model for the JVM is: you can declare  methods if you want to
>> support identity subclass creation, and you can declare ACC_PERMITS_VALUE if
>> you want to support value subclass creation. Independent channels, no need 
>> for
>> them to interact.
>>
>> > Which leads me to the next concern: how javac will compile the "empty
>> > no-arg constructor bodies" required by SoV-2?  Or is the answer we
>> > don't care because the VM won't check anyway?
>>
>> The Java language will produce class files for qualifying abstract classes 
>> with:
>> - ACC_PERMITS_VALUE 

The interfaces IdentityObject and ValueObject must die !

[Remi Forax]

I think we should revisit the idea of having the interfaces 
IdentityObject/ValueObject.

They serve two purposes
 1/ documentation: explain the difference between an identity class and a value 
class
 2/ type restriction: can be used as type or bound of type parameter for 
algorithms that only works with identity class

Sadly, our design as evolved but not those interfaces, they do not work well as 
type restriction, because
the type is lost once an interface/j.l.Object is used and the cost of their 
introduction is higher than previously though.
 
1/ documentation

- Those interface split the possible types in two groups
  but the spec split the types in 3, B1/B2/B3, thus they are not aligned 
anymore with the new design.

- they will be noise in the future, for Valhalla, the separation between 
identity object and value object
  may be important but from a POV of someone learning/discovering the language 
it's a corner case
  (like primitives are). This is even more true with the introduction of B2, 
you can use B2 for a long time without
  knowing what a value type is. So having such interfaces front and center is 
too much.


2/ as type

- Being a value class or a primitive class is a runtime behavior not a compile 
time behavior,
  so representing them with special types (a type is a compile time construct) 
will always be an approximation.
  As a consequence, you can have an identity class (resp value class) typed 
with a type which is not a subtype
  of IdentityObject (resp ValueObject).
  
  This discrepancy is hard to grasp for beginners (in fact not only for 
beginners) and make IdentityObject/ValueObject
  useless because if a method takes an IdentityObject as parameter but the type 
is an interface, users will have
  to cast it into an IdentityObject which defeat the purpose of having such 
interface.
  
  (This is the reason why ObjectOutputStream.writeObject() takes an Object as 
parameter and not Serializable)
  

And the cost of introduction is high

- they are not source backward compatible
  class A {}
  class B {}
  var list = List.of(new A(), new B());
  List list2 = list;

- they are not binary backward compatible
  new Object().getClass() != Object.class

- at least IdentityObject needs to be injected at runtime, which by side effect 
requires patching several
  VM components: reflection/j.l.invoke, JVMTI, etc making the VM spec more 
complex that it should be.


You may not agree with some of my points, nevertheless, the current design is 
not aligned with the B1/B2/B3 world, have a high cost and with little benefit 
so i think it's time to remove those interfaces from the design.

regards,
Rémi

VM model and aconst_init

[Remi Forax]

I've some troubles to wrap my head around those two sentences

"""
aconst_init is the analogue of new for value objects; it leaves a reference to 
the initial value for a value class on the stack. This initial value is 
guaranteed to not be equal to null. The sole operand of this bytecode is a 
reference to a CONSTANT_Class item giving the internal binary name of the value 
class (not its Q descriptor).
"""

and
"""
Both withfield and aconst_init return a Q type if and only if their class is a 
primitive class.
"""

The second is ambiguous because it's not clear if aconst_init can return a 
L-type. I suppose it can not but this is not clear at all.

If this is the case, what is the use case for withfield taking a L-type as 
parameter ??

regards,
Rémi

L-type, Q-type and class mirrors

[Remi Forax]

Hi all,
i want to propose a way to reconcile builtin primitive types and primitive 
classes in the B1/B2/B3 world.

Let's take a detour, and try to answer to the question, how do we do reflection 
on method with Q-types ?
Given that reflection is using the class java.lang.Class, it means that we need 
a class that represents a L-type and a class that represent a Q-type.

The class that represents a Q-type does not have to be a 'real' class, the same 
way int.class (Integer.TYPE) is not a real class,
it's a mirror class that represent 'I' in the method signature, same with 
void.class (Void.TYPE) represent 'V'.

So the class that represents a Q-type is a mirror class synthesized by the VM 
at runtime.
Given that it is synthesized, i believe it should be only accessible using a 
method of java.lang.Class,
by example
  Complex.class.mirrorType()  // the name "mirror" is perhaps not the best

Once we agree about that, we can push a little further and see how we can have 
only one unified kind of primitive class.
If we want to be able to have an ArrayList of int, we need either to see int as 
a Q-type or Integer as a Q-type.
The later is more attractive because it means that at runtime we can see a 
ArrayList as an ArrayList
so call a method of an existing API using ArrayList with the more 
compact ArrayList.

So java.lang.Integer is a primitive class.
But it's not a true primitive class because
 - in a method descriptor because of backward compatibility, it has to be a 
Ljava/lang/Integer;
 - inside the '<' '>' of a generics, we need a way to express a Q-type.

I propose to introduce a new kind of primitive class, the builtin primitive 
class, that
 - use L-type in signature
 - are eagerly loaded by the VM (this is already the case) thus does not need 
to appear in the attribute Preload.
 - this is more controversial, inside a generics, int is translated by the 
compiler to Qjava/lang/Integer; + usual boxing/unboxing operations
   so ArrayList is a specialized generics using the class miror 
corresponding to Qjava/lang/Integer; as argument.

In practice, only the wrapper types, Byte, Short, Integer, etc, can be declared 
as builtin primitive
(by example, the compiler can restrict the use of builtin to java.base or 
java.lang).

Rémi



   

Re: Why do we need .ref class for primtive class ?

[Remi Forax]

- Original Message -
> From: "Remi Forax" 
> To: "valhalla-spec-experts" 
> Sent: Thursday, January 6, 2022 7:50:54 PM
> Subject: Why do we need .ref class for primtive class ?

> It just occurs to me that while ACC_VALUE is a bit that change the runtime
> semantics,
> something the VM should take care of, ACC_PRIMITIVE is not a bit that change 
> the
> runtime semantics, only the javac translation strategy,
> javac emits Q-types instead of L-type + the Preload attribute.
> 
> If value classes and primitive classes are equivalent at runtime, why do we 
> need
> to generate the .ref interface/abstract class ?
> We can use L-type instead.


Just to be clear, it's not about removing Complex.ref from the language but to 
translate it to LComplex;

Rémi

Why do we need .ref class for primtive class ?

[Remi Forax]

It just occurs to me that while ACC_VALUE is a bit that change the runtime 
semantics,
something the VM should take care of, ACC_PRIMITIVE is not a bit that change the
runtime semantics, only the javac translation strategy,
javac emits Q-types instead of L-type + the Preload attribute.

If value classes and primitive classes are equivalent at runtime, why do we 
need to generate the .ref interface/abstract class ?
We can use L-type instead.

Rémi

Re: Updated State of Valhalla documents

[Remi Forax]

> From: "Brian Goetz" 
> To: "valhalla-spec-experts" 
> Sent: Thursday, December 23, 2021 6:14:43 PM
> Subject: Updated State of Valhalla documents

> Just in time for Christmas, the latest State of Valhalla is available!

> [
> https://openjdk.java.net/projects/valhalla/design-notes/state-of-valhalla/01-background
> |
> https://openjdk.java.net/projects/valhalla/design-notes/state-of-valhalla/01-background
> ]
> [
> https://openjdk.java.net/projects/valhalla/design-notes/state-of-valhalla/02-object-model
> |
> https://openjdk.java.net/projects/valhalla/design-notes/state-of-valhalla/02-object-model
> ]
> [
> https://openjdk.java.net/projects/valhalla/design-notes/state-of-valhalla/03-vm-model
> |
> https://openjdk.java.net/projects/valhalla/design-notes/state-of-valhalla/03-vm-model
> ]

> The main focus for the last year has been finding the right way to expose the
> Valhalla features in the user model, in a way that is cleanly factored,
> intuitive, and clearly connects with where the platform has come from. I am
> very pleased with where this has landed.

> There are several more installments in the works, but these should give plenty
> to chew on for now!

I've done a rapid reading, 
in the objec-model 
primitive class Point implements Serializable 

should be 
primitive Point implements Serializable 

"value" is a modifier but "primitive" is a top level type. 

The design in part 3 is cool, because if i'm not mistaken, you can implement 
value classes without the support of Qtype in the classfile. 

Rémi 

Re: JEP update: Value Objects

[Remi Forax]

Brian, 
the last time we talked about IdentityObject and ValueObject, you said that you 
were aware that introducing those interfaces will break some existing codes, 
but you wanted to know if it was a lot of codes or not. 

So i do not understand now why you want to mix IdentityObject/ValueObject with 
the runtime behavior, it seems risky and if we need to backout the introduction 
of those interfaces, it will more work than it should. 
Decoupling the typing part and the runtime behavior seems a better solution. 

Moreover, the split between IdentityObject and ValueObject makes less sense now 
that we have 3 kinds of value objects, the identityless reference (B2), the 
primitive (B3) and the builtin primitive (B4). 
Why do we want these types to be seen in the type system but not by example the 
set containing only B3 and B4 ? 

Rémi 

> From: "Brian Goetz" 
> To: "daniel smith" , "Dan Heidinga"
> 
> Cc: "John Rose" , "valhalla-spec-experts"
> 
> Sent: Lundi 20 Décembre 2021 18:54:01
> Subject: Re: JEP update: Value Objects

> I was working on some docs and am not sure if we came to a conclusion on the
> rules about who may, may not, or must declare ValueObject or IdentityObject.

> Let me see if I can chart the boundaries of the design space. I'll start with
> IdentityObject since it is more constrained.

> - Clearly for legacy classes, the VM is going to have to infer and inject
> IdentityObject.
> - Since IdentityObject is an interface, it is inherited; if my super 
> implements
> IO, so am I.
> - It seems desirable that a user be *allowed* to name IdentityObject as a
> superinterface of an interface or abstract class, which constrains what
> subclasses can do. (Alternately we could spell this "value interface" or 
> "value
> abstract class"; this is a separate set of tradeoffs.)
> - There is value in having exactly one way to say certain things; it reduces 
> the
> space of what has to be specified and tested.
> - I believe our goal is to know everything we need to know at class load time,
> and not to have to go back and do complex checks on a supertype when a 
> subclass
> is loaded.

> The choice space seems to be
> user { must, may, may not } specify IO on concrete classes
> x compiler { must, may, may not } specify IO when ACC_VALUE present
> x VM (and reflection) { mops up }

> where "mopping up" minimally includes dealing with legacy classfiles.

> Asking the user to say "IdentityObject" on each identity class seems 
> ridiculous,
> so we can drop that one.

> user { may, may not } specify IO on concrete classes
> x compiler { must, may, may not } specify IO when ACC_VALUE present
> x VM (and reflection) { mops up }

> From a user model perspective, it seems arbitrary to say the user may not
> explicitly say IO for concrete classes, but may so do for abstract classes. So
> the two consistent user choices are either:

> - User can say "implements IO" anywhere they like
> - User cannot say "implements IO" anywhere, and instead we have an "identity"
> modifier which is optional on concrete classes and acts as a constraint on
> abstract classes/interfaces.

> While having an "identity" modifier is nice from a completeness perspective, 
> the
> fact that it is probably erased to "implements IdentityObject" creates
> complication for reflection (and another asymmetry between reflection and
> javax.lang.model). So it seems that just letting users say "implements
> IdentityObject" is reasonable.

> Given that the user has a choice, there is little value in "compiler may not
> inject", so the choice for the compiler here is "must" vs "may" inject. Which
> is really asking whether we want to draw the VM line at legacy vs new
> classfiles, or merely adding IO as a default when nothing else has been
> selected. Note that asking the compiler to inject based on ACC_VALUE is also
> asking pretty much everything that touches bytecode to do this too, and likely
> to generate more errors from bytecode manglers. The VM is doing inference
> either way, what we get to choose here is the axis.

> Let's put a pin in IO and come back to VO.

> The user is already saying "value", and we're stuck with the default being
> "identity". Unless we want to have the user say "value interface" for a
> value-only interface (which moves some complexity into reflection, but is also
> a consistent model), I think we're stuck with letting the user specify either
> IO/VO on an abstract class / interface, which sort of drags us towards letting
> the user say it (redundantly) on concrete classes too.

> The compiler and VM will always type-check the consistency of the value
> keyword/bit and the implements clause. So the real question is where the
> inference/injection happens. And the VM will have to do injection for at least
> IO at least for legacy classes.

> So the choices for VM infer seem to be:

> - Only inject IO for legacy concrete classes, based on classfile version,
> otherwise require everything to be explicit;
> - Inject 

Re: We have to talk about "primitive".

[Remi Forax]

> From: "Kevin Bourrillion" 
> To: "valhalla-spec-experts" 
> Sent: Mercredi 15 Décembre 2021 19:42:55
> Subject: We have to talk about "primitive".

> (Okay, so we're doing this)
> I think the rename to "primitive classes" happened during my outage last year.
> When I came back I made the decision to like it.

> Since then, I've found that in my explanatory model I'm fighting against it
> constantly. I think it may actually be fatally flawed.

> The points I raise here were surely already known at the time, and I know 
> there
> were good reasons for overriding them. But I feel the need to come back and
> push harder on their importance.

> Background: the textbook definition of "primitive" is centered on their nature
> of being elements-not-molecules, and I see no dispute about it. Also, there's
> no disputing the fact that we're allowed to adopt a different meaning if we so
> choose. So that's not even the fatal flaw.

As already said by John, there are atoms in term of user defined types but not 
at runtime, apart if declared volatile, a long or a double is two 32 bits 
values. 

> The main problem I think we can't escape is that we'll still need some word 
> that
> means only the eight predefined types. (For the sake of argument let's assume
> we can pick one and lean hard on it, whether that's "predefined", "built-in",
> "elemental", "leaf type", or whatever.)

I still hope that we can see used defined primitive and a builtin type the same 
way from a JLS point of view. 
Obviously, from the JVMS POV they are different, but i think one of our goal 
should be that the distinction between a builtin primitive and a user defined 
primitive should not visible in the JLS. 

> Definitely, our trying to minimize their specialness is virtuous. They should 
> be
> like helium: yes, they are molecules when you want a molecule! But on any
> deeper look they will clearly be "actually" elements, and the distinction will
> matter often enough.

> So we have to attempt to shift users' understanding of "primitive" while at 
> the
> same time injecting a new term to mean exactly what primitive used to mean.
> That's the old Indiana Jones switch and I don't have to tell you how that
> turned out for him.

> It would be difficult to pull off in a world where we were just pushing some 
> new
> server and the whole world gets the new model at once. But in this universe
> where every version of Java ever made all have to coexist, it's looking to me
> like a guaranteed source of never-ending confusion.

> I also think it robs us of our ability to smoothly portray the real changes of
> Valhalla. We want to be able to say "elements are still elements! now we have
> molecules too". Pedagogically that is always preferable to "elements aren't
> really what you thought they were". Okay, the real comparison is a little more
> nuanced than that, but I'll get to that now.

I agree retconing is better pedagogically because a lot of people think in term 
of analogy. 

> An alternative that seems to work fine, in my mental model at least, is:

> * Primitive types are examples of value types, and have always been.
>* Java never supported any other kinds of value types before, so we didn't
> distinguish the terms before.
> * Everything you associate with primitive types remains true.
> * But most of those traits really come from their value-type-ness.

> (I plan to make the above shifts to my model document already.)

> * Now we have user-defined value types too.
>* The way we user-define a type is with a class, so a value type is 
> defined by a
> "value class" (sorry B2).
> * The primitive types will now each get a value class.
> * These 8 classes will look as much like user-defined types as Object 
> does.
>* They, like Object, will have a "cheat" in their source code that no one 
> else
>gets to use. (Object's is that there is no implied `extends Object` or
>`super();`; these need no fields because the data they store is magically
> handled by the VM. These feel like similar cheats.)

> Then mopping up the rest:

>* Existing classes probably need a term like "reference classes" (in the 
> model
>I'm going to circulate that doubles down on values-are-not-objects, then 
> this
> wants to be "object classes", even though that feels weird at first).
>* I think the term for bucket 2 classes really ought to center on
>identitylessness, e.g. "noid", "noident", "idfree", or something. Anything 
> else
>is getting away from the essential meaning of the bucket; plus, we want 
> people
> to call bucket 1 classes "identity classes", don't we?

> Footnote: for a more concrete manifestation of this problem: I am sure we 
> cannot
> possibly get away with Class.isPrimitive() being true for these classes. 
> Right?

> Thoughts?

I agree but i don't think we should use "value type" as a term to encompass 
user defined primitive and builtin primitive. 

BTW, i think 

Re: Proposal: Static/final constructors for bucket-3 primitive classes.

[Remi Forax]

> From: "John Rose" 
> To: "Brian Goetz" 
> Cc: "valhalla-spec-experts" , "clement
> cherlin" 
> Sent: Thursday, December 9, 2021 5:30:50 AM
> Subject: Re: Proposal: Static/final constructors for bucket-3 primitive 
> classes.

> We have considered, at various points in the last six years or more, allowing
> user-defined primitive types to define (under user control) their own default
> values. The syntax is unimportant, but the concept is simple: Surely the user
> who defines a primitive type can also define default initializer expressions
> for each of the fields.

> But this would be a trail of tears, which we have chosen to avoid, each time 
> the
> suggestion comes up.

> This feature is often visualized as a predefined bit pattern, which the JVM
> would keep handy, and just stamp down wherever a default initializer is 
> needed.
> It’s can’t really be that simple, but even such a bit pattern is problematic.

> First of all is the problem of declaring the bit pattern. Java natively uses 
> the
> side effects of  to define constants using ad hoc bytecodes; it also
> defines (for some types but not others) a concept of constant expression.
> Neither of those fits well into a classfile that would define a primitive with
> a default bit pattern.

> If the bit pattern is defined using ad hoc bytecode, it must be defined in a 
> new
> pseudo-method (not  ), to execute not during the initialization of the
> newly-declared primitive class, but before . (Surely not! a reader might
> exclaim, but this is the sort of subtlety we have to deal with.) During
> initialization of a class C, all fields of its own type C must be initialized
> before the first bytecode of  executes, so that the static initializer
> code has something to write on. So there must be a “default value definition”
> phase, call it  , added after linking and before
> initialization of C, so C’s  method has something to work with. This
>  is really the body of a no-argument constructor of C, or 
> its
> twin. A no-argument constructor of C is not a problem, but having it execute
> before C’s  block is a huge irregularity, which the JVM spec is not
> organized to support, at present.

> This would turn into both JVMS and JLS spec. complexity, and more odd corners
> (and odd states) in the Java user experience. Sure, a user will say, “but I
> promise not to do anything odd; I just want this field to be the value (int)1
> ”. Yes, but a spec. must define not only the expected usages, but all possible
> usages, with no poorly-defined states.

> OK, so if  is not the place to define to define this elusive
> bit pattern, what about something more declarative, like a ConstantValue
> attribute? Surely we could put a similarly structured DefaultValue attribute 
> on
> every non-static field of a value type, and that would give the JVM enough
> information to synthesize the required bit pattern before it runs  .

> Consider the user model here: A primitive declaration would allow its fields 
> to
> have non-zero default values, but only drawn from the restricted set of
> constant expressions , because those are the ones which fit in the
> ConstantValue attribute. (They are true bit patterns in the constant pool, 
> plus
> String constants.) There is no previous place in Java where we make such a
> restriction, except case labels. Can you hear the groans of users as we try to
> explain why only constant expressions are allowed in that context? That’s the
> muzak of the trail of tears I mentioned above.

> But we have condy to fix that (someone will surely say).

you read my mind :) 

> But that’s problematic, because the resolution of constant pool constants of a
> class C requires C to be at least linked, and if the condy expression makes a
> self-reference to C itself, that will trigger C’s initialization, at an 
> awkward
> moment. Have you ever debugged a tangled initialization circularity, marked by
> mysterious NPEs on variables you know you initialized? I have. It’s a stop on
> the trail of tears I mentioned.

> But if we really worked hard, and added a bunch of stuff to the JVMS and JLS,
> and persuaded users not to bother us about the odd restrictions (to constant
> expressions, or expressions which “don’t touch the class itself”), we could
> define some sort of declarative default value initialization.

> What then? Well, ask the JVM engineers how they initialize heap variables,
> because those are the affected paths. Those parts of the JVM are among the 
> most
> performance-sensitive. Currently, when a new object or array is created, its
> whole body (except the header) is sprayed with a nice even coat of 
> all-zero-bit
> machine words. This is pretty fast, and it’s important to keep it fast. What 
> if
> creating an array required painting some beautifully crafted arabesque of a 
> bit
> pattern defined by a creative user? Well, it’s doable, but much more
> complicated. You need to load the bit pattern into live registers and (if it’s
> an 

Re: [External] : Re: JEP update: Value Objects

[Remi Forax]

> From: "John Rose" 
> To: "daniel smith" 
> Cc: "Dan Heidinga" , "valhalla-spec-experts"
> 
> Sent: Jeudi 2 Décembre 2021 00:56:02
> Subject: Re: [External] : Re: JEP update: Value Objects

> On Dec 1, 2021, at 3:29 PM, Dan Smith < [ mailto:daniel.sm...@oracle.com |
> daniel.sm...@oracle.com ] > wrote:

>> So we went down the path of "maybe there's no need for a flag at all" in 
>> today's
>> meeting, and it might be worth more consideration, but I convinced myself 
>> that
>> the ACC_VALUE flag serves a useful purpose for validation and clarifying 
>> intent
>> that can't be reproduced by a "directly/indirectly extends ValueObject" test.

>> As you suggest, though, we could mandate that ACC_VALUE implies 'implements
>> ValueObject’.

> Assuming ACC_VALUE is part of the design, there are actually four
> things we can specify, for the case when a class file has ACC_VALUE set:

> A. Inject ValueObject as a direct interface, whether or not it was already
> inherited.
> B. Inject ValueObject as a direct interface, if it is not already inherited.
> C. Require ValueObject to be present as a direct interface, whether or not it
> was already inherited.
> D. Require ValueObject to be present as an interface, either direct or
> inherited.

> A and B will look magic to reflection.
> B is slightly more parsimonious and less predictable than A.
> C and D are less magic to reflection, and require a bit more “ceremony” in the
> class file.
> D is less ceremony than C.
> Also, the D condition is a normal subtype condition, while the C condition is
> unusual to the JVM.

> I guess I prefer C and D over A and B because of the reflection magic problem,
> and also because of Dan H’s issue (IIUC) about “where do we look for the
> metadata, if not in somebody’s constant pool?”

> Since D and C have about equal practical effect, and D is both simpler to
> specify and less ceremony, I prefer D best of all.

> I agree that ACC_VALUE is useful to prevent “action at a distance”.

> There is the converse problem that comes from the redundancy:
> What happens if the class directly implements or inherits ValueObject
> and ACC_VALUE is not set? I guess that is an error also.

As Daniel said during the meeting and in a following email, from the POV of 
javac, the compiler should add "implements ValueObject" on all concrete value 
classes even if ValueObject is already present in the hierarchy to avoid action 
at distance (to detect when a super type change from implementing ValueObject 
to implement IdentityObject by example). With that requirement, for the VM, D 
and C are equivalent for all classes generated by javac. 

So D is Ok. 

Rémi 

> — John

Re: JEP update: Value Objects

[Remi Forax]

Hi Daniel,
this is really nice.

Here are my remarks.

"It generally requires that an object's data be located at a fixed memory 
location"
remove "fixed", all OpenJDK GCs move objects.
Again later, remove "fixed" in "That is, a value object does not have a fixed 
memory address ...".

At the beginning of the section "Value class declarations", before the example, 
i think we also need a sentence saying that fields are implicitly final.

Class file and representation, about ACC_PERMITS_VALUE, what's the difference 
between "permits" and "allow" in English ?

In section "Java language compilation",
"Each class file generated by javac includes a Preload attribute naming any 
value class that appears in one of the class file's field or method 
descriptors."
+ if a value class is the receiver of a method call/field access (the receiver 
is not part of the method descriptor in the bytecode).

In section "Performance model"
"... must ensure that fields and arrays storing value objects are updated 
atomically.",
not only stores, loads has to be done atomically too.

The part "Initially, developers can expect the following from the HotSpot JVM" 
is dangerous because it will be read as Hotspot will do that forever.
We have to be more vague here, "a Java VM may ..."

regards,
Rémi

- Original Message -
> From: "daniel smith" 
> To: "valhalla-spec-experts" 
> Sent: Mardi 30 Novembre 2021 01:09:06
> Subject: JEP update: Value Objects

> I've been exploring possible terminology for "Bucket 2" classes, the ones that
> lack identity but require reference type semantics.
> 
> Proposal: *value classes*, instances of which are *value objects*
> 
> The term "value" is meant to suggest an entity that doesn't rely on mutation,
> uniqueness of instances, or other features that come with identity. A value
> object with certain field values is the same (per ==), now and always, as 
> every
> "other" value object with those field values.
> 
> (A value object is *not* necessarily immutable all the way down, because its
> fields can refer to identity objects. If programmers want clean immutable
> semantics, they shouldn't write code (like 'equals') that depends on these
> identity objects' mutable state. But I think the "value" term is still
> reasonable.)
> 
> This feels like it may be an intuitive way to talk about identity without
> resorting to something verbose and negative like "non-identity".
> 
> If you've been following along all this time, there's potential for 
> confusion: a
> "value class" has little to do with a "primitive value type", as we've used 
> the
> term in JEP 401. We're thinking the latter can just become "primitive type",
> leading to the following two-axis interpretation of the Valhalla features:
> 
> -
> Value class reference type (B2 & B3.ref)  | Identity class type (B1)
> -
> Value class primitive type (B3)   |
> -
> 
> Columns: value class vs. identity class. Rows: reference type vs. primitive
> type. (Avoid "value type", which may not mean what you think it means.)
> 
> Fortunately, the renaming exercise is just a problem for those of us who have
> been closely involved in the project. Everybody else will approach this grid
> with fresh eyes.
> 
> (Another old term that I am still finding useful, perhaps in a slightly
> different way: "inline", describing any JVM implementation strategy that
> encodes value objects directly as a sequence of field values.)
> 
> Here's a new JEP draft that incorporates this terminology and sets us up to
> deliver Bucket 2 classes, potentially as a separate feature from Bucket 3:
> 
> https://bugs.openjdk.java.net/browse/JDK-8277163
> 
> Much of JEP 401 ends up here; a revised JEP 401 would just talk about 
> primitive
> classes and types as a special kind of of value class.

Re: EG meeting, 2021-11-17

[Remi Forax]

> From: "Brian Goetz" 
> To: "Kevin Bourrillion" 
> Cc: "Dan Heidinga" , "daniel smith"
> , "valhalla-spec-experts"
> 
> Sent: Jeudi 18 Novembre 2021 23:34:51
> Subject: Re: EG meeting, 2021-11-17

> I think it is reasonable to consider allowing bucket two classes to be 
> abstract.
> They could be extended by other classes which would either be abstract or
> final. The intermediate types are polymorphic but the terminal type is
> monomorphic.

> A similar argument works for records.

I suppose you are talking about empty (no field) abstract classes. 
We need that for j.l.Object, j.l.Number or j.l.Record. 

>From a user POV, it's not very different from an interface with default 
>methods. 

Rémi 

> Sent from my iPad

>> On Nov 18, 2021, at 5:27 PM, Kevin Bourrillion  wrote:

>> On Wed, Nov 17, 2021 at 7:05 PM Dan Heidinga < [ mailto:heidi...@redhat.com |
>> heidi...@redhat.com ] > wrote:

>>> Let me turn the question around: What do we gain by allowing
>>> subclassing of B2 classes?

>> I'm not claiming it's much. I'm just coming into this from a different
>> direction.

>> In my experience most immutable (or stateless) classes have no real interest 
>> in
>> exposing identity, but just get defaulted into it. Any dependency on the
>> distinction between one instance and another that equals() it would be a
>> probable bug.

>> When B2 exists I see myself advocating that a developer's first instinct 
>> should
>> be to make new classes in B2 except when they need something from B1 like
>> mutability (and perhaps subclassability belongs in this list too!). As far 
>> as I
>> can tell, this makes sense whether there are even any performance benefits at
>> all, and the performance benefits just make it a lot more motivating to do 
>> what
>> is already probably technically best anyway.

>> Now, if subclassability legitimately belongs in that list of 
>> B1-forcing-factors,
>> that'll be fine, I just hadn't fully thought it through and was implicitly
>> treating it like an open question, which probably made my initial question in
>> this subthread confusing.

>> --
>> Kevin Bourrillion | Java Librarian | Google, Inc. | [ 
>> mailto:kev...@google.com |
>> kev...@google.com ]

Re: [External] : Re: Consolidating the user model

[Remi Forax]

> From: "John Rose" 
> To: "daniel smith" 
> Cc: "Kevin Bourrillion" , "Brian Goetz"
> , "valhalla-spec-experts"
> 
> Sent: Jeudi 4 Novembre 2021 02:34:52
> Subject: Re: [External] : Re: Consolidating the user model

> On Nov 3, 2021, at 4:05 PM, Dan Smith < [ mailto:daniel.sm...@oracle.com |
> daniel.sm...@oracle.com ] > wrote:

>> (It is, I suppose, part of the model that objects of a given class all have a
>> finite, matching layout when accessed by value, even if the details of that
>> layout are kept abstract. Which is why value types are monomorphic and you 
>> need
>> reference types for polymorphism.)

>> The fact that the VM often discards object headers at runtime is a pure
>> optimization.

> Let’s see what happens if we say that (a) bare values have headers and (b)
> Object::getClass allows the user to observe part of the header contents.

> It follows then that the expression aPointVal.getClass() will show the 
> contents
> of aPointVal’s header, even if it is a compile-time constant.

> Point pv = new Point(42,42); // “class Point” is the definition of Point
> assert pv.getClass() == Point.class; // ok, that’s certainly the class
> assert pv.getClass() != Point.ref.class; // and it’s not a ref, so good

> That is all fine. There’s a little hiccup when you “box” the point and get the
> same Class mirror even though the “header” is a very real-heap resident value
> now:

> Point.ref pr = pv; // same object… now it’s on the heap, though, with a real
> live heap header
> assert pr.getClass() == Point.class; // same class, but...
> assert pr.getClass() != Point.ref.class; // we suppress any distinction the 
> heap
> header might provide

> There’s a bigger hiccup when you compare all that with good old int:

> int iv = 42; // “class int” is NOT a thing, but “class Integer” is
> assert iv.getClass() != int.class; // because int is not a class
> assert iv.getClass() == Integer.class; // ah, there’s the class!
> assert iv.getClass() == int.ref.class; // this works differently from Point
> assert ((Object)iv).getClass() == pr.getClass(); // this should be true also,
> right?

How can you have int.class not being a class and at the same time having the 
notation int.ref ?? 

If you suppose that int is now a primitive class, B3 bucket, then iv.getClass() 
== int.class, 
because it's equivalent to new int(iv).getClass() == int.class 
so 
assert iv.getClass() != Integer.class; //because Integer is the reference 
projection 
assert iv.getClass() != int.ref.class; // because int.ref is equivalent to 
Integer 

If you suppose that Integer is B2 bucket (after all, all other value based 
class are B2), then iv.getClass() == Integer.class 
because it's equivalent to Integer.valueOf(iv).getClass() == Integer.class 
so 
assert iv.getClass() != int.class; //because int.class is a fake type like 
void.class 
assert iv.getClass() != int.ref.class; // does not compile because Integer is 
B2 not B3 

I think i've missed something ? 

Rémi 

Re: identityless objects and the type hierarchy

[Remi Forax]

> From: "Kevin Bourrillion" 
> To: "Brian Goetz" 
> Cc: "valhalla-spec-experts" 
> Sent: Jeudi 4 Novembre 2021 22:34:54
> Subject: Re: identityless objects and the type hierarchy

> On Wed, Nov 3, 2021 at 12:43 PM Brian Goetz < [ mailto:brian.go...@oracle.com 
> |
> brian.go...@oracle.com ] > wrote:

>> On 11/3/2021 3:00 PM, Kevin Bourrillion wrote:

>>> Okay, let's stick a pin in proper-value-types (i.e. try to leave them out of
>>> this discussion) for a moment...

>>> One question is whether the existing design for the bifurcated type 
>>> hierarchy
>>> will carry right over to this split instead.

> Brian, your response reads like it is explaining/defending that design to me.
> But I believe I already understood it and wasn't expressing any problem with
> it.

> Now we're talking about making a smaller split first, "identity objects vs.
> identityless objects" (1 vs 2, not 1 vs 3), so I was inquiring into why that
> class model does or does not also work exactly as-is for this purpose.

> (Note that I assume if bucket 3's arrival requires another such type in the 
> mix,
> there would be a second such bifurcation under IdentitylessObject.)

I don't think a second bifurcation is needed. 
At runtime bucket 2 and bucket 3 behave the same apart from null. 
Given that IdentitylessObject (or whatever the name we choose) is an interface, 
it always accept null, 
so if they are typed as that interface, B2 and B3 behave exactly the same. 

Rémi 

Re: Consolidating the user model

[Remi Forax]

I really like this, it's far better than how i was seeing Valhalla, pushing 
.ref into a corner is a good move. 

I still hope that moving from B1 to B2 can be almost backward compatible, if no 
direct access to the constructor, no synchronized and reasonable uses of ==. 

My only concern now is the dual of Kevin's concern, 
what if people discover that they always want to use the identitiy-free 
reference types (B2), because it is better integrated with the rest of the Java 
world and that in the end, the OG/pure primitive types (B3) are almost never 
used. 

Rémi 

> From: "Brian Goetz" 
> To: "valhalla-spec-experts" 
> Sent: Mardi 2 Novembre 2021 22:18:46
> Subject: Consolidating the user model

> We've been grinding away, and we think we have a reduced-complexity user 
> model.
> This is all very rough, and there's lots we need to write up more carefully
> first, but I'm sharing this as a preview of how we can simplify past where 
> JEPs
> 401 and 402 currently stand.

> # Consolidating the user model

> As the mechanics of primitive classes have taken shape, it is time to take
> another look at the user model.

> Valhalla started with the goal of providing user-programmable classes which
> could be flat and dense in memory. Numerics are one of the motivating use
> cases, but adding new primitive types directly to the language has a very high
> barrier. As we learned from [Growing a Language][growing] there are infinitely
> many numeric types we might want to add to Java, but the proper way to do that
> is as libraries, not as language features.

> In the Java language as we have today, objects and primitives are different in
> almost every way: objects have identity, primitives do not; objects are 
> referred
> to through references, primitives are not; object references can be null,
> primitives cannot; objects can have mutable state, primitive can not; classes
> can be extended, primitive types cannot; loading and storing of object
> references is atomic, but loading and storing of large primitives is not. For
> obvious reasons, the design center has revolved around the characteristics of
> primitives, but the desire to have it both ways is strong; developers continue
> to ask for variants of primitive classes that have a little more in common 
> with
> traditional classes in certain situations. These include:

> - **Nullability.** By far the most common concern raised about primitive
> classes, which "code like a class", is the treatment of null; many developers
> want the benefits of flattening but want at least the option to have `null`
> as the default value, and getting an exception when an uninitialized instance
> is used.

> - **Classes with no sensible default.** Prior to running the constructor, the
> JVM initializes all memory to zero. Since primitive classes are routinely
> stored directly rather than via reference, it is possible that users might be
> exposed to instances in this initial, all-zero state, without a constructor
> having run. For numeric classes such as complex numbers, zero is a fine
> default, and indeed a good default. But for some classes, not only is zero
> not the best default, but there _is no good default_. Storing dates as
> seconds-since-epoch would mean uninitialized dates are interpreted as Jan 1,
> 1970, which is more likely to be a bug than the desired behavior. Classes
> may try to reject bad values in their constructor, but if a class has no
> sensible default, then they would rather have a default that behaves more
> like null, where you get an error if you dereference it. And if the default
> is going to behave like null, it's probably best if the default _is_ null.

> - **Migration**. Classes like `Optional` and `LocalDate` today are
> _value-based_, meaning they already disavow the use of object identity and
> therefore are good candidates for being primitive classes. However, since
> these classes exist today and are used in existing APIs and client code, they
> would have additional compatibility constraints. They would have to continue
> to be passed by object references to existing API points (otherwise the
> invocation would fail to link) and these types are already nullable.

> - **Non-tearability.** 64-bit primitives (`long` and `double`) risk _tearing_
> when accessed under race unless they are declared `volatile`. However,
> objects with final fields offer special initialization-safety guarantees
> under the JMM, even under race. So should primitive classes be more like
> primitives (risking being seen to be in impossible states), or more like
> classes (consistent views for immutable objects are guaranteed, even under
> race)? Tear-freedom has potentially signficant costs, and tearing has
> signficant risks, so it is unlikely one size fits all.

> - **Direct control over flattening.** In some cases, flattening is
> counterproductive. For example, if we have a primitive class with many
> fields, sorting a flattened array may be more expensive 

Re: EG meeting *canceled*, 2021-10-20

[Remi Forax]

I've sent a mail about considering all primitive types as always nullable on 
stack (as parameters or local variables).

Rémi 

- Original Message -
> From: "daniel smith" 
> To: "valhalla-spec-experts" 
> Sent: Mercredi 20 Octobre 2021 16:03:12
> Subject: EG meeting *canceled*, 2021-10-20

> No new topics today, so we'll cancel the meeting. Next scheduled slot is
> November 3.

Re: Addressing the full range of use cases

[Remi Forax]

On October 6, 2021 10:07:09 AM UTC, Maurizio Cimadamore 
 wrote:
>
>On 06/10/2021 10:56, Remi Forax wrote:
>>> - Existing class types like LocalDate are both nullable and atomic. These 
>>> are
>>> useful properties to preserve during migration; nullability, in particular, 
>>> is
>>> essential for source compatibility. We've provided reference-default
>>> declarations as a mechanism to make reference types (which have these
>>> properties) the default, with 'P.val' as an opt-in to value types. But in 
>>> doing
>>> so we take away the many benefits of value types by default, and force new 
>>> code
>>> to work with the "bad name".
>> The existing class LocalDate is not atomic per se, atomic in Java implies 
>> volatile and currently if a LocalDate field is updated in one thread, 
>> another thread may never see that update.
>> LocalDate is currently not tearable, a QLocalDate; is tearable in case of 
>> racy code.
>
>The fact that QLocalDate is tearable is a consequence of the fact that 
>e.g. elements of a QLocalDate[] cannot be read/written atomically - e.g. 
>in a single shot, unlike for references - which are pointers and can be 
>loaded/stored atomically (as per Java Memory Model).
>
>It's true the word "atomic" is sometimes used to refer to operations 
>such as CAS which provide strong inter-thread guarantees - this is not 
>what Dan had in mind here.

Right, I used to think that being tearable was a huge issue.

Now, I don't.
At least, not to the point of making primive classes non tearable the default.

As I said, the code need to have race issues to  observe that behavior and I 
think that a security token should not be a primitive class, i.e. i can live 
with the fact that a user can forge any permutations of my primitive class if 
he writes racy code on purpose. 

>
>Maurizio
>


Rémi

-- 
Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma 
brièveté.

Re: EG meeting, 2021-10-06

[Remi Forax]

Sadly, i will not be able to attend this meeting :(

regards,
Rémi

- Original Message -
> From: "daniel smith" 
> To: "valhalla-spec-experts" 
> Sent: Mercredi 6 Octobre 2021 00:52:20
> Subject: EG meeting, 2021-10-06

> EG Zoom meeting tomorrow, Wednesday October 6, at 4pm UTC (9am PDT, 12pm EDT).
> 
> We can discuss "Addressing the full range of use cases", which concerns how we
> support nullability, atomicity, and migration.

Re: Addressing the full range of use cases

[Remi Forax]

- Original Message -
> From: "daniel smith" 
> To: "valhalla-spec-experts" 
> Sent: Mardi 5 Octobre 2021 01:34:37
> Subject: Addressing the full range of use cases

> When we talk about use cases for Valhalla, we've often considered a very broad
> set of class abstractions that represent immutable, identity-free data. JEP 
> 401
> mentions varieties of integers and floats, points, dates and times, tuples,
> records, subarrays, cursors, etc. However, as shorthand this broad set often
> gets reduced to an example like Point or Int128, and these latter examples are
> not necessarily representative of all candidate value types.

yes !

> 
> Specifically, our favorite example classes have a property that doesn't
> generalize: they'll happily accept any combination of field values as a valid
> instance. (In fact, they're even happy to accept any combination of *bits* of
> the appropriate length.) Many candidate primitive classes don't have this
> property—the constructors do important validation work, and only certain
> combinations of fields are allowed to represent valid instances.

I now believe the mantra "code like a class acts as an int" is harmful.
A class provides encapsulation, an int has no encapsulation, there is a 
mismatch.

> 
> Related areas of concern that we've had on the radar for awhile:
> 
> - The "all zeros is your default value" strategy forces an all-zero instance
> into the class's value set, even if that doesn't make sense for the class. 
> Many
> candidate classes have no reasonable default at all, leading naturally to wish
> for "null is your default value" (or other, more exotic, strategies involving
> revisiting the idea that every type has a default value). We've provided
> 'P.ref' for those use sites that *need* null, but haven't provided a complete
> story for value types that want it to be *their* default value, too.
> 
> - Non-atomic heap updates can be used to create new instances that arbitrary
> combine previously-validated instances' fields. There is no guarantee that the
> new combination of fields is semantically valid. Again, while there's 
> precedent
> for this with 'double' and 'long' (JLS 17.7), those are special cases that
> don't generalize—any combination of double bit fields is *still a valid
> double*. (This is usually described as "tearing", although JLS 17.6 has
> something else in mind when it uses that word...) The language provides
> 'volatile' as a use-site opt-in to atomicity, and we've toyed with a
> declaration-site opt-in as well. But object integrity being "off" by default
> may not be ideal.
> 
> - Existing class types like LocalDate are both nullable and atomic. These are
> useful properties to preserve during migration; nullability, in particular, is
> essential for source compatibility. We've provided reference-default
> declarations as a mechanism to make reference types (which have these
> properties) the default, with 'P.val' as an opt-in to value types. But in 
> doing
> so we take away the many benefits of value types by default, and force new 
> code
> to work with the "bad name".

The existing class LocalDate is not atomic per se, atomic in Java implies 
volatile and currently if a LocalDate field is updated in one thread, another 
thread may never see that update.
LocalDate is currently not tearable, a QLocalDate; is tearable in case of racy 
code.

And yes, nullablibilty is a huge compatibility issue.

> 
> While we can provide enough knobs to accommodate all of these special cases,
> we're left with a complex user model which asks class authors to make n
> different choices they may not immediately grasp the consequences of, and 
> class
> users to keep 2^n different categories straight in their heads.

yes !

> 
> As an alternative, we've been exploring whether a simpler model is workable. 
> It
> is becoming clear that there are (at least) two clusters of uses for value
> types.  The "classic" value types are like numerics -- they'll happily accept
> any combination of field values as a valid instance, and the zero value is a
> sensible (often the best possible) default value.  They make relatively little
> use of encapsulation.  These are the ones that best "work like an int."  The
> "encapsulated" value types are those that are more like typical aggregates
> ("codes like a class") -- their constructors do important validation work, and
> only certain combinations of fields are allowed to represent valid instances.
> These are more likely to not have valid zero values (and hence want to be
> nullable).

I agree.

> 
> Some questions to consider for this approach:
> 
> - How do we group features into clusters so that they meet the sweet spot of
> user expectations and use cases while minimizing complexity? Is two clusters
> the right number? Is two already too many? (And what do we call them? What
> keywords best convey the intended intuitions?)

Two is too many, see below.

> 
> - If there are knobs within the clusters, what 

Re: Factory methods & the language model

[Remi Forax]

- Original Message -
> From: "Brian Goetz" 
> To: "daniel smith" , "Dan Heidinga" 
> 
> Cc: "valhalla-spec-experts" 
> Sent: Vendredi 10 Septembre 2021 20:25:50
> Subject: Re: Factory methods & the language model

>> I'm not particularly interested in settling on a bikeshed color, but am
>> interested in the general mood for pursuing this direction at all. (And not
>> necessarily right away, just—is this a direction we think we'll be going?)
>>
>> A few observations/questions:
>>
>> - 'new Foo()' traditionally guarantees fresh instance creation for identity
>> classes. Primitive classes relax this, since of course there is no unique
>> identity to talk about. Would we be happy with a language that relaxes this
>> further, such that 'new Foo()' can return an arbitrary Foo instance (or maybe
>> even null)? Or would we want to pursue a different, factory-specific 
>> invocation
>> syntax? (And if so, should primitive classes use it too?)
> 
> Let me offer some context from Amber that might be helpful, regarding
> whether we might want "factory" to be a language feature.
> 

[...]

> 
> Example 2 -- "with" expressions / reconstructors.  A number of
> interesting features come out of the pairing of constructors and
> deconstruction patterns with the same (or compatible) argument lists,
> such as `with` expressions (`point with { x = 3 }`). Handling this
> involves doing multiple overload selection operations, first to find a
> deconstructor that yields the right bindings, and then to find a
> compatible constructor that accepts the resulting exploded state.
> 
> Among the many problems of doing this (including the fact that parameter
> names are not required to be stable or meaningful), we also have the
> problem of "what if the class doesn't have a constructor, but a
> factory."  This would require the language to have a notion of factory
> method (e.g., `factory newPoint(int x, int y)`) so the compiler could
> try to match up a compatible factory with the deconstructor.

I think there is a way to not introduce a weird with expression syntax by 
piggybacking on the fact that a record is a weird tuple.

A record in Java is not just a tuple, i.e. a vector of values, but because all 
components are named also a compact key/value set.

The "with" expression is a case where we want to see a record as key/value set 
more than as a vector of values.

If we have a syntax to construct a record as a key/value set, this syntax can 
be slightly extended to express a "with" expression.

By example, if we have a syntax like
  Point p = Point { x: 3, y: 4 };

Then the syntax for the with expression will be to something like
  Point p2 = Point { x: 7, p };

I can hear you saying that i'm trying to trick you to add a new syntax for 
creating a record which is bigger that just the "with" expression.
And that's partially true.

I believe that the "key/value set" syntax for a record is something we should 
introduce in Amber anyway because it's a declarative syntax, the same way a 
Stream is, making the code easier to read. And people want a syntax like this 
so badly that they are up to writing a full builder pattern in their code just 
for being able to see the name of the parameters when creating complex objects.

regards,
Rémi

Re: Factory methods & the language model

[Remi Forax]

I will take the scenic road to answer :)

There is currently an issue with the fact that we present primitive class as 
constructor in Java the language but is translated not to a constructor in the 
class file.
This introduce a false sense of compatibility, the code is identical if it's a 
classical class and a primitive class, but it's a trap because those are not 
binary compatible.

We start with the idea of "code like a class" but the current runtime model is 
different for a primitive class and a classical class, Q-type and factory at 
construction.
I think Java should reflect that difference in the code instead of trying to 
retrofit the constructor syntax to work with primitive class.

Apart from the "code like a class" mantra, it seems we do not want users to use 
a syntax corresponding to withfield directly, the withfield are implicitly 
generated by the compiler from the this.x = x.

In Amber, in a compact constructor, we have chosen another way, to avoid users 
to write this.x = x, they are generated automatically by the compiler.
I think we can use the same idea here.

In term of syntax, instead of having a constructor, we can have a method 
__new__ (syntax up to debate obviously)
and the compiler generates the withfield to initialize the primitive object 
before returning it

  public primitve class Point {
private final int x;
private final int y;

public __new__(int x, int y) {
  if (x > 0 || y > 0) {
throw ...
  }
  // at the end the compiler generates the withfield to initialize the 
fields x and y with the value of the local variable x and y
}
  }

The idea is that in the method __new__,
- is implicitly static (so "this" is not available)
- there should be a local variable defined inisde __new__ with the same name as 
each field
- you can have as many overloads you want, using __new__() instead of "this()" 
to call in between the overloads the same way record constructors work for you 
to delegate between constructors

For the reflection API, we can either go and introduce a new kind of member 
reflected by a class like j.l.r.NewFactoryMethod or with flag-like method on 
j.l.r.Method (isNewFactoryMethod()) saying that it's a special kind of static 
method. I prefer the later because most existing code will still work with that 
model.

To summarize, i think the syntax should reflect the fact that a primitive class 
is not initialized the same way a classical class is.

So if we want to be consistent with that idea, it means that for the JVM, those 
factory methods __new__ are only available to primitive classes.
So the VM should reject
 - a primitive class with a method 
 - a reference class with a method 

This is i believe your solution (1).

regards,
Rémi

- Original Message -
> From: "daniel smith" 
> To: "valhalla-spec-experts" 
> Sent: Jeudi 9 Septembre 2021 16:23:41
> Subject: Factory methods & the language model

> JEP 401 includes special JVM factory methods, spelled  (or, 
> alternatively,
>  with a non-void return), which are needed as a standardized way to
> encode the Java language's primitive class constructors.
> 
> We have a lot of flexibility in how much we restrict use of these methods. Too
> many restrictions seem arbitrary and incoherent from the JVM's point of view;
> but too few restrictions risk untested corner cases, unfortunate compatibility
> obligations, and difficulties mapping back to the Java language model.
> 
> Expanding on that last one: for tools that operate with a Java language model,
> there are essentially three strategies for dealing with factory methods 
> outside
> of the core primitive class construction use case:
> 
> 1) Have the JVM reject them
> 2) Ignore them
> 3) Expand the model to include them
> 
> Taking javac as an example, here's what that looks like:
> 
> 1) If factory methods outside of primitive classes are illegal, javac can 
> treat
> classes with such methods as malformed and report an error.
> 
> 2) Or if javac sees a factory method in a non-primitive class, it can just 
> leave
> it out when it maps the class file to a language-level class. (There's
> precedent for this in, e.g., the treatment of fields with the same name and
> different descriptors.)
> 
> 3) Or we can allow javac to view factory methods in any class as 
> constructors. A
> few complications:
> 
>- Constructors of non-final classes have both 'new Foo()' and 'super()' 
> entry
>points; factories only support the first. So we either need to validate 
> that a
>matching pair of  and  exist, or expand the language to model
>factories independently from constructors.
> 
>- The language expects instance creation expressions to create fresh 
> instances.
>We need to either validate this behavior (does the factory look like
>"new/dup/"?) or relax the language semantics (perhaps this is in the 
> grey
>area of mixed binaries?)
> 
>- Factories can appear in abstract classes and interfaces. 

Re: Objects.newIdentity update

[Remi Forax]

- Original Message -
> From: "daniel smith" 
> To: "valhalla-spec-experts" 
> Cc: "Roger Riggs" 
> Sent: Lundi 19 Juillet 2021 19:59:03
> Subject: Objects.newIdentity update

[I've added Mark in CC]

> An update on Objects.newIdentity for Java 17: Roger did some work to put the
> feature together and get it reviewed.
> 
> https://bugs.openjdk.java.net/browse/JDK-8269096
> 
> However, while the implementation is straightforward, for libraries folks not
> deeply familiar with the Valhalla Project, the concept of a method that does
> the same thing as 'new Object()' did not seem particularly justified. I think
> they're especially uncomfortable with the idea of talking about creating an
> "identity" in a world in which all objects have identity.
> 
> https://bugs.openjdk.java.net/browse/JDK-8269097

Not all objects have an identity, we have already introduced value based class 
https://docs.oracle.com/en/java/javase/16/docs/api/java.base/java/lang/doc-files/ValueBased.html
a long time ago.

The JLS also says that lambdas have no identity.

First, i believe there is a problem of communication somewhere, the fact that 
newIdentity() returns new Object() is not important,
we all now, and this is especially true for people dealing with the core 
libraries that implementations change.

In a sense, it's normal to have a discussion about what identity means and 
Object being the root of everything, because introducing Objects.newIdentity() 
is a move to make current developers more aware of that. We have started that 
discussion with the introduction of value based class, more recently with JEP 
390, Objects.newIdentity() is just the continuation of the logic of delivering 
features in pieces.

With that in mind, not adding Objects.newIdentity() now seems shortsighted.

Rémi

[1] https://openjdk.java.net/jeps/390

Re: JEP 401 -- reflection and class literals

[Remi Forax]

> From: "Brian Goetz" 
> To: "valhalla-spec-experts" 
> Sent: Mardi 29 Juin 2021 18:48:25
> Subject: Re: JEP 401 -- reflection and class literals

> The general consensus here is that this stacking is slightly better than the
> previous one, so let's take this as the plan of record. Now, to explore the
> next turn of the crank...

> The concerns that were raised could be characterized by "do we *really* need 
> to
> even have separate class literals for Foo.ref and Foo.val?" And the answer is,
> sort of yes, sort of no, so there is a possible next turn of the crank that
> might or might not be a further improvement.

> Two existing constraints we have are:

> - Reflection (and MethodHandle) already do everything with jl.Class. That 
> means,
> for example, we had to invent the primitive mirrors a long time ago, so that
> you could describe a field or method parameter of type int. "Fixing" this is
> not really on the table right now.
> - The language has given this some surface; you can say `int.class` (which 
> maps
> to`getstatic Integer.TYPE`.)

> Given all this, reflection needs to be able to differentiate between LFoo and
> QFoo, which suggests a second mirror. And the user needs to be able to express
> this mirror to call getMethod() or Lookup::findXxx (and to interpret the
> results of reflective lookups). But maybe we don't need to give it so much air
> *in the language*, especially given how confusing it is because the language
> and VM views (which reflection is closer to) don't align entirely.

> Rather than talking about the ref vs val mirror (which is confusing, because
> getClass returns the same thing whether something is stored in a .ref or a
> .val), perhaps we can reframe in terms of the "class mirror and the flattened
> representation mirror", or the "primary mirror and the restricted mirror", or
> some other such (though, even the term "mirror" is foreign to users, since all
> the see is jl.Class.)

> In this model, Point.class would be the only mirror expressible directly in 
> the
> language, and it would always be the primary mirror. Reflection users would
> need to navigate to the secondary mirror through something like
> `Point.class.secondaryMirror()` or some other method on jl.Class. By not
> putting this in the language, we avoid some of the confusing aspects of this
> story.
+1 

Rémi 

> On 6/23/2021 11:13 AM, Brian Goetz wrote:

>> In working through the details of reflective support in JEP 401, I think 
>> we've
>> fallen into a slight "false consistency" regarding class literals. (This is a
>> complicated network with many interrelated moving parts, so I'm not looking 
>> for
>> "quick answers" here, as much as balancing global consistency with an
>> understandable user story.)

>> In the language, a primitive class declaration gives rise to a class (P) and
>> three types: P, P.ref and P.val. P is an alias for one of them; most of the
>> time, it's an alias for P.val, but for migrated value-based class, it's an
>> alias for P.ref. The language has a concept for tuning this mapping, 
>> currently
>> under the name `ref-default`. (The VM would like to remain ignorant of 
>> details
>> like this.)

>> In the VM, there is one class (P), whose instances can have two 
>> representations,
>> flattened and indirect, described by two descriptors (QP and LP).

>> The VM and reflection need mirrors for both of these descriptor types. This 
>> is a
>> problem we face today to some degree for the primitive types; there's a
>> "neutered" mirror for int.class that doesn't correspond to an actual declared
>> class with limited operational capability. These "secondary" mirrors are only
>> used for reflection / method handles, to reflect method and field 
>> descriptors.

>> We double down on this story for mirrors for primitive classes. Each 
>> primitive
>> class has a primary (unrestricted) mirror corresponding to the L descriptor,
>> and a secondary (restricted, flattened, possibly null-free) mirror
>> corresponding to the Q descriptor. The secondary mirror has only one job:
>> reflecting Q descriptors in method and field descriptors (and supporting 
>> method
>> handles in their emulation of same.)

>> When you ask an object for getClass(), it always hands back the primary 
>> mirror.

>> As a bonus, this has a nice compatibility story with Integer today; 
>> reflective
>> code is used to testing for Integer.class when an int is expected (reflection
>> is always boxed), so this will continue to work, because instances of `int`
>> will report `Integer.class` from `getClass()`.

>> All of this feels like the low-energy-state for extending mirrors to 
>> primitives.
>> Here's where I think we bobbled a little bit, and can correct.

>> Currently, we say "Well, if Point is an alias for Point.val, then Point.class
>> must be Point.val.class". This is consistent, but kind of useless. Because
>> then, for example:

>> void puzzler() {
>> assertEquals(Point.class, new Point(0,0).getClass());
>> 

Re: [External] : Re: JEP 401 -- reflection and class literals

[Remi Forax]

- Original Message -
> From: "Brian Goetz" 
> To: "Peter Levart" 
> Cc: "valhalla-spec-experts" 
> Sent: Dimanche 27 Juin 2021 22:01:21
> Subject: Re: [External] : Re: JEP 401 -- reflection and class literals

>>
>>
>> Seems like we are forced to re-use Class objects to hold this
>> additional info. Remi says that there should only be "primary" (ref
>> type) primitive class literals (Foo.class) and no (Foo.ref.class or
>> Foo.val.class) and that val type mirrors should be obtained by calling
>> a method (such as Foo.class.asPrimitiveValType()), but that makes
>> methods/constructors using val-typed parameters second class citizens
>> when using reflection.
> 
> And even worse with MethodHandle.

java.lang.reflect and java.lang.invoke are actually second class citizen APIs, 
they are built that way.
At least until we have to deal with API points throwing Throwable where the 
only sensible way to deal with it is to rethrow it.

Rémi

Re: JEP 401 -- reflection and class literals

[Remi Forax]

> From: "Brian Goetz" 
> To: "valhalla-spec-experts" 
> Sent: Mercredi 23 Juin 2021 17:13:27
> Subject: JEP 401 -- reflection and class literals

> In working through the details of reflective support in JEP 401, I think we've
> fallen into a slight "false consistency" regarding class literals. (This is a
> complicated network with many interrelated moving parts, so I'm not looking 
> for
> "quick answers" here, as much as balancing global consistency with an
> understandable user story.)

> In the language, a primitive class declaration gives rise to a class (P) and
> three types: P, P.ref and P.val. P is an alias for one of them; most of the
> time, it's an alias for P.val, but for migrated value-based class, it's an
> alias for P.ref. The language has a concept for tuning this mapping, currently
> under the name `ref-default`. (The VM would like to remain ignorant of details
> like this.)

> In the VM, there is one class (P), whose instances can have two 
> representations,
> flattened and indirect, described by two descriptors (QP and LP).

> The VM and reflection need mirrors for both of these descriptor types. This 
> is a
> problem we face today to some degree for the primitive types; there's a
> "neutered" mirror for int.class that doesn't correspond to an actual declared
> class with limited operational capability. These "secondary" mirrors are only
> used for reflection / method handles, to reflect method and field descriptors.

> We double down on this story for mirrors for primitive classes. Each primitive
> class has a primary (unrestricted) mirror corresponding to the L descriptor,
> and a secondary (restricted, flattened, possibly null-free) mirror
> corresponding to the Q descriptor. The secondary mirror has only one job:
> reflecting Q descriptors in method and field descriptors (and supporting 
> method
> handles in their emulation of same.)

> When you ask an object for getClass(), it always hands back the primary 
> mirror.
yes ! 

> As a bonus, this has a nice compatibility story with Integer today; reflective
> code is used to testing for Integer.class when an int is expected (reflection
> is always boxed), so this will continue to work, because instances of `int`
> will report `Integer.class` from `getClass()`.
yes ! 

> All of this feels like the low-energy-state for extending mirrors to 
> primitives.
> Here's where I think we bobbled a little bit, and can correct.

> Currently, we say "Well, if Point is an alias for Point.val, then Point.class
> must be Point.val.class". This is consistent, but kind of useless. Because
> then, for example:

> void puzzler() {
> assertEquals(Point.class, new Point(0,0).getClass());
> }

> will fail (unless Point is ref-default, at which point it succeeds.) I think 
> the
> mistake we made is too literally following the model of "Point is an alias for
> Point.val". So, I'm proposing a slight adjustment, that treats the unqualified
> class name contextually:

> - In the context of a variable type, Point is an alias for Point.val, or
> Point.ref, if ref-default (no change from current);
> - Where a class is needed (e.g., new Point(), instanceof Point), Point refers 
> to
> the class Point, not any of its derived types, and Point.{ref,val} are not
> valid in these contexts (no change from current);
> - Where a reflective literal is needed, the unqualified Point.class is always
> the primary mirror, which always matches the behavior of Object::getClass
> (change);
> - If an explicit literal is needed for reflection (e.g., calling getMethod()),
> you can be explicit and say Point.{ref,val}.class to say what you mean (no
> change).

> This aligns the meaning of "Point.class" with the actual behavior of other
> reflective API points.
I disagree on your last bullet point, 
I think that less is more in that context, we do not have to have a syntax to 
express the secondary class exactly like we do not allow Foo.class. 
So instead of allowing Point.ref.class and Point.val.class, i think it's better 
to have no language support given that we can already write 
Point.class.asPrimitiveObjectClass(). 

Not having a syntax for Point.ref.class/Point.val.class is important because as 
you say it's inconsistent, for good, with Point being either an identity class 
or a primitive object class. 
Not surfacing that inconsistency in the language make things easier to 
understand and using methods of java.lang.Class instead make the intent more 
clear at the expanse of being more verbose. 

regards, 
Rémi 

Re: Making Object abstract

[Remi Forax]

- Mail original -
> De: "John Rose" 
> À: "Brian Goetz" 
> Cc: "daniel smith" , "valhalla-spec-experts" 
> 
> Envoyé: Mercredi 16 Juin 2021 23:50:35
> Objet: Re: Making Object abstract

> On Jun 2, 2021, at 7:57 AM, Brian Goetz  wrote:
>> 
>> A minor bikeshed comment: We're asking users to change their `new Object()` 
>> to
>> `IdentityObject.newIdentity()`, and they may ask "why do I have to say
>> 'Identity' twice"?  (And by ask, I mean grumble, because we're already asking
>> them to change their working code.)
>> 
>> After a few minutes of thought, I think it might be a better fit to put this 
>> at
>> Objects::newIdentity.  The methods in Objects are conveniences that users 
>> could
>> write themselves, which this basically is -- there's nothing special about 
>> this
>> method, other than having a preferred alternative to `new Object()` which 
>> users
>> will understand.  So parking this where the Object conveniences go seems
>> slightly lower friction.
> 
> I think this is OK.
> 
> As a stretch move, I think we can even retro-upgrade
> the type checking of Objects::newIdentity with type
> variable trickery, when IdentityObject becomes real.
> 
> Please see:
> 
> https://bugs.openjdk.java.net/secure/attachment/95170/Foo.java
> https://bugs.openjdk.java.net/browse/JDK-8268919


I wonder if a simple way to avoid to allow any Ts if to allow to specify an 
intersection type as parameter type and/or return type of methods (we have 
'var' inside the body)
so instead of

  public static  T newIdentity() {

we can write
  
  public static Object & IdentityObject newIdentity() {

This requires a grammar change but it's exactly the type we want.


> 
> — John

Rémi

Re: JEP draft: Better-defined JVM class file validation

[Remi Forax]

- Mail original -
> De: "daniel smith" 
> À: "valhalla-spec-experts" 
> Envoyé: Vendredi 4 Juin 2021 19:11:11
> Objet: Re: JEP draft: Better-defined JVM class file validation

>> On Jun 4, 2021, at 10:41 AM, Dan Smith  wrote:
>> 
>> Posted a new JEP draft, here:
>> 
>> http://openjdk.java.net/jeps/8267650

Hi Dan,
that's a lot of nice cleanups.

If i can emit a critic, i would have preferred to have the text of the JEP to 
clearly separate spec issues from Hotspot issues,
to the point where i wonder if it's not better to have two JEPs.

> 
> I'll draw your attention to this section, in case it raises any red flags:
> 
> ---
> 
> Risks and Assumptions
> 
> Changing JVM validation behavior is often a risk, because it may cause legacy
> classfiles to fail with new errors, or, more subtly, new class files with old
> version numbers to be accepted, but then fail on older JVMs.
> 
> In general, the HotSpot changes proposed in this JEP are narrow in scope, 
> often
> in corner cases that real world code is unlikely to probe. And many of the
> changes only modify the type of error being thrown or the timing of an error
> check. That said, the most likely areas of concern are:
> 
>   • New errors caused by improper appearances of the Module, 
> ModulePackages,
>   ModuleMainClass, and ConstantValue attributes.

yes, those attributes should be checked by the VM an not only in Java code.

> 
>   • New errors caused by pre-51 class files that declare a useless method 
> with
>   name  and 1 or more parameters.

I don't see the point of this change, unlike all the others, this one will 
reject classes that were valid before.

> 
>   • Accepting class files with malformed optional attributes, even though 
> those
>   class files could fail to load on an older JVM.

It's not clear to me if this change is only for classfile version >= V18 or not.
I would vote for having that change applied on all classes given it's an 
Hotspot bug, not a spec bug.

> 
> Besides the risk to JVM users, there is some risk that, by relaxing the
> constraints on optional attributes, downstream tools will be surprised by
> unvalidated attribute contents in class files that can be successfully loaded.

Actually, the fact that the InnerClasses attribute is validated by the VM is 
confusing because it implies that the VM uses the values of that attributes to 
check access which it does not.
For me, the spec says that those attributes are optionals so Hotspot should 
behave as the spec says.

And i'm a little worry about rejecting the ConstantValue attribute if the field 
is not static because i think i've seen a code using ConstantValue on all 
fields but i'm not able to recall where so i've not idea if it was in library 
used or not. I suppose this behavior will be gated with a V18 checks so it 
should not be an issue anyway.

> 
> These risks need to be balanced against the cost of the extra complexity
> required to fully specify and maintain longstanding, often ad hoc HotSpot
> behavior.
> 
> ---

regards,
Rémi

Re: [External] : Re: Making Object abstract

[Remi Forax]

> De: "Brian Goetz" 
> À: "Dan Heidinga" 
> Cc: "daniel smith" , "valhalla-spec-experts"
> 
> Envoyé: Samedi 5 Juin 2021 17:21:11
> Objet: Re: [External] : Re: Making Object abstract

> Rampdown is next week; time is fleeting.

> I think the path of adding Objects::newIdentity in 17 seems the best
> alternative. If there are objections, make them now.
no objection, quite the opposite, i agree with Dan H analysis. 

Rémi 

> On 6/4/2021 4:10 PM, Dan Heidinga wrote:

>> Quoting from several previous emails in this thread:

>> Brian said:

>>> I agree that we can introduce the new API point immediately.  The 17 window
>>> hasn't even closed yet!  But we'd have to get a move on.  But 
>>> realistically, we
>>> can expect it to be several years before we are comfortable erroring on the
>>> `new Object()` constructor.

>> Dan S. responded:

>>> Of course these details can evolve, but as it stands, there's not a good 
>>> path to
>>> introducing the API before we introduce primitive objects—IdentityObject 
>>> has no
>>> reason to exist without primitive objects. So efforts to get people to 
>>> migrate
>>> to something new will need to wait until then. (As discussed earlier in the
>>> thread, that's fine—we're inevitably going to have a period of time when 
>>> 'new
>>> Object()' does something weird.)

>> And Brian proposes:

>>> What I like about the Objects placement (thought took me a while to come
>>> around to this) is that it fits into the other Objects operations, in
>>> that they are all sugar for code we could write without help.  I don't
>>> think we need or want a canonical "class of all anonymous identity
>>> objects" type.

>> This gives us a pretty reasonable story for where to put a minimal
>> helper API today, before the window for 17 closes.  Even taking the
>> "several years before we are comfortable erroring" into account, it
>> would help the ecosystem migrate to Valhalla if we get the replacement
>> API into the upcoming LTS so the adoption is higher before we start
>> erroring on it.

>> We know - for better or for worse - that many applications leap from
>> LTS to LTS which requires library authors to support previous LTS
>> releases while adding support for current release.  And that apps
>> can't leap faster than their dependencies so we want to smooth the
>> path that lets applications track the release cadence.

>> This Objects::newIdentity api seems like an easy one to add today that
>> we can point libraries at it as they adopt 17.  I don't see much risk
>> that Valhalla will change in a way that invalidates this api.
>> Wouldn't it be better to lay that foundation now, before 17 ships,
>> then to wait?

>> --Dan

Make new Object() backward compatible

[Remi Forax]

I think there is a way to avoid all the issues we have with new Object().

The problem:
  We want to inject the interface IdentityObject on all classes that are 
neither a "no field" abstract class nor a primtive class.
  But java.lang.Object acts as a "no field" abstract class but it's not an 
abstract class.

Clearly, there are two java.lang.Object, the one which is the root of all 
classes (the "no field" abstract one) and the one which can be instantiated 
that should implements IdentityObject.

The logical conclusion is that java.lang.Object is a parametric class with a 
parameter saying if it should implement IdentityObject or not.

The "raw" version does not implement IdentityObject, that the one used when a 
class extends Object.
The parameterized version implements IdentityObject and is the one used by new 
Object()

So the classfile Object.class should contain a SpecializationAnchor, and being 
a "no field" class or not is an attribute that references the Anchor.

To maintain backward compatibility, when the VM sees the opcode NEW 
java/lang/Object, it adds a new constant pool entry of kind 
SpecializationLinkage that reference java/lang/Object with a constant asking 
for version that implements IdentityObject and rewrite NEW java/lang/Object to 
NEW Linkage(java/lang/Object).

Obviously, there is the bootstraping issue with the code in java.lang.invoke 
making the BSM working not be initialized at the time java.lang.Object need to 
be available, so the VM has to inject the binding early as if it was resolved 
by the BSM but i think it's workable.

When the VM starts, if there are some new Object() executedbefore the code of 
java.lang.invoke is ready, those code can be rewritten to use a special class 
defined in the JDK for that like
  package jdk.internal.whatever;
  class LockObject { }

Even if it's not strictly necessary, javac can be changed to emit the 
Linkage(java/lang/Object) each time a new Object() is needed to the rewriting 
by the VM described above need to be done only for the classfiles that are 
using previous versions.

I believe this change is fully source and binary compatible and nice property 
is that 
  new Object().getClass() == Object.class
is still true.

Rémi

Re: Meeting today: IdentityObject

[Remi Forax]

There is a 3rd way, uses the TypeRestriction attribute to still be binary 
backward compatible.

Let suppose that we have a way to say that javac needs to use TypeRestriction 
when generating a specific type in the class file
like
  import ...

  __restrict__ java.lang.IdentityObject __to__ java.lang.Object;

Rémi

- Mail original -
> De: "Remi Forax" 
> À: "valhalla-spec-experts" 
> Envoyé: Mercredi 5 Mai 2021 16:39:01
> Objet: Meeting today: IdentityClass

> If it's possible, i would like to discuss about IdentityClass again.
> 
> As noted in the last draft, adding IdentityClass automatically at runtime is 
> not
> a compatible change if tests that uses Class::getInterfaces() are not written
> correctly,
> and sadly, a lot of tests are written that way. I'm guilty to having written
> those tests too.
> I don't believe that the solution is to tweak the reflection because adding
> IdentityClass at runtime is already a hack and introducing a hack² (hack 
> square
> hack) is usually where we should stop before anyone sanity goes under the bus.
> 
> The purpose of IdentityClass is
> - to have a classfile for the javadoc describing the behavior of all identity
> class
> - to be used as type or bound of type parameter.
> 
> Apart the result of Class::getInterfaces() being hardcoded, IdentityClass has 
> to
> other issues, as javadoc container, given that IdentityClass is inserted by 
> the
> VM, it means there is no place in the Java code where someone can click to see
> the javadoc in an IDE, we have the same issue with java.lang.Record currently,
> the class is added by javac automatically and unlike java.lang.Enum, there is
> no method useful on java.lang.Record (equals/hashCode and toString are defined
> directly on the record) so very few of my student where able to understand why
> a record with a NaN value was equals to itself.
> But there is a more serious issue, using IdentityClass is not backward
> compatible with Object.
> When we have introduced IdentityClass, one scenario was to be able to declare
> that the type parameter corresponding to the keys (K) to only support identity
> class.
> This is not possible using IdentityClass because the erasure of K will be
> IdentityClass instead of Object (IdentityClass also appears when the compiler
> will to a lub, so the common class of String and URI will be computed as
> IdentityClass instead of Object leading to source compatibility issues).
> 
> I think at that point, we should go back to our blackboard and see if there is
> no other solution.
> 
> I see two, one is to do *nothing* and do not add a type saying that only
> identity class is a corner case after all,
> the other is to piggyback on erasure not unlike Scala does, i.e. IdentityClass
> is a fake class that is erased to Object at runtime.
> 
> regards,
> Rémi

Re: EG meeting, 2021-05-05

[Remi Forax]

- Mail original -
> De: "daniel smith" 
> À: "valhalla-spec-experts" 
> Envoyé: Mercredi 5 Mai 2021 16:23:08
> Objet: EG meeting, 2021-05-05

> The next EG Zoom meeting is today at 4pm UTC (9am PDT, 12pm EDT).
> 
> Topics to discuss:
> 
> "consolidated VM notes for primitive classes": Updated section of State of
> Valhalla addressing JVM changes
> 
> "Classfile artifacts to support compilation/reflection": Brian listed 
> properties
> of primitive classes and value types that need to be encoded in class files

+ IdentityClass

Rémi

Meeting today: IdentityClass

[Remi Forax]

If it's possible, i would like to discuss about IdentityClass again.

As noted in the last draft, adding IdentityClass automatically at runtime is 
not a compatible change if tests that uses Class::getInterfaces() are not 
written correctly,
and sadly, a lot of tests are written that way. I'm guilty to having written 
those tests too.
I don't believe that the solution is to tweak the reflection because adding 
IdentityClass at runtime is already a hack and introducing a hack² (hack square 
hack) is usually where we should stop before anyone sanity goes under the bus.

The purpose of IdentityClass is
- to have a classfile for the javadoc describing the behavior of all identity 
class
- to be used as type or bound of type parameter.

Apart the result of Class::getInterfaces() being hardcoded, IdentityClass has 
to other issues, as javadoc container, given that IdentityClass is inserted by 
the VM, it means there is no place in the Java code where someone can click to 
see the javadoc in an IDE, we have the same issue with java.lang.Record 
currently, the class is added by javac automatically and unlike java.lang.Enum, 
there is no method useful on java.lang.Record (equals/hashCode and toString are 
defined directly on the record) so very few of my student where able to 
understand why a record with a NaN value was equals to itself.
But there is a more serious issue, using IdentityClass is not backward 
compatible with Object.
When we have introduced IdentityClass, one scenario was to be able to declare 
that the type parameter corresponding to the keys (K) to only support identity 
class.
This is not possible using IdentityClass because the erasure of K will be 
IdentityClass instead of Object (IdentityClass also appears when the compiler 
will to a lub, so the common class of String and URI will be computed as 
IdentityClass instead of Object leading to source compatibility issues).

I think at that point, we should go back to our blackboard and see if there is 
no other solution.

I see two, one is to do *nothing* and do not add a type saying that only 
identity class is a corner case after all,
the other is to piggyback on erasure not unlike Scala does, i.e. IdentityClass 
is a fake class that is erased to Object at runtime.

regards,
Rémi

Parametric VM class file format

[Remi Forax]

Hi all,
at least as an exercise to understand the proposed class file format for the 
parametric VM, i will update ASM soon (in a branch) to see how things work.

As usual with ASM, there is the question of sharing the same index in the 
constant pool,
i.e. should two anchors (SpecializationAnchor) that have the same kind and the 
same BSM + constant arguments share the same constant pool index ?
And same question with two linkages (SpecializationLinkage).

I believe the answer is yes for both, the index is shared.

The other questions are about "ldc Linkage", first, why do we need a ldc 
Linkage with a Linkage that reference a class given that we already have Condy ?
And second question, is there a ldc Linkage with the linkage referencing 
something else than a class ?

regards,
Rémi

parametric-vm and stacktrace

[Remi Forax]

I wonder if the ParameterBinding should not have kind of name or at least a 
textual representation so you can see them in stacktraces,
it will be really helpful for debugging.

Rémi

parametric-vm / section other upcalls

[Remi Forax]

In the sub-section "other upcalls",
for type-testing, the VM does an upcall to the method "isAssignableFrom" of the 
Species but it's not clear for me how a user can creates such "Species" given 
that it seems that the way to configure a species is to return a 
ParameterBinding properly configured.

It looks like a vestige of an old behavior ? right ?

Rémi

Parametric-vm spec / unused CONSTANT_Parameter is illegal

[Remi Forax]

Hi all,
slowly reading the Parametric-vm spec.

With my ASM hat,
"As a structural constraint, it is illegal for a CONSTANT_Parameter constant to 
be unused."

This is different of all other CONSTANTs and doesn't work well with the idea 
that you can patch a classfile by copying the existing constant pool and 
replace only the method(s) you want, because if i replace a parametric method 
by a non parametric one, the existing CONSTANT_Parameter will stay in the 
constant pool with no reference to it anymore.

Being able to patch a classfile like this is very important in term of speed 
for some transformers / agents.

If the VM is able to find that a CONSTANT_Parameter is unused, instead of 
throwing an error, why not ignoring it ?

Rémi

Re: EG meeting, 2021-03-10

[Remi Forax]

cool !

Rémi

- Mail original -
> De: "daniel smith" 
> À: "valhalla-spec-experts" 
> Envoyé: Mercredi 10 Mars 2021 17:12:46
> Objet: EG meeting, 2021-03-10

> The next EG Zoom meeting is today at 5pm UTC (9am PST, 12pm EST).
> 
> No new email threads, but John would like to spend some time talking about the
> specialization design.

Inline Record vs JLS / Reflection

[Remi Forax]

Hi all,
currently an inline record like this
  public @__inline__ record IntBox(int value) {
  }

generates two classes, IntBox and IntBox$ref, IntBox being the record itself 
and IntBox$ref being the abstract class representing the nullable version of an 
inline that inherits from java.lang.Record.

This doesn't works well with the JLS view / the reflection API because for them 
a record class is a direct super class of java.lang.Record while in this case 
java.lang.Record is not a direct super type.
I believe the reflection should be changed to say that a record is a subtype of 
java.lang.Record.

Rémi

Use of Class.getSimpleName() in java.lang.invoke and InnerClass attribute verification

[Remi Forax]

Most of the error message form the error that can be thrown by classes of 
java.lang.invoke are using Class.getSimpleName() which makes debugging hidden 
classes painful because getSimpleName() scrubs the hexadecimal number part of a 
hidden class name.

It also seems that the VM has a zealot stance on the InnerClass attribute 
verification, which i think is wrong.
  see 
https://mail.openjdk.java.net/pipermail/core-libs-dev/2020-November/071813.html

Rémi

Using a Condy instead of a Constant_Utf8

[Remi Forax]

Using a condy that returns a String instead of a method descriptor in the 
bytecode ?
I'm currently trying to remove the uses of Unsafe.defineAnonymousClass to use 
Lookup.defineHiddenClass instead.
There is one case where i dynamically patch a method descriptor so I can select 
how many arguments will be sent to a closure,
i.e. in the bytecode i put all arguments on the stack but because i've patched 
the callee descriptor, only some of them will be used.

I see defineAnonymousClass has a way to push data to a template class file and 
defineHiddenClass has a way to pull the data from the template class file.
But i can not currently pull data that will be used by a NameAndType info 
because the descriptor_index has to be a Constant_Utf8 index and can not be a 
condy with a String as descriptor.

This is related to Lazy static final field [1] and also to class templating in 
general because it equivalent to seeing all the template holes as condy even if 
it"s less efficient because you need to duplicate the whole constant pool.

Rémi

[1] https://bugs.openjdk.java.net/browse/JDK-8209964

Re: EG meeting, 2020-11-04

[Remi Forax]

- Mail original -
> De: "daniel smith" 
> À: "valhalla-spec-experts" 
> Envoyé: Mercredi 4 Novembre 2020 19:18:02
> Objet: Re: EG meeting, 2020-11-04

>> On Nov 3, 2020, at 4:53 PM, Dan Smith  wrote:
>> 
>> The next EG Zoom meeting is Wednesday, 5pm UTC (12pm EST, 9am PST).
>> 
>> We'll talk about "Source code analysis: calls to wrapper class constructors",
>> including tentative plans for helping existing clients of Integer.,
>> Double., etc., migrate to a VM in which these are primitive classes.
> 
> Summarizing: consensus that the proposed path (lean hard on deprecation, but
> also provide bytecode rewrite tooling) is the "least bad" option. But we
> discussed some additional areas of concern/risk:
> 
> - The proposal is to perform bytecode rewrites as an opt-in—e.g., via an agent
> provided at the command line. But if it turns out that it's quite common to
> need to use this incantation, this could be a major source of friction to
> adoption. On the other hand, if we make it the default behavior, we're talking
> about changing JVM semantics. That might be possible, but there are reasons to
> be wary of burning this behavior into JVMS.
> 
> - It's possible people will be frustrated that sources written for pre-9 javac
> will fail to compile in the primitive classes version of javac. Ideally, they
> will first try to compile on 11, 17, etc., and see the warnings. In a sense,
> this is just how deprecation works, but it's also true that this is an
> especially sensitive/widespread API being deprecated. The solution would be to
> allow javac to do something special with 'new Integer', or perhaps to just 
> keep
> the public constructors in the primitive class (as a "discouraged" but
> available API point).
> 
> - Deprecation warnings are good for source, not so good for binary 
> dependencies.
> The ecosystem could really use better tooling to flag usages of deprecated
> APIs, including at build time (IDEs, Maven, etc.) and runtime (JVM warnings).
> 
> For the first two, it's fair to say that it's hard to predict how those risks
> will play out, but we should keep them in mind until the release gets closer.
> 
> Another idea, briefly discussed: if this plan works for 'new Integer', might 
> it
> also be best for 'new Object'? We're more comfortable baking special-case
> behavior into the JVM in that case, because the rewrite is simpler, but we
> could revisit that decision.

Not sure new Object() is simpler because in term of bytecode INVOKEPECIAL 
Object ()V has two meanings (init call and super call) and we want to 
re-write only the former but not the later, while INVOKEPECIAL Integer 
(I)V can not be used as a super call.

Rémi

Re: Source code analysis: calls to wrapper class constructors

[Remi Forax]

- Mail original -
> De: "John Rose" 
> À: "daniel smith" 
> Cc: "valhalla-spec-experts" 
> Envoyé: Mercredi 28 Octobre 2020 05:56:29
> Objet: Re: Source code analysis: calls to wrapper class constructors

> On Oct 27, 2020, at 12:27 PM, Dan Smith  wrote:
>> 
>> This tooling will support common bytecode patterns like 'new Foo; dup; ...;
>> invokespecial Foo.;', but will not be a comprehensive solution.
>> (Mimicking the behavior of instance initialization method invocation in full
>> generality would be a very difficult task.)
> 
> One of the reasons it’s not going to be comprehensive
> is code like new Integer(complicatedExpr()), in which
> the `new` and `invokespecial ` are separated
> by (almost) arbitrarily complex bytecode.  The two
> instructions don’t even have to be in the same basic
> block (at the bytecode level):
> 
> new Integer(foo() ? bar() : baz())
> // compiles to 4 BB’s in a diamond
> 
> If we add switch expressions with large sub-blocks,
> I think we get peak separation of the start and
> end parts of the new/init dance:
> 
> new Integer(switch (x) {
>  case 1 -> { complicatedBlock: try { … } catch ... ; return 0;
>  default -> { for (;;) … }} )
> 
> All of this gives me yet one more reason we would have
> been better off with factory methods instead of
> open-coding the new/init dance.  It was, in hindsight,
> a false economy to open code the object creation “guts”
> instead of putting them in factory API points.
> 
> And with an eye toward future evolutions of legacy code
> (legacy code not yet in existence!), and uniformity with
> the factory methods of inline classes, let’s try harder
> to get rid of the new/init dance for identity objects.

I believe there is a quick and dirty trick,
replace new java/lang/Integer by 3 NOPs and replace INVOKESPECIAL 
java/lang/Integer  (I)V by INVOKESTATIC java/lang/Integer valueOf 
(I)Ljava/lang/Integer;

It has to be done after the code is verified because the new execution doesn't 
push java/lang/Integer on the stack anymore before calling the arbitrary init 
expression thus any StackMapTables in between the NOPs and INVOKESTATIC are 
invalid.

> 
> — John

Rémi

Re: Source code analysis: calls to wrapper class constructors

[Remi Forax]

- Mail original -
> De: "daniel smith" 
> À: "valhalla-spec-experts" 
> Envoyé: Mardi 27 Octobre 2020 20:27:39
> Objet: Re: Source code analysis: calls to wrapper class constructors

>> On Oct 19, 2020, at 6:01 PM, Dan Smith  wrote:
>> 
>> In the context of the Warnings for Value-Based Classes JEP, we're looking for
>> usages of the deprecated wrapper class constructors ('new Integer(...)', 'new
>> Double(...)', etc.). When do these get used? How often is this motivated by
>> wanting a unique object vs. legacy code that has no particular reason not to
>> use 'valueOf'?
>> 
>> We've got some investigations going on at Oracle, including looking at some
>> open-source projects, but I'm interested in examples from other
>> companies/projects as well. Please investigate and share what you find!
>> 
>> I'll reply in a few days with our analysis for the code we look at.
> 
> So, some conclusions that we've drawn:
> 
> - In 2020, the constructor calls are fairly pervasive, even in recently 
> released
> binaries. Removing these constructors may be the most disruptive single API
> change we've ever made.
> 
> - The trend is good—serious projects have mostly responded to the deprecation
> warnings introduced in 9. In 2024 (for example), the picture may be much
> better.
> 
> - It is impossible, given the current JVM model for primitive classes, for
> Integer to both be a primitive class and support 'new java/lang/Integer'.
> Binaries calling the deprecated constructors simply won't work.
> 
> - There is a meaningful semantic difference, in terms of promising unique
> identity, between code that does 'new Integer' and code that does
> 'Integer.valueOf' or implicitly boxes. The latter is better aligned with the
> class's future behavior as a primitive class.
> 
> - Almost no usages care about the identity distinction—they just want boxing.
> But it's difficult to automatically detect the usages that do.
> 
> These points lead to the following tentative plan:
> 
> We'll proceed as planned with deprecation for removal. The message for all 
> Java
> programs is to stop using these constructors if you want to run in future 
> JDKs.
> Typically, the simplest refactoring is to replace 'new Integer(x)' with just
> 'x', but in some cases you might want 'Integer.valueOf(x)'. When you refactor,
> you should confirm that the change in identity semantics is acceptable.
> 
> We'll encourage IDEs to provide tooling for updating sources, including batch
> processing. (As an example, IntelliJ already highlights deprecated APIs and
> suggests conversions, although I'm not familiar with its batch processing
> features.)
> 
> For legacy binaries that have not been updated but that need to run on a 
> future
> JDK, we'll provide tooling to rewrite their bytecode containing constructor
> calls, either as a preprocessing step on jar files, or as a runtime
> command-line argument.
> 
> This tooling will support common bytecode patterns like 'new Foo; dup; ...;
> invokespecial Foo.;', but will not be a comprehensive solution.
> (Mimicking the behavior of instance initialization method invocation in full
> generality would be a very difficult task.) It will also carry behavioral
> compatibility risks. For these reasons, it will be available as a workaround,
> not as default JVM behavior.

Three remarks:
- the compiler can warn because a code is using new Integer(value) or warn 
because the compiler will automatically transform all new Integer(value) to 
Integer.valueOf() once Valhalla is integrated,
  i prefer the second solution, having a warning message saying that in the 
future new Integer() will not be supported and that all new Integer(...) will 
be transformed by the compiler automatically to Integer.valueOf(). 
- the introduction of the strong encapsulation in 9 was a very similar 
challenge, the first thing to do is to raise awareness, having a warning at 
runtime (so emitted by the VM) per callsite using new Wrapper(...)
  will help a lot. People can detect easily if they are using a dependency that 
use something that will not be backward compatible (this warning should be 
emitted by Java 16+, because there is no point to wait
  and because of JEP 396, there will be a second wave of people wanting to 
update the library they maintain, so they can fixed both issues in one pass.
- IDE should inspect the jars downloaded by Maven and Gradle, and report the 
use of deprecated for removal APIs, again to raise awareness,
  a warning directly on the tag dependency in the POM saying that this 
dependency (or one of it's sub-dependencies) is using deprecated for removal 
APIs will help a lot. 

Rémi

Re: EG meeting, 2020-10-07

[Remi Forax]

RestrictedField ?

https://bugs.openjdk.java.net/browse/JDK-8254022

Rémi 

- Mail original -
> De: "daniel smith" 
> À: "valhalla-spec-experts" 
> Envoyé: Mercredi 7 Octobre 2020 16:28:55
> Objet: EG meeting, 2020-10-07

> The next EG Zoom meeting is today at 4pm UTC (9am PDT, 12pm EDT).
> 
> Recent threads to discuss:
> 
> - "Terminology update: primitive objects": I outline new terminology we're
> proposing to talk about Valhalla's values, classes, and types
> 
> May be a short meeting, depending on how long this topic takes.

Re: Terminology update: primitive objects

[Remi Forax]

I'm fine with that change. 

For me, it's more where you put the emphasis 
- how it behaves on stack, it behavse like a primitive type ,the "pass by 
value" Dan is talking about 
- how it behaves on heap, it behaves by inlining itdelf in its container. 

So they are a primitive inlining class :) 

Rémi 

> De: "Brian Goetz" 
> À: "daniel smith" , "valhalla-spec-experts"
> 
> Envoyé: Lundi 5 Octobre 2020 21:22:15
> Objet: Re: Terminology update: primitive objects

> There's always been a duality between whether inline classes are "faster
> objects" or "user-programmable primitives." Until now, we're been erring on 
> the
> "faster objects" side of the line, but after some thought, we think that
> flipping the perspective will better frame what these new features are for.

> The obvious stumble you have to get over before this idea is appealing is "but
> these are not primitives." So we have to be explicit about that. But, once you
> buy that, I think these terms work much better.

> On 10/5/2020 3:18 PM, Dan Smith wrote:

>> We've been struggling with some uncomfortable rough edges of the "inline
>> class"/"inline type" terminology for awhile. After multiple rounds of
>> bikeshedding, here's an alternative that the Oracle team feels pretty good
>> about:

>> - A *primitive object* is a new kind of object that lacks identity. It has
>> special behavior with respect to equality, synchronization, and related
>> operations. *Identity object* describes all other objects (including arrays).
>> (The new objects are "primitive" in the sense that they are lighter-weight 
>> and
>> represent simple, immutable values.)

>> - A *primitive class* (formerly *inline class*) is a special class whose
>> instances are primitive objects. Primitive classes are always concrete and
>> final, and their declarations are subject to various restrictions. A class 
>> that
>> is not primitive is either an *identity class* or an *abstract class* (or
>> Object, if we don't end up making it abstract).

>> - A *primitive value type* (formerly *inline type*) is a type whose values 
>> are
>> primitive objects—the objects themselves, not *references* to the objects. 
>> Each
>> primitive class has a primitive value type, typically denoted by the class
>> name.

>> - A *primitive reference type* is a type whose values are references to
>> primitive objects, or null. Each primitive class has a primitive reference
>> type, typically denoted as ClassName.ref.

>> - The general term *primitive type* refers to either a primitive value type 
>> or a
>> primitive reference type. The general term *reference type* continues to 
>> mean a
>> type whose values are reference to objects (of unspecified kind), or null.

>> - In the Java language, the existing primitive values will become primitive
>> objects, with java.lang.Integer, etc., acting as their primitive classes. 
>> When
>> needed, *built-in primitive value type*, etc., can be used to refer to their
>> types. In the JVM, something like *primitive object type* might be 
>> appropriate
>> to distinguish between primitive objects and the built-in numerics.

>> The type terminology leans on intuitions about "pass by value" and "pass by
>> reference". Some languages pass *variables* by value or reference, but in 
>> Java
>> the, er, primitive, is passing *objects* by value or by reference. Similar
>> properties apply in both contexts.

>> ---

>> This is a brief sketch, just enough to define the terms. Future documents,
>> including project overviews, JEPs, and specs, will illustrate use of the 
>> terms
>> in the broader context of the language and VM.

Re: no good default issue

[Remi Forax]

- Mail original -
> De: "Brian Goetz" 
> À: "John Rose" 
> Cc: "valhalla-spec-experts" 
> Envoyé: Samedi 1 Août 2020 16:41:11
> Objet: Re: no good default issue

>> Personally, I view such tactics as possible but expensive,
>> and would like to try to get by without JVM support for
>> them, to start with.  The JVM engineering teams are already
>> overworked for Valhalla.
>>
> 
> And, more than that.  The technique of "ask a no-arg ctor" (or some
> other stateless source) only works for the classes in Bucket 2, which,
> as Dan and Kevin have pointed out, are actually the smaller and far less
> interesting group than Bucket 3, where there is simply no good default,
> and anything returned by the no-arg ctor would just be a different
> flavor of ad-hoc null.
> 
> One example of such a class is:
> 
>     inline record Name(String first, String last) { }
> 
> What would be the default?  Two empty strings?  Strings containing
> ""?  Are those really better than (null, null)?
> 
> Kevin has argued that Date equally belongs in this bucket.  Sure, we can
> interpret 1970 as the default date, but in the situations where someone
> uses an uninitialized date, what percentage of the time are they
> satisfied with 1970 as the result?  I don't think it's going out on a
> limb to say "not that often."  And if the date is used as an input to,
> say, an interest calculation, that's pretty bad.
> 
> The lesson of the most recent discussion is that "let the user pick the
> default" is not really better, and often worse, than "let zero be the
> default."  Really, this is not about defaults, but about detecting use
> of uninitialized data.

It depends if throwing an exception in the default constructor is allowed or 
not ?
i.e. if an exception is a valid state for a default value, like an exception is 
a valid value when evaluating a condy/ldc.

> 
> Refs don't have this problem, because when you try to do anything
> nontrivial with a null (other than move it around or compare it to
> another ref), the VM tells you clearly that you tried to use
> uninitialized data.
> 
> By the NNN Directive, winning looks like, to the extent we do anything
> here, aligning the behavior of dereferencing an uninitialized NGD value
> with that of dereferencing a null.

Rémi

Re: Revisiting default values

[Remi Forax]

So the default value may a valid value or may be an invalid value,
if it's an invalid value it should be the author of the class that say that 
because in Java we prefer declaration site to use site.

One way is to try to teach the VM how to do the conversions, i want to explore 
another way where we try to solve that issue at the language level, to avoid to 
have a more complex VM.

A default value which is invalid should behave like null, i.e. calling any 
methods on the default value should result in an exception.
Doing that at the language level means adding a check before calling any 
instance methods and before accessing any instance fields.

So there are two parts to solve,
1/ how to specify the check, is it just this == Inline.default or is it 
whatever the user want (or something in the middle, like a field check)
2/ how to execute that check when accessing a field or a method ?

Let explore the solution that offers the maximum freedom for the author of the 
inline class, i.e. for 1/, the check is user defined.
For that we can introduce a new kind of initializer, like the static block, 
let's call it the invariant block
  inline class Foo {
private final Object o;

invariant {
  if (o == null) {
throw new InvalidFooException();
  }
}
  }
this invariant block is translated into a method (that has the name  
see later why) and is called each time a method or a field is accessed.

For 2/, we can either change the spec of the VM so the invariant block is 
called automatically by the VM or we can use invokedynamic.
invokedynamic has the advantage of not requiring more VM support at the expanse 
of the bootstrap issue. 

The main issue with invokedynamic is that it's not a backward compatible change 
because it requires to change the call sites.
So we can lessen the requirement like this, requiring only the call to 
 when accessing an instance method because
we suppose that people will not be foolish enough to declare the fields public,
In that case, there is no need for using invokedynamic because a call to the 
invariant method can be inserted by the compiler at the beginning of any 
instance method.
This solution also has the advantage of lowering the cost at runtime compared 
to using invokedynamic.

In term of performance, i believe the language spec should say that the 
invariant block has to be idempotent.
Because in that case, the VM is free to not execute several calls to the 
 method once one is executed on a specific instance
(like the JITs do nullchecks collapsing currently).

To summarize, i believe we should allow more value based classes to be 
retrofitted as inline class by adding the concept of invariant block to the 
Java language spec.
An invariant block being a simple idempotent method called at the beginning of 
every instance methods.  

Rémi

- Mail original -
> De: "daniel smith" 
> À: "valhalla-spec-experts" 
> Envoyé: Vendredi 10 Juillet 2020 20:23:25
> Objet: Revisiting default values

> Brian pointed out that my list of candidate inline classes in the Identity
> Warnings JEP (JDK-8249100) includes a number of classes that, despite being
> "value-based classes" and disavowing their identity, might not end up as 
> inline
> classes. The problem? Default values.
> 
> This might be a good time to revisit the open design issues surrounding 
> default
> values and see if we can make some progress.
> 
> Background/status quo: every inline class has a default instance, which 
> provides
> the initial value of fields and array components that have the inline type
> (e.g., in 'new Point[10]'). It's also the prototype instance used to create 
> all
> other instances (start with 'vdefault', then apply 'withfield' as needed). The
> default value is, by fiat, the class instance produced by setting all fields 
> to
> *their* default values. Often, but not always, this means field/array
> initialization amounts to setting all the bits to 0. Importantly, no user code
> is involved in creating a default instance.
> 
> Real code is always useful for grounding design discussions, so let's start
> there. Among the classes I listed as inline class candidates, we can put them
> in three buckets:
> 
> Bucket #1: Have a reasonable default, as declared.
> - wrapper classes (the primitive zeros)
> - Optional & friends (empty)
> - From java.time: Instant (start of 1970-01-01), LocalTime (midnight), 
> Duration
> (0s), Period (0d), Year (1 BC, if that's acceptable)
> 
> Bucket #2: Could have a reasonable default after re-interpreting fields.
> - From java.time: LocalDate, YearMonth, MonthDay, LocalDateTime, 
> ZonedDateTime,
> OffsetTime, OffsetDateTime, ZoneOffset, ZoneRegion, MinguoDate, HijrahDate,
> JapaneseDate, ThaiBuddhistDate (months and days should be nonzero; null
> Strings, ZoneIds, HijrahChronologies, and JapaneseEras require special
> handling)
> - ListN, SetN, MapN (null array interpreted as empty)
> 
> Bucket #3: No good default.
> - Runtime.Version 

Re: Evolving the wrapper classes

[Remi Forax]

> De: "Brian Goetz" 
> À: "Tobi Ajila" 
> Cc: "valhalla-spec-experts" ,
> "valhalla-spec-experts" 
> Envoyé: Vendredi 19 Juin 2020 20:18:09
> Objet: Re: Evolving the wrapper classes

> Zooming out, what we've been trying to do is shake out the places where the 
> JVM
> treats primitives and references differently, and aligning them, so that we 
> are
> able to broaden the approach of "generics erase T to Object" to include 
> inlines
> and primitives. The war cry might be:

> Object is the new Any

> L-World does much of this for inlines, but we don't want to leave primitives 
> out
> in in the cold in the programming model; being able to get good behavior for
> Foo but not the same for Foo would be a missed opportunity to
> provide a uniform programming model. Much of this is either handled by 
> existing
> L-World behavior (e.g., behavior of ==), but this seam is one that needs to be
> covered. We can cover some in the static compiler (conversions between I and
> Qint) but when it comes to arrays, the invariance of arrays would expose our
> tricks, and we'd have to have awful restrictions like "you can't use arrays in
> generics."

> Note that [I and [QInteger$val have the exact same layout, so it is really a
> matter of treating the two type names as referring to the same underlying
> runtime type.
yes, but at the same time descriptor are matched by name and you need to have 
the proper descriptor when overriding/implementing a method, 
so the strategy of blindly replacing every I by QInteger$val; doesn't really 
work. 

Usually the solution is to use bridges but bridges only work with subtyping 
relationship not equivalence relationship (because you can travel in both 
direction). 
I believe we need to bring the forward/bridge-o-matic at the same time we 
retrofit primitive to inline. 

Rémi 

> On 6/19/2020 1:07 PM, Tobi Ajila wrote:

>>> Because arrays have identity (not to mention potentially large copying 
>>> costs),
>>> there is simply no reasonable conversion we can define; any "conversion" 
>>> would
>>> involve copying all the data, changing identity, or both. Just as with the
>>> array subtyping requirements (Point[] <: Point.ref[] <: Object[]), these are
>> > things only the VM can do for us.

>> I suspected that this was likely due to the large cost of converting between
>> `[I` and `[java/lang/Integer$val`. However, I am still a little unclear as to
>> what the motivation is for this. Is this solely for specialized generics?

>> In Dan's examples with `I` and `java/lang/Integer$val`, the only places where
>> conversions are needed are when primitives are used as type parameters or to
>> call instance methods on them, both of which can already be done with 
>> primitive
>> arrays. So in the LW3 - LW20 timeframe would we have any need for these
>> conversions? If so, could you provide some examples?

>> In the case of specialized generics, is the intention that `[I` (and I 
>> suppose
>> `I` as well) will appear in generic code?

Re: null checks vs. class resolution, and translation strategy for casts

[Remi Forax]

- Mail original -
> De: "Frederic Parain" 
> À: "valhalla-spec-experts" 
> Envoyé: Lundi 13 Avril 2020 15:24:18
> Objet: Re: null checks vs. class resolution, and translation strategy for 
> casts

>> On Apr 8, 2020, at 14:43, John Rose  wrote:
>> 
>> I have a proposal for a translation strategy:
>> 
>> 1. Translate casts to inline classes differently from “classic”
>> casts.  Add an extra step of null hostility.  For very low-level
>> reasons, I suggest using “ldc X” followed by Class::cast.
>> 
>> Generally speaking, it’s a reasonable move to use reflective
>> API points (like Class::cast) on constant metadata (like X.class)
>> to implement language semantics.
> 
> 
> There’s an alternative way to implement this:
> 
> Casts to inline classes C can be translated to
>#ldc C
>#checkcast C
> 
> with this new definition of checkast:
> 
> "If objectref is null then:
>   - if type C has not been loaded yet, the operand stack is unchanged,
>   - if type C has already been loaded:
>   - if type C is not an inline type, the operand stack is unchanged
>   - otherwise the checkcast instruction throws a ClassCastException

so it's more:
   #ldc C
   #checkcast 0<---

> 
> Otherwise, the named class, array, or interface type is resolved (§5.4.3.1). 
> If
> objectref can be cast to the resolved class, array, or interface type, the
> operand stack is unchanged; otherwise, the checkcast instruction throws a
> ClassCastException."
> 
> 
> This new definition doesn’t change the behavior of checkcast for old class
> files,
> and doesn’t change the behavior nor the translation strategy for casts to
> non-inline types.
> 
> In new class files, javac will use the ldc/checkcast sequence whenever a cast 
> to
> an
> inline type is required. Note that in many cases, type C would have already be
> loaded
> before ldc is executed (by pre-loading or eager loading).
> 
> With migrated types, an old class file can still have a standalone checkcast
> (without ldc)
> referencing a type which is now an inline type, causing the null reference to
> pass the checkcast
> successfully. This is not a new issue. The same situation can be created by
> reading a field
> declared as ‘LC;’, getfield would simply read the field (only possible value 
> is
> null) and push
> it on the stack without checking if C is an inline type or not. This ’null’
> reference to an
> invalid type can only be used by code that have the wrong information about 
> type
> C, and this
> is in fact the only possible value for this type (any attempt to create a real
> instance of
> ‘LC;’ would fail). In order to use this reference as a legitimate reference to
> the real
> type ‘QC;’, another checkcast, using the proper sequence above, would be
> required and would
> throw an exception.
> 
> Ill formed or malicious class files could be aware that C is an inline type, 
> but
> use a
> single checkcast instruction (without preceding ldc) anyway. This is part of a
> bigger
> problem that has not been discussed yet: L/Q consistency inside a class file.
> the SoV
> document stipules that the value projection only exists in the Q-form and the
> reference
> projection only exists in the L-form. As of today, there’s no verification of
> such kind
> performed on class files. Nothing prevent a class file from declaring a field 
> of
> type
> ‘LC;’ and another of type ‘QC;’, and the same remark applies to method
> arguments.
> 
> 
> Going back to the modified specification of checkcast, new tests are easy to
> implement
> in the interpreter, and can easily be optimized by JIT compilers (most types
> would be
> loaded at compilation time), and there’s no bootstrapping issues.
>

How it is better than a new opcode "unbox" with exactly the same semantics ?
 

> Fred

Rémi

Re: null checks vs. class resolution, and translation strategy for casts

[Remi Forax]

I don't fully understand why not using checkcast because from the user POV the 
message will be better than just NPE.

Rémi

- Mail original -
> De: "John Rose" 
> À: "Brian Goetz" 
> Cc: "valhalla-spec-experts" 
> Envoyé: Jeudi 9 Avril 2020 22:16:06
> Objet: Re: null checks vs. class resolution, and translation strategy for 
> casts

> On Apr 9, 2020, at 1:03 PM, Brian Goetz  wrote:
>> 
>> 
>>> I have a proposal for a translation strategy:
>> 
>> Casts to inline classes from their reference projections will be frequent.
>> Because the reference projection is sealed to permit only the value
>> projection, a cast is morally equivalent to a null check. We want to preserve
>> this performance model, because otherwise we're reinventing boxing.
>> 
>> Going through `ldc X.class / invokevirtual Class.cast` will surely be slow in
>> the interpreter, but also risks being slow elsewhere (as do many of the other
>> options.)
>> 
>> So let me add to your list: is it time for a `checknonnull` bytecode, which
>> throws NPE if null, or some other more flexible checking bytecode?
>> (Alternatively, if we're saving bytecodes: `invokevirtual
>> Object.`), where  is a fake method that always links 
>> to a
>> no-op, but invokevirtual NPEs on a null receiver.)
> 
> Um, this feels a lot like a premature optimization.  Let’s not add
> `checknonnull` intrinsics to the interpreter (the very most
> expensive way to do it) until we have tried the other alternatives
> (Objects.requireNonNull, etc.) and have proven that the costs
> are noticeable.  And a spec EG is not the place to evaluate such
> questions; it has to be demonstrated in a prototype.
> 
> I see now why you are angling for verifier rules that know about
> sealing relations.  I think that also is premature optimizations.
> Actually, verifier rules (not interpreter bytecodes) are the most
> costly way to get anything done.
> 
> Sorry to be a party pooper here, but that’s how it looks right now.
> 
> — John

Re: Updated SoV documents

[Remi Forax]

> De: "Brian Goetz" 
> À: "valhalla-spec-experts" 
> Envoyé: Vendredi 27 Mars 2020 20:59:38
> Objet: Updated SoV documents

> I've updated the SoV documents, including the new sections on VM model and
> translation:

> [ http://cr.openjdk.java.net/~briangoetz/valhalla/sov/01-background.html |
> http://cr.openjdk.java.net/~briangoetz/valhalla/sov/01-background.html ]

Inline classes: 
"cannot be layout-polymorphic", given they can be parametrized and the type 
argument being itself an inline type, they can. 
"or multiple return." -> multiple return value ? 

regards, 
Rémi 

java.lang.Record vs InlineObject/IdentityObject

[Remi Forax]

Hi,
currently a record can only be an IdentityObject but at some point, we want to 
introduce an inline record ?

Now the question: What the subtyping relation between java.lang.Record and 
InlineObject/IdentityObject ?

An identity record (resp. an inline record) should be a subtype of both 
java.lang.Record and IdentityObject (resp InlineObject).
Given that java.lang.Record has to be an abstract class because you can not 
re-abstract an interface method.
So both IdentityObject and InlineObject are interface and java.lang.Record 
inherits from Object.

Am i right ?
Or do we really care about re-abstracting the methods of j.l.Record ?

regards,
Rémi

Re: for review: 8236522: "always atomic" modifier for inline classes to enforce atomicity

[Remi Forax]

[Moving to valhalla-spec-experts]

- Mail original -
> De: "John Rose" 
> À: "Tobias Hartmann" 
> Cc: "valhalla-dev" 
> Envoyé: Vendredi 21 Février 2020 11:23:14
> Objet: Re: for review: 8236522: "always atomic" modifier for inline classes 
> to enforce atomicity

> I’ve come back around to this feature, after (SMH) realizing
> it should be a marker interface (java.lang.NonTearable) instead
> of a new modifier and access flag.  Thanks, Brian…

In my opinion, using an annotation here cross the rubicon,
user available annotations are not supposed to change the semantics of a 
language construct, they are metadata.

Do you really want the memory model to make reference of an annotation ?
Or worst, do you think that you can avoid to change the JMM to describe the 
effects of always atomic inline classes ?

What is the reason to change/move the line here ?
Are you envisioning other annotations that can change the semantics ?

Why can not be a keyword at source level and an attribute at classfile level ?

Rémi

> 
> Also, I’ve re-integrated my changes on top of the signature
> changes and field layout changes.
> 
> Please have another look:
> 
> http://cr.openjdk.java.net/~jrose/values/atomic-8236522
> 
> (Old webrev is atomic-8236522.1)
> 
> — John
> 
> P.S. There’s a slightly over-engineered string matcher included.
> I intend to use it for other purposes as well, when there are
> ad hoc class lists to assign special properties to, such as
> “please track locking events” and so on.  The code in
> CompilerOracle was not repurposable, sadly.
> 
>> On Feb 12, 2020, at 7:36 AM, Tobias Hartmann  
>> wrote:
>> 
>> Hi John,
>> 
>> I've just sent the RFR for 8238780 which will fix these issues. I found one
>> additional place where a
>> StoreStore barrier was missing.
>> 
>> Thanks,
>> Tobias
>> 
>> On 22.01.20 21:20, John Rose wrote:
>>> On Jan 20, 2020, at 6:54 AM, Tobias Hartmann >> > wrote:
 
> 
> How did it reproduce for you? It would be nice to have a regression test.
 
 Okay, -XX:+StressLCM did the trick :)
>>> 
>>> Thanks for taking it out for a spin.
>>> 
>>> The jtreg test ValueTearing.java failed reliably (P ~ 50%)
>>> on my MacBook.  Debugging that led me to add that S-S barrier.
>>> 
> >> — John

Re: Locking on primitive boxes

[Remi Forax]

> De: "Brian Goetz" 
> À: "valhalla-spec-experts" 
> Envoyé: Mercredi 19 Février 2020 17:45:38
> Objet: Locking on primitive boxes

> One of the main impediments to migrating the primitive wrappers to be the
> reference projection of an inline class is locking -- there may be code out
> there that locks on instances of primitive wrappers. Let's look at how we 
> might
> wean users off, and what the compatibility constraints really are.

> First, note that all the wrappers have had static `valueOf()` factories, and 
> the
> corresponding constructors have been deprecated since Java 9. It would be
> reasonable to deprecate these for removal at any time. (These would not
> actually be removed, but instead made private.)
You can make them synthetic instead of private, so there will not be available 
at compile time but still exist at runtime. 

> Having removed the public constructors, now the only way to get your hands on 
> an
> Integer is through some mechanism like `valueOf()`. I claim now that:

> Under such a scheme, any code outside of java.lang that synchronizes on a
> wrapper is
> inherently broken

> That is, it is synchronizing on an object whose locking protocol you are not
> party to, which is not guaranteed non-aliased, and which may in fact be
> permanently locked by some other thread. Therefore, it is a forseeable outcome
> for such a sync attempt to block forever.

> This was the intent behind [ https://openjdk.java.net/jeps/169 |
> https://openjdk.java.net/jeps/169 ] , though it wasn't obvious from the title
> -- the ability to mark certain objects as permanently locked.

> So, concrete proposal:

> - Move forward with the deprecation-for-removal of the constructors;

> - Add hortatory notes in the spec for `valueOf()` that locking on objects 
> whose
> provenance you do not control is a risky business, and may well never 
> complete;

> - Poison instances of primitive wrappers by marking their headers at creation
> with a bit pattern that will send locking through the slow path, where it can
> see that these are poisoned objects and cause lock attempts to never complete
> (or throw)

> - Provide a JDK-scoped option to turn off the above behavior
> (-Xx:AllowStupidBrokenLockingOnPrimitiveWrappers)

> I don't see anything stopping us from doing any of these immediately.
I still think it's a bit early, i would prefer to have reified generics before 
deciding how to retrofit primitive wrapper, i.e. if we can still have two 
different behavior, box and value, from the same class. 

That said, i agree with what is proposed here (with synthetic instead of 
private), there is no point to continue to allow people to shoot themselves in 
the foot with synchonizing on wrapper types. 

Rémi 

Re: Superclasses for inline classes

[Remi Forax]

Hi all,
sorry, was not available for the meeting today (i'm officially on vacation).

I prefer (2) with a restriction like (1).

I don't think users should be able to declare this kind of abstract class 
because you can not evolve them easily. I'm worried that if they become a tool 
available, people will over-use them (they are like a kind of abstract class 
with super-power, as a user it's like a shiny cookie), so in the end we will 
have to introduce inheritance of inline classes.
This rule out (3).

What (4) try to do is too clever IMO, a garbage class like 
java.util.Collections (with an 's' at the end) validate all the conditions but 
should not have an abstract constructor.
It's too easy to create such kind of abstract class without knowing it.

So for me, it's (2) + (1), at language level, a constructor can be declared 
abstract but it's restricted to class inside java.lang or java.base (enforced 
by the compiler and the VM).

Rémi

- Mail original -
> De: "daniel smith" 
> À: "valhalla-spec-experts" 
> Envoyé: Mercredi 12 Février 2020 00:54:10
> Objet: Re: Superclasses for inline classes

> So I think the JVM story is fairly settled; I've spun it off into another
> thread, with a proposed spec.
> 
> The language story is still uncertain. Here are four alternative designs, all 
> of
> which are, I think, reasonable approaches to consider. Some discussion about
> trade-offs is at the end.
> 
> -
> 
> Alternative 1: No language support
> 
> The language requires all inline classes to extend Object (or perhaps 
> prohibits
> the 'extends' clause). Abstract  methods are available only as a 
> compiler
> tool.
> 
> Some special exceptions are necessary:
> 
> - Somehow, class files for Object and Number must be generated with abstract
>  methods
> 
> - Integer, Double, etc., are inline classes but are allowed to extend Number
> 
> -
> 
> Alternative 2: Abstract constructors in the language
> 
> abstract class C {
>public abstract C();
> }
> 
> Like a method, a constructor can be declared 'abstract' and omit a body.
> (Bikeshed: maybe the 'abstract' keyword is spelled differently, or left off
> entirely.)
> 
> These declarations can align closely with the JVM's rules for  methods:
> - The superclass must also have an abstract constructor
> - No instance initializers or instance field initializers are allowed
> - It's okay to overload the constructor, but 'abstract' only works on a no-arg
> constructor
> - (Perhaps) the class doesn't have to be abstract
> - It's allowed (as a no-op) to invoke the constructor (new Object() or 
> super())
> 
> We'll need one further restriction that isn't checked by the JVM and isn't as
> principled: no instance fields are allowed, even if they're
> default-initialized. Otherwise, inline classes will have to search for private
> fields to decide if extension is legal or not, breaking encapsulation.
> 
> The abstract-ness of the constructor is part of the API—appears in javadoc,
> changing it is incompatible. Having an empty or default constructor isn't the
> same as having an abstract constructor.
> 
> Inline classes can only extend classes with abstract constructors (and maybe 
> no
> 'synchronized' instance methods).
> 
> -
> 
> Alternative 3: Inline-friendly property in the language
> 
> inlineable abstract class C {
> }
> 
> The 'inlineable' property (bikeshed: how to spell this?) enforces some
> constraints and authorizes children to be inline classes.
> 
> Specific constraints:
> - Can't have instance fields
> - Can't declare a constructor or instance initializer
> - Must extend an inlineable class
> - Must be an abstract class or Object (maybe?)
> - Must not have synchronized methods (probably?)
> 
> An annotation spelling is an option, too, although that crosses a 
> line—there's a
> precedent for annotations that prompt compiler errors, but not for annotations
> that influence bytecode output.
> 
> An inlineable class's bytecode has an abstract  method.
> 
> The 'inlineable' property is part of the API—appears in javadoc, changing it 
> is
> incompatible.
> 
> Inline classes can only extend inlineable classes.
> 
> -
> 
> Alternative 4: Infer no constructor
> 
> Typically, for each class that lacks a constructor declaration, a default
> constructor is provided that does some simple initialization. But in the
> following circumstances, we claim there is no constructor at all:
> 
> - Class doesn't declare a constructor or instance initializer
> - Class doesn't declare fields
> - Superclass has no constructor
> - Class is abstract or Object (maybe?)
> 
> Again, blank private fields may not *need* initialization, but inline 
> subclasses
> need to know that they exist, and the best way to communicate that is through
> the constructor.
> 
> 'new Object()' (and perhaps 'new Foo()' if we drop the abstract class
> requirement) doesn't reference any constructor at all. In that case, a fresh
> instance is allocated without any code execution.
> 
> 

Re: atomicity for value types

[Remi Forax]

In the context of Java, we are already using the term 'atomic', in 
AtomicInteger, AtomicLong, etc, 
and in that case the semantics is volatile + atomic operations (at least CAS), 
so i think using atomic or any keyword derived from it will not help to our 
users to understand the semantics for an inline class. 

As Doug said, for a VarHandle, the semantics we want is called opaque, so 
"opaque" is ok for me. 

Otherwise, the idea is that you can not cut the loads/stores, so "indivisible" 
is also ok. 

Rémi 

> De: "John Rose" 
> À: "valhalla-spec-experts" 
> Envoyé: Mercredi 15 Janvier 2020 01:40:34
> Objet: Re: atomicity for value types

> On Dec 18, 2019, at 5:46 PM, John Rose < [ mailto:john.r.r...@oracle.com |
> john.r.r...@oracle.com ] > wrote:

>> - Define a contextual keyword “alwaysatomic" (working title 
>> “__AlwaysAtomic”).

> I just referred more carefully to the draft JEP on keyword
> management [ https://openjdk.java.net/jeps/8223002 |
> https://openjdk.java.net/jeps/8223002 ] and
> realize that it guides us toward a so-called “hyphenated
> contextual keyword” in preference to a “unitary contextual
> keyword”. That is, “always-atomic” is more in keeping with
> that document than “alwaysatomic”.

> I do think this looks more jarring than the unitary keyword:

> always-atomic inline class Cursor { … }

> But, that’s only because it’s an early hyphenated keyword,
> which nobody is eye-trained on yet. If we believe that
> hyphenated keywords are the wave of the future, as I do,
> then we should embrace it, rather than the old-school
> unitary token “alwaysatomic”.

> In the prototype I’m using a temporary (unitary contextual) keyword-ato
> __AlwaysAtomic, plus a temporary annotation @__alwaysatomic__.

> In the JVMs the draft of the corresponding modifier bit looks like this:

> ACC_ALWAYSATOMIC 0x0040 Instances of this inline type are never torn.

Re: Superclasses for inline classes

[Remi Forax]

It occurs to me that we may have jump into the conclusion that we should use 
inheritance/implementation a little to fast.
At least, i think it worth exploring another complementary option.

In Java, you have several ways to have subtyping,
saying a FooInline is a subclass FooInline.ref is one way,
but using wildcard is another one, if Foo can be a template classes with 
Foo the inline.

It will not solve the retrofitting of java.lang.Number so it's a complementary 
approach but it can solve the retrofitting of java.lang.Integer in a better way 
than to use subclasses.

Let say java.lang.Integer is a template class, but not a template class over a 
type like a specialized generics are but a template class over the inline bit.
In that case java.lang.Integer can have two specializations, one is an indirect 
type, the result of new Integer(8), and one is an inline type the result of 
Integer.valueOf(8).

The nice properties of this organization are:
- object.getClass() will return Integer.class for both specializations (it's 
not the same species but it's the same class).
- java.lang.Integer can still be final (because final means no subclass but not 
no subtype).

Rémi

Re: Lifting operations from inline classes to reference projections

[Remi Forax]

- Mail original -
> De: "Brian Goetz" 
> À: "Maurizio Cimadamore" , 
> "valhalla-spec-experts"
> 
> Envoyé: Lundi 16 Décembre 2019 15:27:59
> Objet: Re: Lifting operations from inline classes to reference projections

>>
>> I'm not too worried about having a V.ref with its own members - after
>> all, V.ref should be a language fiction (for the VM, the only real
>> thing is V). So, in my mental model, it will be javac, not the VM, to
>> do most of the lifting here. Am I wrong in my assumption?
> 
> The VM team has expressed a preference that these members not be
> duplicated, so we're exploring alternatives.  One alternative that has
> been suggested is: if X is a reference projection of V, then
> `invokeinterface X.m()`, be adjusted to look for members on V. (Some
> adjustments would be made to the resolution for the case of private
> implementations, where the access check would be relaxed on the
> implementation _class_ but not the _member_.  This sidesteps the problem
> with the obvious solution (interface methods on V.ref) having to do with
> non-public methods.)

I don't think that private implementation is a real issue.

Let say when have an inline type HashEntry,

public class HashMap {
  private static inline class HashEntry implements Map.Entry {
private K key;
private V value;

public K getKey() { return key; }  
...
  }
  ...
}

now if we introduce a public HashEntry.ref, we have
 
public class HashMap {
  private static inline class HashEntry implements Map.Entry, 
HashEntry.ref {
private K key;
private V value;

public K getKey() { return key; }  
...
  }

  public interface HashEntry.ref extends Map.Entry {
// empty
  }
  ...
}   

>From outside HashMap, HashEntry.ref.key is not valid, but 
>that's expected,
HashEntry.ref.getKey() is accessible because a ref has the same 
set of interfaces as the corresponding inline type and this is how public 
methods are getting exposed.

Rémi

IdentityObject and InlineObject

[Remi Forax]

Hi Brian,
really nice write-up.

I've not a lot to say apart about IdentityObject and InlineObject because I 
fully agree on the major points (and i've promised to not tak about == but wait 
and see instead).
As we briefly talk last Wednesday, i believe there is a less disruptive way to 
add IdentityObject in the language.

Let see if we can do better, there are three reasons to have IdentityObject 
(and InlineObject)
- documentation (support for javadoc)
- bound for generic type parameter
- class at runtime for instanceof

The main drawbacks of Brian's proposal are:
- IdentityObject needs to be injected dynamically by the VM.
- replacing occurrences of Object by IdentityObject (in IdentityHashMap by 
example) is not a binary backward compatible change.
- new Object() needs to be deprecated (for removal?) and replaced by 
IdentityObject.newIdentityObject.

The only way to solve the point 2 is to have a different view at compile time 
and at runtime.
For that,
- the compiler should see IdentityObject as a valid super type of every 
identity classes.
- the compiler erase IdentityObject to java.lang.Object (in particular, 
IdentityObject.newIdentityObject() return type is erased to Object).
- synchronized emit a warning if the the argument is not typed as an 
IdentityObject.

If IdentityObject is erased to Object,
- IdentityObject.java exists as support for documentation.
- it doesn't need to be injected dynamically by the VM anymore
- because IdentityObject is still present in the generic signature, it can be 
used as bound of generic type parameter
- replacing some occurrences of Object by IdentityObject is backward compatible
- IdentityObject can not be used in instanceof or cast, but the compiler can 
suggest to use if (!(o instanceof InlineObject)) instead.

I believe this solution is less disruptive and provide a better way to 
introduce IdentityObject and InlineObject in a backward compatible way.

regards,
Rémi

Re: Record classfile and runtime API

[Remi Forax]

sorry wrong list.

Rémi

- Mail original -
> De: "Remi Forax" 
> À: "valhalla-spec-experts" 
> Envoyé: Dimanche 1 Septembre 2019 17:27:13
> Objet: Record classfile and runtime API

> Hi all,
> i think the current generated classfile bytecodes can be improved.
> 
> For a record Point(int x, int y) { }
> 
> The classfile contains:
>  - a specific class attribute Record
>Record:
> int x;
>   descriptor: I
> int y;
>   descriptor: I
>which allows compiler separate compilation.
>  - two bootstrap methods
>- one for the invodynamic calls used inside toString/equals/hashCode
>  this bootstrap method takes 4 arguments
>#8 fr/umlv/record/Point
>#51 x;y
>#53 REF_getField fr/umlv/record/Point.x:I
>#54 REF_getField fr/umlv/record/Point.y:I
>- one for the constant dynamic used inside the pattern extractor/handler
>  this bootstrap method takes 3 arguments
>#8 fr/umlv/record/Point
>#53 REF_getField fr/umlv/record/Point.x:I
>#54 REF_getField fr/umlv/record/Point.y:I
> 
> so we have 3 different ways to represent exactly the same thing, a record 
> Point
> is composed of two component int x and int y, i believe we can do better.
> 
> The arguments of the bootstrap methods are redundant, you don't need to pass 
> the
> record class because the Lookup object contains the record class (you have the
> API in Lookup to create an intermediary lookup from a Lookup and a class so no
> need to duplicate that information). For the first boostrap methods, the
> parameter containing "x;y" is due to the fact that a constant method handle is
> opaque by default so you can not get the corresponding field name. First, if
> you have the lookup that have created that object, you can get back the name
> with a "reveal". But i think here, the problem is that the arguments are the
> constant method handles and not the constant decriptor (ConstantDec).
> 
> The java.lang.invoke API has currently the limitation that you can not use the
> ConstantDesc API to type values of the constant pool.
> 
> How can we do better ?
> We only need the attribute Record, all other informations can be retrieved at
> runtime from the attribute Record.
> 
> Now we have to talk about the reflection API, currently the method that 
> reflects
> the attribute Record is Class.getRecordAccessors() that returns an array of
> Method corresponding to the accessors and there are many things wrong with 
> this
> method.
> First, while returning the accessors may be a good idea, it should not be the
> sole API, we should have an API that reflects the data of the attribute Record
> itself. Then it returns Method so a live object that may have triggered the
> classloading of the types of each record component that may notexist at
> runtime, we have introduce the constable API, we should using it here. And the
> implementation has several issues, first it can throws a NoSuchMethodException
> which is a checked exception, then it uses getDeclaredMethod which returns the
> methods with the most precise return type which is not what you want here.
> Here is an example that show why it's buggy, le suppose i have an interface 
> and
> a record
>  interface I {
>default Object x() { return 0; }
>  }
>  record R(Object x) implements I { }
> now let say i change I without recompiling R
>  interface I {
>default String x() { return 0; }
>  }
> R.class.getRecordAccessors() will return the method x() that returns a String
> instead of the one that returns an Object.
> 
> So first, let's have a method getRecordComponentFields() in java.lang.Class 
> that
> returns an array of DirectMethodHandleDesc corresponding to method ref to the
> fields of a Record. It uses a constant desc, so no weird classloading will
> appear here. And it's better than an array of MethodHandle because a
> DirectMethodHandleDesc is not an opaque object.
> 
> With that, we can unify the bootstrap methods to have only one bootstrap 
> method
> with zero argument, the one that return a pattern/extractor can be implemented
> into another class but can be queried from the same bsm that the one that is
> needed for toString/equals/hashCode.
> 
> BTW, why do we need an extractor for a record ? we should not need one because
> we can inside the algorithm that create the pattern tree have a if
> class.isRecord() ? (the same way the serailization is specilized for enums by
> example).
> 
> On problem with this approach is that while it reduces the bytecode size, it
> will also resolve the constant method handles to the accessors several times
> but i think it's better to have this kind of cache inside java.lang.Class than
> to introduce a constant dynamic that will return a constant list of constant
> method handles instead. Introducing something in the constant pool for caching
> reason seems not a good idea if in the future we want to change the way it
> works.
> 
> Rémi

Record classfile and runtime API

[Remi Forax]

Hi all,
i think the current generated classfile bytecodes can be improved.

For a record Point(int x, int y) { }

The classfile contains:
  - a specific class attribute Record
Record:
 int x;
   descriptor: I
 int y;
   descriptor: I
which allows compiler separate compilation.
  - two bootstrap methods
- one for the invodynamic calls used inside toString/equals/hashCode
  this bootstrap method takes 4 arguments
#8 fr/umlv/record/Point
#51 x;y
#53 REF_getField fr/umlv/record/Point.x:I
#54 REF_getField fr/umlv/record/Point.y:I
- one for the constant dynamic used inside the pattern extractor/handler
  this bootstrap method takes 3 arguments
#8 fr/umlv/record/Point
#53 REF_getField fr/umlv/record/Point.x:I
#54 REF_getField fr/umlv/record/Point.y:I

so we have 3 different ways to represent exactly the same thing, a record Point 
is composed of two component int x and int y, i believe we can do better.

The arguments of the bootstrap methods are redundant, you don't need to pass 
the record class because the Lookup object contains the record class (you have 
the API in Lookup to create an intermediary lookup from a Lookup and a class so 
no need to duplicate that information). For the first boostrap methods, the 
parameter containing "x;y" is due to the fact that a constant method handle is 
opaque by default so you can not get the corresponding field name. First, if 
you have the lookup that have created that object, you can get back the name 
with a "reveal". But i think here, the problem is that the arguments are the 
constant method handles and not the constant decriptor (ConstantDec).

The java.lang.invoke API has currently the limitation that you can not use the 
ConstantDesc API to type values of the constant pool.

How can we do better ?
We only need the attribute Record, all other informations can be retrieved at 
runtime from the attribute Record.

Now we have to talk about the reflection API, currently the method that 
reflects the attribute Record is Class.getRecordAccessors() that returns an 
array of Method corresponding to the accessors and there are many things wrong 
with this method.
First, while returning the accessors may be a good idea, it should not be the 
sole API, we should have an API that reflects the data of the attribute Record 
itself. Then it returns Method so a live object that may have triggered the 
classloading of the types of each record component that may notexist at 
runtime, we have introduce the constable API, we should using it here. And the 
implementation has several issues, first it can throws a NoSuchMethodException 
which is a checked exception, then it uses getDeclaredMethod which returns the 
methods with the most precise return type which is not what you want here.
Here is an example that show why it's buggy, le suppose i have an interface and 
a record
  interface I {
default Object x() { return 0; }
  }
  record R(Object x) implements I { }
now let say i change I without recompiling R
  interface I {
default String x() { return 0; }
  }
R.class.getRecordAccessors() will return the method x() that returns a String 
instead of the one that returns an Object.

So first, let's have a method getRecordComponentFields() in java.lang.Class 
that returns an array of DirectMethodHandleDesc corresponding to method ref to 
the fields of a Record. It uses a constant desc, so no weird classloading will 
appear here. And it's better than an array of MethodHandle because a 
DirectMethodHandleDesc is not an opaque object.

With that, we can unify the bootstrap methods to have only one bootstrap method 
with zero argument, the one that return a pattern/extractor can be implemented 
into another class but can be queried from the same bsm that the one that is 
needed for toString/equals/hashCode.

BTW, why do we need an extractor for a record ? we should not need one because 
we can inside the algorithm that create the pattern tree have a if 
class.isRecord() ? (the same way the serailization is specilized for enums by 
example).

On problem with this approach is that while it reduces the bytecode size, it 
will also resolve the constant method handles to the accessors several times 
but i think it's better to have this kind of cache inside java.lang.Class than 
to introduce a constant dynamic that will return a constant list of constant 
method handles instead. Introducing something in the constant pool for caching 
reason seems not a good idea if in the future we want to change the way it 
works.

Rémi

Re: Equality for values -- new analysis, same conclusion

[Remi Forax]

I think we should take a step back on that subject,
because you are all jumping to the conclusion too fast in my opinion.

Let starts by the beginning,
the question about supporting == on inline type should first be guided by what 
we should have decided if inline types were present from the inception of Java, 
it's the usual trick when you want to retcon a feature. 
If we had inline types from the beginning, i believe we will never had allowed 
== on Object, the root type of the hierarchy, but have a special method call 
that will only work on indirect type like in C#.
So Object== is a kind of liability, so we are not here to provide a nice 
semantics to Object== but to deprecate Object== and provide a backward 
compatible way to make inline types to work with old codes.

So first, we have to clearly convey that == should be deprecated apart on 
primitive type,
i propose
- to banned V== (compile error)
- to make Object==and T== emit a compiler warning explaining that the code 
should be changed
- add a method System.identityEquals(RefObject, RefObject) as replacement

Now, the second thing that disturb me is that no email of this thread, lists 
the two issues of the substitutibility test that make it unsuitable as an 
implementation of Object==.
- it's not compatible with the primitive == on float and double, by example,
  inline class InlineFloat {
float value;

public boolean equals(Object o) {
  if (!(o instanceof InlineFloat i)) {
return false;
  }
  return value == i.value;
}
  }
  has the stupid property of having == being true and equals() being false if 
value is NaN.

- it can be really slow 
1) Object== can be megamorphic
2) Object== can do a recursive call
  so it destroys the assumption that Object== is faster than equals.

Hopefully that we are not trying to make == to work on inline types, but only 
to make Object== to have a compatible semantics when one of the operand is an 
inline type. 

so the only choice we have is to return false is the left or the right operand 
is an inline type.

And yes, people will find it weird but that's why we are deprecating it after 
all.

Rémi

- Mail original -
> De: "Brian Goetz" 
> À: "valhalla-spec-experts" 
> Envoyé: Vendredi 9 Août 2019 17:46:19
> Objet: Equality for values -- new analysis, same conclusion

> Time to take another look at equality, now that we’ve simplified away the
> LFoo/QFoo distinction. This mail focuses on the language notion of equality
> (==) only.  Let’s start with the simplest case, the ==(V,V) operator.  There 
> is
> a range of possible interpretations:
> 
> - Not allowed; the compiler treats == as not applicable to operands of type V.
> (Note that since V <: Object, == may still be called upon to have an opinion
> about two Vs whose static types are Object or interface.)
> - Allowed, but always false.  (This appeals to a concept of “aggressive
> reboxing”, where a value is reboxed between every pair of byte codes.)
> - Weak substitutability.  This is where we make a “good faith” attempt to 
> treat
> equal points as equal, but there are cases (such as those where a value hides
> behind an Object/interface) where two otherwise equal objects might report not
> equal.  This would have to appeal to some notion of invisible boxing, where
> sometimes two boxes for the same value are not equal.
> - Substitutability.  This is where we extend == field wise over the fields of
> the object, potentially recursively.
> 
> As noted above, we don’t only have to define ==V, but ==Object when there may 
> be
> a value hiding behind the object. It might be acceptable, though clearly 
> weird,
> for two values that are ==V to not be ==Object when viewed as Objects. 
> However,
> the only way this might make sense to users is if this were appealing to a
> boxing conversion, and its hard to say there’s a boxing conversion from V to
> Object when V <: Object.  There is a gravitational force that says that if two
> values are V==, then they should still be == when viewed as Object or
> Comparable.
> 
> Let’s take a look at the use cases for Object==.
> 
> - Direct identity comparison.  This is used for objects that are known to be
> interned (such as interned strings or Enum constants), as well as algorithms
> that want to compare objects by identity. such as IdentityHashMap.  (When the
> operands are of generic (T) or dynamic (Object) type, the “Interned” case is
> less credible, but the other cases are still credible.)
> - As a fast path for deeper equality comparisons (a == b || a.equals(b)), 
> since
> the contract of equals() requires that == objects are equals().
> - In comparing references against null.
> - In comparing references against a known sentinel value, such as a value
> already observed, or a sentinel value provided by the user.
> 
> When generics are specialized, T== will specialize too, so when T specializes 
> to
> a value, we will get V==, and when T is erased, we will get 

Re: Collapsing the requirements

[Remi Forax]

> De: "Brian Goetz" 
> À: "Frederic Parain" 
> Cc: "valhalla-spec-experts" 
> Envoyé: Mardi 6 Août 2019 18:50:26
> Objet: Re: Collapsing the requirements

>> So, legal signatures will be:
>> - QV;
>> - LI;
>> and that’s it, right?

>> Q will continue to have its current semantic (flattenable, non-nullable,
>> triggers pre/eager-loading).
>> L will continue to have its legacy semantic (indirection, nullable, no new
>> loading rules)

> Correct. Nice and simple!

I believe 'Q' should be only mean preload and the fact that the class has a 
inline bit should imply flattenable and non-nullable. 
Yes, we consume one of these precious bit but at the same time, we nicely 
decouple the meaning of the descriptor from the meaning of the class itself. 

Rémi 

Re: Collapsing the requirements

[Remi Forax]

- Mail original -
> De: "Remi Forax" 
> À: "Brian Goetz" 
> Cc: "valhalla-spec-experts" 
> Envoyé: Samedi 3 Août 2019 19:48:01
> Objet: Re: Collapsing the requirements

> - Mail original -
>> De: "Brian Goetz" 
>> À: "valhalla-spec-experts" 
>> Envoyé: Samedi 3 Août 2019 18:37:56
>> Objet: Collapsing the requirements
> 
>> As Remi noted, we had some good discussions at JVMLS this week.  Combining 
>> that
>> with some discussions John and I have been having over the past few weeks, I
>> think the stars are aligning to enable us to dramatically slim down the
>> requirements.  The following threads have been in play for a while:
>> 
>> - John: I hate the LPoint/QPoint distinction
>> - Brian: I hate null-default types
>> - Remi: I hate the V? type
>> 
>> But the argument for each of these depended, in some way, on the others.  I
>> believe, with a few compromises, we can now prune them as a group, which 
>> would
>> bring us to a much lower energy state.
>> 
>> ## L^Q World — Goodbye `LV;`
>> 
>> We’ve taken it as a requirement that for a value type V, we have to support 
>> both
>> LV and QV, where LV is the null-adjunction of QV.  This has led to a lot of
>> complexity in the runtime, where we have to manage dual mirrors.
>> 
>> The main reason why we wanted LV was to support in-place migration.  (In
>> Q-world, LV was the box for QV, so it was natural for migration.)  But, as
>> we’ve worked our migration story, we’ve discovered we may not need LV for
>> migration. And if we don’t, we surely don’t need it for anything else;
>> worst-case, we can erase LV to `LValObject` or even `LObject` (or, if we’re
>> worried about erasure and overloading, to something like `LObject//V` using
>> John’s type-operator notation.)
>> 
>> Assuming we can restructure the migration story to not require LV to 
>> represent a
>> VM-generated “box" — which I believe we can, see below — we can drop the
>> requirement for LV.  An inline class V gives rise to a single type 
>> descriptor,
>> QV (or whatever we decide to call it; John may have plans here.)
>> 
>> ## Goodbye `V?`
>> 
>> The other reason we wanted LV was that it was the obvious representation for 
>> the
>> language type `V?` (V adjoined with null.)  Uses for `V?` include:
>> 
>> - Denoting non-flattened value fields;
>> - Denoting non-flattened value arrays;
>> - Denoting erased generics over values (`Foo`);
>> - Denoting the type that is the adjunction of null to V (V | Null), when we
>> really want to talk about nullability.
>> 
>> But, we can do all this without a `V?` type; for every V, there is already at
>> least one super type of V that includes `V|Null` — Object, and any interface
>> implemented by V.  If we arrange that every value type V has a super type V’,
>> not implemented by any other type — then the value set of this V’ is exactly
>> that of `V?`.  And we can use V’ to do all the things `V?` did with respect 
>> to
>> V — including sub typing.  The language doesn’t need the `?` type operator, 
>> it
>> just needs to ensure that V’ always exists.  Which turns out to be easy, and
>> also turns out to be essential to the migration story.
>> 
>>  Eclairs
>> 
>> We can formalize this by requiring that every value type have a companion
>> interface (or abstract class) supertype.  Define an envelope-class pair
>> (“eclair”) as a pair (V, I) such that:
>> 
>> - V is an inline class
>> - I is a sealed type
>> - I permits V (and only V)
>> - V <: I
>> 
>> (We can define eclairs for indirect classes, but they are less interesting —
>> because indirect classes already contain null.)
>> 
>> If every value type be a member of an eclair, we  can use V when we want the
>> flattenable, non-nullable, specializable type; and we use I when we want the
>> non-flattenable, nullable, erased “box”.  We don’t need to denote `V?`; we 
>> can
>> just use I, which is an ordinary, nominal type.
>> 
>> Note that the VM can optimize eclairs about as well as it could for LV; it 
>> knows
>> that I is the adjunction of null to V, so that all non-null values of I are
>> identity free and must be of type V.
>> 
>> What we lose relative to V? is access to fields; it was possible to do
>> `getfield` on a LV, but not on I.  If this is important (and maybe it’s not),
>> we can handle this in other ways.
>> 
>>  With suga

Re: Collapsing the requirements

[Remi Forax]

- Mail original -
> De: "Brian Goetz" 
> À: "valhalla-spec-experts" 
> Envoyé: Samedi 3 Août 2019 18:37:56
> Objet: Collapsing the requirements

> As Remi noted, we had some good discussions at JVMLS this week.  Combining 
> that
> with some discussions John and I have been having over the past few weeks, I
> think the stars are aligning to enable us to dramatically slim down the
> requirements.  The following threads have been in play for a while:
> 
> - John: I hate the LPoint/QPoint distinction
> - Brian: I hate null-default types
> - Remi: I hate the V? type
> 
> But the argument for each of these depended, in some way, on the others.  I
> believe, with a few compromises, we can now prune them as a group, which would
> bring us to a much lower energy state.
> 
> ## L^Q World — Goodbye `LV;`
> 
> We’ve taken it as a requirement that for a value type V, we have to support 
> both
> LV and QV, where LV is the null-adjunction of QV.  This has led to a lot of
> complexity in the runtime, where we have to manage dual mirrors.
> 
> The main reason why we wanted LV was to support in-place migration.  (In
> Q-world, LV was the box for QV, so it was natural for migration.)  But, as
> we’ve worked our migration story, we’ve discovered we may not need LV for
> migration. And if we don’t, we surely don’t need it for anything else;
> worst-case, we can erase LV to `LValObject` or even `LObject` (or, if we’re
> worried about erasure and overloading, to something like `LObject//V` using
> John’s type-operator notation.)
> 
> Assuming we can restructure the migration story to not require LV to 
> represent a
> VM-generated “box" — which I believe we can, see below — we can drop the
> requirement for LV.  An inline class V gives rise to a single type descriptor,
> QV (or whatever we decide to call it; John may have plans here.)
> 
> ## Goodbye `V?`
> 
> The other reason we wanted LV was that it was the obvious representation for 
> the
> language type `V?` (V adjoined with null.)  Uses for `V?` include:
> 
> - Denoting non-flattened value fields;
> - Denoting non-flattened value arrays;
> - Denoting erased generics over values (`Foo`);
> - Denoting the type that is the adjunction of null to V (V | Null), when we
> really want to talk about nullability.
> 
> But, we can do all this without a `V?` type; for every V, there is already at
> least one super type of V that includes `V|Null` — Object, and any interface
> implemented by V.  If we arrange that every value type V has a super type V’,
> not implemented by any other type — then the value set of this V’ is exactly
> that of `V?`.  And we can use V’ to do all the things `V?` did with respect to
> V — including sub typing.  The language doesn’t need the `?` type operator, it
> just needs to ensure that V’ always exists.  Which turns out to be easy, and
> also turns out to be essential to the migration story.
> 
>  Eclairs
> 
> We can formalize this by requiring that every value type have a companion
> interface (or abstract class) supertype.  Define an envelope-class pair
> (“eclair”) as a pair (V, I) such that:
> 
> - V is an inline class
> - I is a sealed type
> - I permits V (and only V)
> - V <: I
> 
> (We can define eclairs for indirect classes, but they are less interesting —
> because indirect classes already contain null.)
> 
> If every value type be a member of an eclair, we  can use V when we want the
> flattenable, non-nullable, specializable type; and we use I when we want the
> non-flattenable, nullable, erased “box”.  We don’t need to denote `V?`; we can
> just use I, which is an ordinary, nominal type.
> 
> Note that the VM can optimize eclairs about as well as it could for LV; it 
> knows
> that I is the adjunction of null to V, so that all non-null values of I are
> identity free and must be of type V.
> 
> What we lose relative to V? is access to fields; it was possible to do
> `getfield` on a LV, but not on I.  If this is important (and maybe it’s not),
> we can handle this in other ways.
> 
>  With sugar on top, please
> 
> We can provide syntax sugar (please, let’s not bike shed it now) so that an
> inline clause _automatically_ acquires a corresponding interface (if one is 
> not
> explicitly provided), onto which the public members (and type variables, and
> other super types) of C are lifted.  For sake of exposition, let’s say this is
> called `C.Box` — and is a legitimate inner class of C (which can be generated
> by the compiler as an ordinary classfile.)  We’ve been here before, and
> abandoned it because “Box” seemed misleading, but let’s call it that for now.
> And now it is a real nominal type, not a fake type.  In the simplest case,
> merely declaring an inline class could give rise to V.Box.
> 
> Now, the type formerly known as `V?` is an ordinary, nominal interface (or
> abstract class) type.  The user can say what they mean, and no magic is needed
> by either the language or the VM.  Goodbye `V?`.
> 
>  Boxing 

Toward LW10, the inline interface proposal

[Remi Forax]

Hi all,

We (Maurizio, John and i) have spent some time to play with another proposal of 
what LW10 can be based on the "confinent" idea i.e. an inine type can not 
appear in a public API and you need an interface to use it in erased generics.

So the idea is to have at Java language level a construct that desugar itself 
into an interface and a package-private inline type.

I will use the following syntax just to be able to explain the semantics and 
not as a committement on any syntax. Let say you can declare an "inline 
interface" (again wrong name) that will contains the impleemntation of the 
inline type and surface its API as an interface.

public inline interface Foo {
  private int value;

  private Foo(int value) {
this.value = value;
  }

  public Foo add(Foo foo) {
return new Foo(value + foo.value);
  }

  public static Foo of(int value) {
return new Foo(value);
  }
}

this code is desugared into Foo and Foo$val,
with Foo and Foo$val nestmates (but not inner/outer classes).

public sealed interface Foo permit Foo$val {
  public Foo add(Foo foo);

  public static Foo of(int value) {
return new Foo$val(value);
  }
}
/* package-private */ final inline class Foo$val implements Foo {
  private int value;

  /* package-private */ Foo(int value) {
this.value = value;
  }

  public Foo add(Foo foo) {
return new Foo(value + foo.value);
  }
}

So an inline interface contains only private or public members:
- private members are moved into the inline class
- public fields and public constructors are not allowed
- public instance methods are copied into the inline class, the signature is 
copied as abstract method into the interface 
- default method may stay in the interface (if supported)
- reference to the interface constructor (meta: that's why it's the wrong name) 
are changed to reference to the inline class constructor.
- Foo$val is denotable as Foo.val, but it can only appear in non public context 
(local variable, parameter type/return type of a private method, private field) 
and it can not be a type argument or a bound of a generic class/method.

With that, i think we may not need null-default inline class anymore.

For value based class, we have the issue that those are classes and not 
interfaces.
My hope here is than we can teach the VM that we can retrofit invokevirtual and 
invokeinterface to work on both classes and interfaces. It will also make the 
transformation from any class to an interface binary compatible which will be a 
nice tool if you want to grow a library.

For LW100, we will enable the support of public constructors wich will make 
Foo$val visible denotable. 

Obviously, it's just a proposal and i hope i'm not too far in my description of 
what we have discussed.

Rémi

Face to face meeting of yesterday

[Remi Forax]

Hi all, here is my takeaway of the valhalla face to face meeting we had 
yesterday.

On the VM side, the model can be simplified by having all classes being either 
a Q-type or a L-type but not both at the same time. This will simplify the JVMS 
and solve the naming issue i have with Dan because the descriptor side and the 
runtime side will be aligned. Obviously, this has to be prototyped.

Doing that move force us to reconsider the migration scenario for the value 
based class (Optional, LocalDateTime, etc). One possible solution to migrate 
those types is to introduce null default inline types aka inline types that 
have a special bit pattern to encode null. Again, this has to be prototyped (in 
the VM and javac too) because introducing those types may have an impact on the 
non nullable inline type performance.
Brian has confirmed that the migration of the value based classes is not 
necessarily part of LW10, so the prototyping is not a huge priority, at the 
same time, given that LW10 has to be compatible with LW100, the migration story 
has to be figure out before the release of LW10.

For LW10, we are currently not able to find a Java syntax to represent nullable 
inline type that interact with non-reified generics. The question mark notation 
seems to be the least worst compomise. Another possible syntax is John's 
Indirect class. Anyway, no proposal is a clear winner.

A new data is that we now have Panama replacement of the ByteBuffer API, the 
Vector API, Doug Lea's new HashMap and Remi's version of the carrier object of 
the Amber's pattern matching that have dependencies on valhalla and it may be 
wise to introduce an intermediary target between now and LW10 (LW5?) which as 
proposed by Maurizio will only allow non public inline classes. In that case, 
in order to use an inline type with a non reified generics, the user will have 
to declare an interface. LW5 can be a good intermediary step until we figure 
out how LW10 works.

One risk of LW5 is that it's not a release targeting all Java developpers but 
only the ones interrested in the performance of their libraries. We will have 
to be careful about how we communicate about LW5 to avoid frustration.

I think part of the issue of figuring what LW10 is, is that we have, at least i 
have, a fuzzy vision of what LW100 is. I don't even know what are the different 
steps to introduce reified generics, i.e the path in between LW10 and LW100. I 
think we need a minimal prototype of what the reified generics might be, 
similar to the MVT for the inline types.

As an eternal optimist, i think we can deliver LW5 next year ?

Karen replacement ?
This is something we have forgotten to discuss, Karen retires, who will take 
the mantle ? Or said differenly, I'm worry that the retirement of Karen may 
impact our velocity. Having this Wednesday meetings is important, it's one of 
the things that have allow us to have a good velocity. If can have a say in 
this, i vote for David Holmes, because like Karen, he doesn't hesitate to call 
our bluff. Is there another candidate ?

Rémi 

-- 
Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma 
brièveté.

Re: Draft LW2 spec

[Remi Forax]

Hi Daniel, 

section 4.1: in the table 4.1-A, 
13/57/45..57 is missing given you talk about later in this section. 

section 4.3.2: 
I don't think that using null as a diferentiator (Nullable/NonNullable) is a 
good idea. 
Yes, an inline type is not nullable, but it's also flattenable, loaded early, 
not circular, etc. This introduce a false dichotomy which we have already spent 
too much time. I don't remember the exact words that John is using, but it was 
making more sense too me. 
Perhaps only renaming NullFreeClassType to InlineClassType is enough ? 

section 4.10.1.2: 
the bottom right of the schema is wrong because a reference type can be a 
nullable type or an inline type which is not nullable 
I propose 

reference type 
/ \ 
/ \ 
nullable type inline class 
| 
| 
null 

Rémi 

> De: "daniel smith" 
> À: "valhalla-spec-experts" 
> Envoyé: Jeudi 13 Juin 2019 22:57:31
> Objet: Draft LW2 spec

> Here's a first look at a preview-feature-quality spec for
> values^H^H^H^H^H^Hinline classes.

> [
> http://cr.openjdk.java.net/~dlsmith/inline-classes/specs/jvms-inline-classes.html
> |
> http://cr.openjdk.java.net/~dlsmith/inline-classes/specs/jvms-inline-classes.html
> ]

> It reflects my understanding of what LW2 is supposed to look like. I'm sure 
> I've
> gotten out of sync on some things, so please take some time to check on things
> I might be missing.

> There are a few "design discussion" blocks that identify areas that may evolve
> further (in LW3, say) or where the design choices we've settled on aren't
> necessarily essential.

> One area that required making some nontrivial choices, and could use some 
> extra
> scrutiny, is the treatment of preparation and initialization. We know there 
> are
> some invariants we want enforced, but there's a lot of wiggle room in deciding
> how they are enforced.

> Note that I've included a suite of bug fixes/presentational improvements in a
> separate document, linked to in the introduction. If you notice differences
> between the official JVMS text and the text I'm quoting, that's probably why. 
> I
> don't think most people will be interested in those details, but if you are,
> feel free to review the second document as well!

V? and generics

[Remi Forax]

Let's say i have this interface:
  interface Orderable> {
   boolean lessThan(T t);
  }

and i want my inline class to implement it:
  @__inline__ class FooOrderable implements Orderable {
...
@Override
public boolean lessThan(FooOrderable? foo) {
  return false;
}
  }

how i'm suppose to write it given that the bound of FooOrderable as to be 
Orderable but Orderable is not a valid type ?

I think that we have moved from a generic of an inline class is unsafe (because 
of the NPEs) to it's an illegal type a little to fast.

I believe we should emit an unchecked warning instead with the definition of 
unchecked to be widened to include not only the CCEs but also the NPEs.

Rémi

It's not safe until it's in the coffer

[Remi Forax]

I would like to propose an alternative to the V? syntax we are currently using 
for representing an inline class encoded as a L-type.

I propose to use Coffer instead of Complex? (with Complex an inline 
class).

as notice several of us, the V? syntax as several shortcoming:
- as Brian said, using '?' as a character makes a type that mix wildcards and 
V? syntax hard to parse.
- as John said, using ? is misleading because other languages have use ? to 
indicate a nullable type but the Ltype of an inline class is an identityless 
nullable reference (and being an identityless reference is as important as 
being nullable)
- as i tried to say, the syntax V? means that V? is a supertype of V, so 
something that should be preferred to V as type of parameters in methods but we 
don't want to encourage people to write APIs that make heavy use of V? (using 
the words of Brian).


I think that a good alternative is to introduce a new class named 
java.lang.Coffer and makes the compiler to erase Coffer to LComplex;
with a mechanism named auto-coffering to go from an inline class to its coffer 
and vice-versa. Note that because auto-coffering is introduced at the same type 
as inline class is introduced in the lange, we are free to specify whatever 
semantics we want for overloaded methods that takes a coffer or its 
corresponding inline type.


Here are the advantage of using java.lang.Coffer
- we can document the exact semantics of a coffer in the javadoc of 
java.lang.Coffer.
- nullability and being a reference comes from free because it's a classical 
class from the user POV.
- it's obvious that if you want to call a method on a value type, you have to 
un-coffer it first which may lead to a NPE at that point.
- because auto-coffering is a conversion, Coffer is not a supertype of 
Complex.

There are two oddities:
- while the class java.lang.Coffer exists, you can not have an instance of 
Coffer at runtime, so you have these special rules:
  - java.lang.Coffer.class can exist but it can not be instantiated (like 
java.lang.Void).
  - java.lang.Coffer can not be a raw type and doens't accept any wildcard 
bounded or unbounded, because we don't know how to erase them.
- Coffer is a reified type so
  - instanceof Coffer and new Coffer[42] are ok.

So the main drawback is that a Coffer is a new kind of type with some very 
specific rules. 

Rémi

Re: Is V? what we need ?

[Remi Forax]

- Mail original -
> De: "Brian Goetz" 
> À: "Doug Lea" 
> Cc: "valhalla-spec-experts" 
> Envoyé: Mercredi 1 Mai 2019 16:39:59
> Objet: Re: Is V? what we need ?

> This is the point I’ve been failing to get across to Remi as well.  Suppose 
> you
> have a big value, and you want a sparse ArrayList of that value?  You need a
> way to say “Arraylist of BigValue, but not flattened.”  And that’s a use-site
> concern.  ArrayList gives you that.
> 
> Which is to say: the claim that V? is useless once we have reified generics is
> simply untrue.

i've said this is a corner case,
anyway i don't get this example given that both our proposal work the same on 
this example, N? is the L view of a value type in both proposal, we only differ 
on the way we interact with the type system, at the VM level, both proposal are 
equals.

Rémi

> 
>> On May 1, 2019, at 10:32 AM, Doug Lea  wrote:
>> 
>> On 5/1/19 10:12 AM, Brian Goetz wrote:
>>> 
>>> You’re assuming the utility of V? after reification is zero, but
>>> that’s not the case. For example, suppose ArrayList is reified.  Then
>>> the user gets the choice of erasure or reification by selecting
>>> ArrayList or ArrayList. Erasure has a bad rap, but sometimes
>>> it is desirable.
>> 
>> As I tried but failed to get across in post on sorting, choosing V vs V?
>> can lead to 5X cost differences in *either direction* for common
>> aggregate (collection and array) operations. So programmers will need to
>> learn some of these consequences. (Which is made harder/worse here
>> because the cost tradeoffs will probably change over time as VM support
>> improves.)
>> 
>> Which is mainly an argument for Brian's position, except for those who
>> believe that Java programmers should not have to encounter these kinds
>> of issues that C++, C#, OCAML, etc programmers face all the time;
>> betting that somehow VMs will make the right choice.
>> 
>> (Maybe this is not the right time to mention this, but most other
>> languages supporting this range of use have a middle-ground of
>> call-by-ref for extracting small parts of large things, as seen in
>> comparators, hash-codes, etc that can make a big performance difference.
>> I expect people will complain about not supporting this.)
>> 
>> -Doug
>> 
>> 

Re: Is V? what we need ?

[Remi Forax]

Actually,
you are more on my side because what i'm proposing make V? less present in 
public methods of API, which i think is a good move in term of perf model and 
for ArryList, both proposal work the same.

Rémi

- Mail original -
> De: "Doug Lea" 
> À: "valhalla-spec-experts" 
> Envoyé: Mercredi 1 Mai 2019 16:32:18
> Objet: Re: Is V? what we need ?

> On 5/1/19 10:12 AM, Brian Goetz wrote:
>> 
>> You’re assuming the utility of V? after reification is zero, but
>> that’s not the case. For example, suppose ArrayList is reified.  Then
>> the user gets the choice of erasure or reification by selecting
>> ArrayList or ArrayList. Erasure has a bad rap, but sometimes
>> it is desirable.
> 
> As I tried but failed to get across in post on sorting, choosing V vs V?
> can lead to 5X cost differences in *either direction* for common
> aggregate (collection and array) operations. So programmers will need to
> learn some of these consequences. (Which is made harder/worse here
> because the cost tradeoffs will probably change over time as VM support
> improves.)
> 
> Which is mainly an argument for Brian's position, except for those who
> believe that Java programmers should not have to encounter these kinds
> of issues that C++, C#, OCAML, etc programmers face all the time;
> betting that somehow VMs will make the right choice.
> 
> (Maybe this is not the right time to mention this, but most other
> languages supporting this range of use have a middle-ground of
> call-by-ref for extracting small parts of large things, as seen in
> comparators, hash-codes, etc that can make a big performance difference.
> I expect people will complain about not supporting this.)
> 
> -Doug