Re: Java port maintainers: What should be installed on the buildbot workers?

[Aaron Madlon-Kay]

> You had mentioned Java 9 introducing changes that were not backward 
> compatible. Does that only apply to the JDK, and not the JRE? Should the Java 
> 9 JRE be able to run stuff made with any earlier Java version?

Compared to previous updates, Java 9 breaks a lot of things and no, a lot of 
things won’t just run unmodified on the 9 JRE. But ensuring the system-default 
JRE is Java 8 won’t really help anything unless you’re going to be running 
`open` on a JAR or JNLP file. The java portgroup (correctly) gets JAVA_HOME 
with /usr/libexec/java_home, which always gives you the newest Java no matter 
what unless you specify a version with -v, so the default Java for building 
will still be 9.

This is why I was talking about having ports self-report their required Java 
version. A properly packaged project will have a launch script or some other 
way of specifying the right Java when running the final product, but it’s 
usually up to the user to use the right Java when building. Making all the 
Javas available doesn’t solve the problem of using the right one when building.

See here for what I did in an older version of Jython that required Java 7 to 
build (would not build on 6 or 8):
https://github.com/macports/macports-ports/commit/8ea9763eb25ca1484a62084e460b87963ba7aed0#diff-68dc98f4ea15b7ea36d6d5d515a327aaR31

I suppose if you want the setup that is most likely to work with the largest 
number of existing ports for the time being, do what you said but skip Java 9.

-Aaron

Re: Java port maintainers: What should be installed on the buildbot workers?

[Ryan Schmidt]

On Feb 28, 2018, at 08:08, Aaron Madlon-Kay wrote:

>> Ok. Then my plan is to install the Apple Java 6 JDK and the Oracle Java 7, 8 
>> and maybe 9 JDK, and the Oracle Java 8 JRE. Does that cover all the bases 
>> for now?
> 
> The JDKs will install the JRE automatically, and since I don’t think it ever 
> downgrades, you will end up with the Java 9 JRE after installing all the 
> JDKs. If you really want to have the Java 8 JRE, you can install in this 
> order: 6, 7, 9, delete the JRE, then 8. But I don’t think you need to care 
> about the JRE.
> 
> Other than the above I think that sounds good.

You had mentioned Java 9 introducing changes that were not backward compatible. 
Does that only apply to the JDK, and not the JRE? Should the Java 9 JRE be able 
to run stuff made with any earlier Java version?

Re: Java port maintainers: What should be installed on the buildbot workers?

[Aaron Madlon-Kay]

> Ok. Then my plan is to install the Apple Java 6 JDK and the Oracle Java 7, 8 
> and maybe 9 JDK, and the Oracle Java 8 JRE. Does that cover all the bases for 
> now?

The JDKs will install the JRE automatically, and since I don’t think it ever 
downgrades, you will end up with the Java 9 JRE after installing all the JDKs. 
If you really want to have the Java 8 JRE, you can install in this order: 6, 7, 
9, delete the JRE, then 8. But I don’t think you need to care about the JRE.

Other than the above I think that sounds good.

-Aaron

Re: Java port maintainers: What should be installed on the buildbot workers?

[Ryan Schmidt]

On Feb 26, 2018, at 23:55, Aaron Madlon-Kay wrote:

>> Thanks, you've cleared up another misconception I had. I had read that Java 
>> 7 installations were being automatically upgraded to Java 8, so I thought 
>> there was no point to trying to install Java 7.
> 
> The system-default Java will prompt for updates, but I don't think I've seen 
> it replace an existing installation without prompting, at least on Mac (it 
> should require administrator privileges, so I don't think it can). 
> Regardless, it doesn't prevent you from having multiple JDKs installed.
> 
>> I guess I still don't understand what JRE or JVM are. You mentioned 
>> /Library/Java/JavaVirtualMachines, which looks like it contains folders with 
>> names like jdk1.8.0_144.jdk. So does this mean that a JDK is a JVM? Or that 
>> a JDK includes a JVM?
> 
> - JRE is the Java Runtime Environment; this is what you need to run a 
> compiled Java program.
>   - /Library/Internet Plug-Ins/JavaAppletPlugin.plugin is where the blessed 
> system-default JRE lives. This path is an artifact of the fact that Java's 
> raison d'être used to be browser applets, but at this point it's just kind of 
> confusing. This JRE is used to run raw JARs or JNLPs you open in the Finder.
> 
> - JDK is the Java Development Kit; this is what you need to compile Java 
> source. The JDK includes a JRE, so if you are a developer you can just 
> install the JDK and not worry about anything else.
>   - /Library/Java/JavaVirtualMachines is where JDKs live
> 
> - JVM is the Java Virtual Machine; this is a binary included in the JRE (and 
> therefore in the JDK as well). This is not a separate package at all, but I 
> appreciate that the alphabet soup is confusing. For our purposes you can 
> ignore this word.
> 
>>> any properly packaged program will come with its own JRE
>> Does this include Java-using ports in MacPorts?
> 
> No, sorry, that was misleading. I was talking more about large, suite-style 
> applications like IntelliJ IDEs, where the JRE is included in the binary 
> distfile, inside the .app bundle. Though I suppose it's not impossible that a 
> piece of software available in MacPorts might have a distribution that 
> includes a JRE.
> 
>> You mentioned /Library/Internet Plug-Ins/JavaAppletPlugin.plugin. [...] is 
>> this plugin also involved somehow in any part of the MacPorts Java 
>> experience?
> 
> Per above, you should think of this less as a plugin and more as the 
> poorly-named default JRE.
> 
> A user who has only installed the JRE will have this plugin, and will be able 
> to *run* Java programs compatible with that JRE.
> 
> A user who has installed the JDK will have this plugin in addition to a JDK 
> at /Library/Java/JavaVirtualMachines, and will be able to *build and run* 
> Java programs compatible with that JDK, or any other installed JDK with 
> proper configuration.
> 
> Allow me to preemptively confuse you with one more thing: You may notice 
> java, javac, javaws etc. in /usr/bin; these are symlinks to binaries in 
> /System/Library/Frameworks/JavaVM.framework, which in turn are stubs that 
> forward to the proper Java installation (if available; otherwise it guides 
> users to download from Oracle). This allows you to invoke /usr/bin/java 
> without caring where the "real" java binary is. When multiple Javas are 
> available you should specify the one you want by setting the JAVA_HOME envar, 
> usually with the output of /usr/libexec/java_home.

Ok. Then my plan is to install the Apple Java 6 JDK and the Oracle Java 7, 8 
and maybe 9 JDK, and the Oracle Java 8 JRE. Does that cover all the bases for 
now?

Re: Java port maintainers: What should be installed on the buildbot workers?

[Aaron Madlon-Kay]

> Thanks, you've cleared up another misconception I had. I had read that
Java 7 installations were being automatically upgraded to Java 8, so I
thought there was no point to trying to install Java 7.

The system-default Java will prompt for updates, but I don't think I've
seen it replace an existing installation without prompting, at least on Mac
(it should require administrator privileges, so I don't think it can).
Regardless, it doesn't prevent you from having multiple JDKs installed.

> I guess I still don't understand what JRE or JVM are. You mentioned
/Library/Java/JavaVirtualMachines, which looks like it contains folders
with names like jdk1.8.0_144.jdk. So does this mean that a JDK is a JVM? Or
that a JDK includes a JVM?

- JRE is the Java Runtime Environment; this is what you need to run a
compiled Java program.
  - /Library/Internet Plug-Ins/JavaAppletPlugin.plugin is where the blessed
system-default JRE lives. This path is an artifact of the fact that
Java's raison
d'être used to be browser applets, but at this point it's just kind of
confusing. This JRE is used to run raw JARs or JNLPs you open in the Finder.

- JDK is the Java Development Kit; this is what you need to compile Java
source. The JDK includes a JRE, so if you are a developer you can just
install the JDK and not worry about anything else.
  - /Library/Java/JavaVirtualMachines is where JDKs live

- JVM is the Java Virtual Machine; this is a binary included in the JRE
(and therefore in the JDK as well). This is not a separate package at all,
but I appreciate that the alphabet soup is confusing. For our purposes you
can ignore this word.

>> any properly packaged program will come with its own JRE
> Does this include Java-using ports in MacPorts?

No, sorry, that was misleading. I was talking more about large, suite-style
applications like IntelliJ IDEs, where the JRE is included in the binary
distfile, inside the .app bundle. Though I suppose it's not impossible that
a piece of software available in MacPorts might have a distribution that
includes a JRE.

> You mentioned /Library/Internet Plug-Ins/JavaAppletPlugin.plugin. [...]
is this plugin also involved somehow in any part of the MacPorts Java
experience?

Per above, you should think of this less as a plugin and more as the
poorly-named default JRE.

A user who has only installed the JRE will have this plugin, and will be
able to *run* Java programs compatible with that JRE.

A user who has installed the JDK will have this plugin in addition to a JDK
at /Library/Java/JavaVirtualMachines, and will be able to *build and run*
Java programs compatible with that JDK, or any other installed JDK with
proper configuration.

Allow me to preemptively confuse you with one more thing: You may notice
java, javac, javaws etc. in /usr/bin; these are symlinks to binaries in
/System/Library/Frameworks/JavaVM.framework, which in turn are stubs that
forward to the proper Java installation (if available; otherwise it guides
users to download from Oracle). This allows you to invoke /usr/bin/java
without caring where the "real" java binary is. When multiple Javas are
available you should specify the one you want by setting the JAVA_HOME
envar, usually with the output of /usr/libexec/java_home.

-Aaron

Re: Java port maintainers: What should be installed on the buildbot workers?

[Ryan Schmidt]

On Feb 26, 2018, at 02:04, Aaron Madlon-Kay wrote:

>> Do you think there is any value to installing Apple Java 6 first, then 
>> installing Oracle Java 8, so that both JDKs are there? I'm just thinking of 
>> ancient ports that might be hardcoded to look for the old version that used 
>> to be bundled with macOS.
> 
> The order of installation doesn't really matter, but I think we might as well 
> install Java 6, 7, AND 8, as any given project may require any to build. (And 
> if we're going that far, we should have 9 as well but make 8 the default as 
> below.)
> 
> To clarify my previous suggestions, I think the java portgroup should have a 
> required_java_version property (default=1.8) that would be used to set 
> JAVA_HOME via /usr/libexec/java_home -f -v ${required_java_version} and fail 
> the operation if unavailable.

Thanks, you've cleared up another misconception I had. I had read that Java 7 
installations were being automatically upgraded to Java 8, so I thought there 
was no point to trying to install Java 7.

http://www.oracle.com/technetwork/java/javase/downloads/autoupdatejre7tojre8-2389085.html

Upon rereading that, I see now that it's talking about JRE, not JDK. So I guess 
that means we *can* install multiple JDKs as you said.

I guess I still don't understand what JRE or JVM are. You mentioned 
/Library/Java/JavaVirtualMachines, which looks like it contains folders with 
names like jdk1.8.0_144.jdk. So does this mean that a JDK is a JVM? Or that a 
JDK includes a JVM?

You said:

> any properly packaged program will come with its own JRE

Does this include Java-using ports in MacPorts? If so, how is this 
accomplished? Is the JRE built as part of the port building process, or is 
copied from somewhere? (from inside the JDK?) Or distributed in each port's 
source tarball?

You mentioned /Library/Internet Plug-Ins/JavaAppletPlugin.plugin. I assumed 
based on its name and path that it was only used for Java content loaded by a 
web page into a web browser. Or is this plugin also involved somehow in any 
part of the MacPorts Java experience?

Re: Java port maintainers: What should be installed on the buildbot workers?

[Michael]

On 2018-02-24, at 5:57 AM, Clemens Lang  wrote:
> Overall, I think we should install JDK 8 where possible. If we need
> compatibility with older versions, we should make sure to pass -target
> 1.7 when compiling.

And -target 1.5 or 1.5 machines :-).
(seriously, I have to bring one back to life ... It is nice to know there is a 
build bot for this machine.)

---
Entertaining minecraft videos
http://YouTube.com/keybounce

Re: Java port maintainers: What should be installed on the buildbot workers?

[Aaron Madlon-Kay]

> Do you think there is any value to installing Apple Java 6 first, then
installing Oracle Java 8, so that both JDKs are there? I'm just thinking of
ancient ports that might be hardcoded to look for the old version that used
to be bundled with macOS.

The order of installation doesn't really matter, but I think we might as
well install Java 6, 7, AND 8, as any given project may require any to
build. (And if we're going that far, we should have 9 as well but make 8
the default as below.)

To clarify my previous suggestions, I think the java portgroup should have
a required_java_version property (default=1.8) that would be used to set
JAVA_HOME via /usr/libexec/java_home -f -v ${required_java_version} and
fail the operation if unavailable.

-Aaron

Re: Java port maintainers: What should be installed on the buildbot workers?

[Ryan Schmidt]

On Feb 25, 2018, at 06:58, Aaron Madlon-Kay wrote:

>> Huh. The system requirements linked from the download page say "Intel-based 
>> Mac running Mac OS X 10.7.3 (Lion) or later.”
> 
> Very interesting. I was able to install JDK 8u161 in my OS X 10.7.5 VM. 
> Apparently Oracle has had contradictory advice about this for a long time, 
> and they never cleared it up:
> https://derflounder.wordpress.com/2015/01/10/oracles-java-8-and-mac-os-x-10-7-x/

Because Oracle!


> I stand corrected.
> 
>> The Mac FAQ also says "If installing an older version of Java is absolutely 
>> required, you must first uninstall the current version"; from this I assumed 
>> that simultaneous installation of multiple versions is not possible.
> 
> That’s probably talking about the JRE at /Library/Internet 
> Plug-Ins/JavaAppletPlugin.plugin; an older one will never overwrite a newer 
> one already present there. However we are concerned with the JDK, which lives 
> at /Library/JavaVirtualMachines, and it is most definitely possible to have 
> multiple JDKs installed there.

Do you think there is any value to installing Apple Java 6 first, then 
installing Oracle Java 8, so that both JDKs are there? I'm just thinking of 
ancient ports that might be hardcoded to look for the old version that used to 
be bundled with macOS.


>> Apparently Java 9 exists. It is mentioned on oracle.com but not on java.com. 
>> I don't know if I should be attempting to install 8 or 9.
> 
> Java 9 is very new, and it has changes that break building for a large number 
> of projects that are not yet specifically targeting it. I would stick with 
> Java 8 for the near future.

Ok.

Re: Java port maintainers: What should be installed on the buildbot workers?

[Aaron Madlon-Kay]

> Huh. The system requirements linked from the download page say "Intel-based 
> Mac running Mac OS X 10.7.3 (Lion) or later.”

Very interesting. I was able to install JDK 8u161 in my OS X 10.7.5 VM. 
Apparently Oracle has had contradictory advice about this for a long time, and 
they never cleared it up:
https://derflounder.wordpress.com/2015/01/10/oracles-java-8-and-mac-os-x-10-7-x/

I stand corrected.

> The Mac FAQ also says "If installing an older version of Java is absolutely 
> required, you must first uninstall the current version"; from this I assumed 
> that simultaneous installation of multiple versions is not possible.

That’s probably talking about the JRE at /Library/Internet 
Plug-Ins/JavaAppletPlugin.plugin; an older one will never overwrite a newer one 
already present there. However we are concerned with the JDK, which lives at 
/Library/JavaVirtualMachines, and it is most definitely possible to have 
multiple JDKs installed there.

> Apparently Java 9 exists. It is mentioned on oracle.com but not on java.com. 
> I don't know if I should be attempting to install 8 or 9.

Java 9 is very new, and it has changes that break building for a large number 
of projects that are not yet specifically targeting it. I would stick with Java 
8 for the near future.

-Aaron

Re: Java port maintainers: What should be installed on the buildbot workers?

[Ryan Schmidt]

On Feb 24, 2018, at 06:17, Aaron Madlon-Kay wrote:

>> Oracle Java 7 and 8 are compatible with Lion and later.
> 
> Java 8 requires Mountain Lion:
> http://www.oracle.com/technetwork/java/javase/certconfig-2095354.html

Huh. The system requirements linked from the download page say "Intel-based Mac 
running Mac OS X 10.7.3 (Lion) or later."

https://www.java.com/en/download/mac_sysreq-sm.jsp


The Mac FAQ says the same:

https://www.java.com/en/download/faq/java_mac.xml#sysreq

If the latest version does not work on 10.7 anymore, then maybe an older 
version can be found that does.


The Mac FAQ also says "If installing an older version of Java is absolutely 
required, you must first uninstall the current version"; from this I assumed 
that simultaneous installation of multiple versions is not possible.

https://www.java.com/en/download/faq/java_mac.xml#newerver


According to this page, "When you install the JRE, you can install only one JRE 
on your system at a time."

https://docs.oracle.com/javase/9/install/installation-jdk-and-jre-macos.htm#JSJIG-GUID-E8A251B6-D9A9-4276-ABC8-CC0DAD62EA33


Apparently Java 9 exists. It is mentioned on oracle.com but not on java.com. I 
don't know if I should be attempting to install 8 or 9.

Re: Java port maintainers: What should be installed on the buildbot workers?

[Aaron Madlon-Kay]

> I'm not very familiar with Java open source project best practices, but
> maybe it is common practice to specify the -source and -target options
> anyway, so that our choice of JDK version wouldn't matter?

This is what I was talking about being “frowned upon”. Even if you specify 
-source and -target, you can accidentally compile in e.g. calls to methods that 
don’t exist in older JDK standard libraries, so the program will fail at 
runtime on old JDKs. In practice it’s not that common a problem.

-Aaron

Re: Java port maintainers: What should be installed on the buildbot workers?

[Clemens Lang]

On Fri, Feb 23, 2018 at 06:43:27PM -0600, Ryan Schmidt wrote:
> I think I should probably install Oracle's latest Java 8 on the Lion
> and later buildbot workers. Does that seem reasonable? I don't know if
> our java portgroup is compatible with that.

The java portgroup uses /usr/libexec/java_home, which does support
Oracle's Java 8. On my High Sierra install with Oracle Java 8:

 $> /usr/libexec/java_home
 /Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home

> I also don't know what happens if we do that, and then a user who has
> Apple's legacy Java 6 installed receives a binary archive of a
> Java-using port. Does it work? Does it fail?

"The class file version for Java SE 8 is 52.0 as per the JVM
Specification. Version 52.0 class files produced by a Java SE 8 compiler
cannot be used in earlier releases of Java SE." [1]

This means that binaries produced on our buildbot with a Java 8 compiler
will only work with Java 8 VMs. Binaries produced with a Java 7 compiler
will work on Java 8 VMs. Java compilers can usually be asked with a
command line switch to emit bytecode compatible with older versions of
Java. See the "-target" option in [2].

I'm not very familiar with Java open source project best practices, but
maybe it is common practice to specify the -source and -target options
anyway, so that our choice of JDK version wouldn't matter?

> A second option is to install Apple's legacy Java 6 on the Lion and
> later buildbot workers. I'm guessing this is not the right choice,
> since it is old and probably has security problems. And the same
> questions arise again: Would this be compatible with the java
> portgroup? And what happens if a user who has Oracle Java 7 or 8
> installed gets a binary archive built using legacy Java 6? Does it
> work or fail?

It will work.


Overall, I think we should install JDK 8 where possible. If we need
compatibility with older versions, we should make sure to pass -target
1.7 when compiling.


[1] 
http://www.oracle.com/technetwork/java/javase/8-compatibility-guide-2156366.html#A999170
[2] https://docs.oracle.com/javase/8/docs/technotes/tools/windows/javac.html

Re: Java port maintainers: What should be installed on the buildbot workers?

[Aaron Madlon-Kay]

> Oracle Java 7 and 8 are compatible with Lion and later.

Java 8 requires Mountain Lion:
http://www.oracle.com/technetwork/java/javase/certconfig-2095354.html

> My understanding is that one must choose either Apple's Java 6 or Oracle's 
> Java 7 or 8; they can't coexist.

I’m not aware of any scenario where this is the case. You can have umpteen 
installations of Java coexist at once, living in 
/Library/Java/JavaVirtualMachines. You will generally set your JAVA_HOME to the 
one you want to use by default.

You can only have one Java in /Library/Internet 
Plug-Ins/JavaAppletPlugin.plugin, and I believe launching raw JARs or JNLPs 
will use this Java, but any properly packaged program will come with its own 
JRE, or have a launch script that uses e.g. /usr/libexec/java_home to discover 
the appropriate Java.

> I think I should probably install Oracle's latest Java 8 on the Lion and 
> later buildbot workers. Does that seem reasonable?

I think that sounds reasonable. Older Javas are already EOL. However per above 
it will have to be Mountain Lion and later.

> I also don't know what happens if we do that, and then a user who has Apple's 
> legacy Java 6 installed receives a binary archive of a Java-using port. Does 
> it work? Does it fail?

Java binaries have some minimum required JDK version, and are usually 
forward-compatible. A binary that requires Java 7 or 8 would fail in that 
scenario (assuming the user *only* has Java 6 installed). A binary requiring 
Java 6 will generally run on 6 or later, so it would work.

I don’t think there’s a good way to test for the minimum required Java for a 
given binary. I think it would make sense to make that a portgroup property 
that would be set in each individual portfile.

> However, if a Java port would build differently depending on whether Apple's 
> legacy Java 6 or Oracle's Java 7 or 8 is installed, we may have to prevent 
> distribution of binary archives of Java ports anyway, at least in the case 
> where we detect that the Java the user has installed is not the Java that we 
> decide to install on the buildbot workers.

Generally you can build projects on a newer Java than what the project targets, 
e.g. you can build a Java 7 project with Java 8 and still have it run on Java 
7. However this is frowned upon (see 
https://stackoverflow.com/a/12050087/448068) so if you’re going to be strict 
about it you probably need to install all available Java versions and make sure 
only to build with the one required by that project.

-Aaron


> On Feb 24, 2018, at 9:43, Ryan Schmidt  wrote:
> 
> I'd appreciate advice from maintainers of ports that use Java. What Java 
> should be installed on the buildbot workers?
> 
> Currently, only our Leopard and Snow Leopard buildbot workers have Java 
> installed, because Apple's Java was included in the Mac OS X installer. It's 
> a very old version (Java 2 build 1.5.0_30-b03-389-9M3425 on Leopard, Java SE 
> build 1.6.0_65-b14-462-10M4609 on Snow Leopard) but it's something.
> 
> Apple removed its Java from the OS X installer in Lion, and I have not 
> separately installed Java on the Lion and later buildbot workers, because I 
> didn't know what should be installed. 
> 
> Apple's legacy Java 6 distribution can be installed on Lion and later up to 
> and including High Sierra, but Apple doesn't recommend it. They recommend 
> using the latest Oracle Java. Oracle Java 7 and 8 are compatible with Lion 
> and later.
> 
> My understanding is that one must choose either Apple's Java 6 or Oracle's 
> Java 7 or 8; they can't coexist.
> 
> I think I should probably install Oracle's latest Java 8 on the Lion and 
> later buildbot workers. Does that seem reasonable? I don't know if our java 
> portgroup is compatible with that. I also don't know what happens if we do 
> that, and then a user who has Apple's legacy Java 6 installed receives a 
> binary archive of a Java-using port. Does it work? Does it fail?
> 
> A second option is to install Apple's legacy Java 6 on the Lion and later 
> buildbot workers. I'm guessing this is not the right choice, since it is old 
> and probably has security problems. And the same questions arise again: Would 
> this be compatible with the java portgroup? And what happens if a user who 
> has Oracle Java 7 or 8 installed gets a binary archive built using legacy 
> Java 6? Does it work or fail?
> 
> A third option is to do nothing. I think this isn't great because it means 
> that ports that use Java fail on the Lion and later buildbot workers, which 
> prevents maintainers from learning about legitimate build failures their 
> ports may have, and prevents users from receiving binary archives of 
> distributable Java ports. However, if a Java port would build differently 
> depending on whether Apple's legacy Java 6 or Oracle's Java 7 or 8 is 
> installed, we may have to prevent distribution of binary archives of Java 
> ports anyway, at least in the case where we detect