Re: RFR: JDK-8195689 Remove generated-configure.sh and instead use autoconf

[Erik Joelsson]

This version looks good to me.

/Erik


On 2018-01-25 13:33, Magnus Ihse Bursie wrote:

On 2018-01-18 19:37, Erik Joelsson wrote:

Hello Magnus,

Nice to see this finally happening!

Thanks!



In building.md, when getting autoconf for Cygwin, I believe you also 
need the autoconf wrapper scripts package. 
Hm, well, no, it's the other way around. "autoconf" is the name of the 
cygwin "autoconf wrapper scripts" package. It depends on "m4", 
"autoconf2.69" (or whatever), etc. I have verified that just 
installing "autoconf" will be enough, and will pull in all needed 
dependencies.


Perhaps it's also worth mentioning that you can download the autoconf 
src and build/install from there?
Well, I'm already giving a link to 
http://www.gnu.org/software/autoconf in the build readme. I'm not sure 
if it's helpful for anyone to download the autoconf source by 
themselves. Most users will have it available prebuilt. And if not, 
you'll also most likely need to pull down and build m4, and that's 
trickier. The reason why we used to download autoconf and build it 
from source was to make sure that we got the exactly right version, 
but that only made sense when we checked in the 
generated-configure.sh. (And this worked fine for us, since we had m4...)


In jib-profiles.js, you need to use build_platform instead of 
target_platform when creating the module name for the dependency. 

Good point. Fixed.

I would also like if you added the dependency on the other main 
profiles as well. At least linux-x86, linux-arm64 and windows-x86 are 
still possible to build and should work fine with this. For 
solaris-x64 you would need to generate the package for it to work.

Sure.


The TMPDIR logic looks weird. Are you unconditionally setting it to 
/cygdrive/t/...? That's a valid dir in our build farm, but not on my 
local windows box:


mkdir: cannot create directory ‘/cygdrive/t’: No such file or directory
autom4te: cannot create 
/cygdrive/t/workspace/.build/tmpdir/am4t7009.7040: No such file or 
directory
 at 
/cygdrive/c/cygwin64/var/tmp/jib-erik/install/jpg/infra/builddeps/autoconf-windows_x64/2.69+1.0.1/autoconf-windows_x64-2.69+1.0.1.tar.gz/usr/bin/autom4te 
line 954.

Error: Failed to generate runnable configure script

Yeah, that was broken by design. :-(

Some offline discussion later, it turned out that the reason for the 
trouble was mismatch between cygwin-32 and cygwin-64. I've solved this 
by generating two different packages, for cygwin-32 and cygwin-64. 
I've also made some other minor fixes to the build script, e.g. using 
perl instead of sed due to the braindead sed being available on 
solaris. :(


Updated webrev:
http://cr.openjdk.java.net/~ihse/JDK-8195689-remove-generated-configure/webrev.02/ 



/Magnus




/Erik


On 2018-01-18 05:28, Magnus Ihse Bursie wrote:
Currently, we require all developers who modify the configure script 
to run autoconf locally, to update the generated-configure.sh 
script, which is then checked in. This is the only instance of 
checked in "compiled" code in OpenJDK, and this has brought along 
several problems:


* Only a specific version of autoconf, 2.69, can be used, to avoid 
large code changes in the generated file. Unfortunately, Ubuntu 
ships a version of autoconf that claims to be 2.69 but is actually 
heavily patched. This requires all Ubuntu users to compiler their 
own autoconf from source.


* The Oracle JDK closed sources has a closed version that needs to 
be updated. In practice, this has meant that all non-Oracle 
developers, need an Oracle sponsor for patches modifying the 
configure script.


* If the configure script is not properly updated, the build will 
fail. The same happens on the Oracle side if the closed version is 
not in sync with the open version. It is easy to miss re-generating 
the script after the last fix of a typo in the comments in an .m4 
file...


* Merging between two changes containing configure modifications is 
almost impossible. In practice, the entire generated-configure.sh 
needs to be thrown away and regenerated.


The entire benefit of having the file in the repo is to save 
first-time developers the hassle of installing autoconf. On most 
platforms, this is a no-brainer (like "apt install autoconf"), and 
the requirement is similar to other open source projects using 
autoconf and "./configure". It's just not worth it.


Bug: https://bugs.openjdk.java.net/browse/JDK-8195689
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8195689-remove-generated-configure/webrev.01


/Magnus

Re: RFR: JDK-8195689 Remove generated-configure.sh and instead use autoconf

[Magnus Ihse Bursie]

On 2018-01-18 19:37, Erik Joelsson wrote:

Hello Magnus,

Nice to see this finally happening!

Thanks!



In building.md, when getting autoconf for Cygwin, I believe you also 
need the autoconf wrapper scripts package. 
Hm, well, no, it's the other way around. "autoconf" is the name of the 
cygwin "autoconf wrapper scripts" package. It depends on "m4", 
"autoconf2.69" (or whatever), etc. I have verified that just installing 
"autoconf" will be enough, and will pull in all needed dependencies.


Perhaps it's also worth mentioning that you can download the autoconf 
src and build/install from there?
Well, I'm already giving a link to http://www.gnu.org/software/autoconf 
in the build readme. I'm not sure if it's helpful for anyone to download 
the autoconf source by themselves. Most users will have it available 
prebuilt. And if not, you'll also most likely need to pull down and 
build m4, and that's trickier. The reason why we used to download 
autoconf and build it from source was to make sure that we got the 
exactly right version, but that only made sense when we checked in the 
generated-configure.sh. (And this worked fine for us, since we had m4...)


In jib-profiles.js, you need to use build_platform instead of 
target_platform when creating the module name for the dependency. 

Good point. Fixed.

I would also like if you added the dependency on the other main 
profiles as well. At least linux-x86, linux-arm64 and windows-x86 are 
still possible to build and should work fine with this. For 
solaris-x64 you would need to generate the package for it to work.

Sure.


The TMPDIR logic looks weird. Are you unconditionally setting it to 
/cygdrive/t/...? That's a valid dir in our build farm, but not on my 
local windows box:


mkdir: cannot create directory ‘/cygdrive/t’: No such file or directory
autom4te: cannot create 
/cygdrive/t/workspace/.build/tmpdir/am4t7009.7040: No such file or 
directory
 at 
/cygdrive/c/cygwin64/var/tmp/jib-erik/install/jpg/infra/builddeps/autoconf-windows_x64/2.69+1.0.1/autoconf-windows_x64-2.69+1.0.1.tar.gz/usr/bin/autom4te 
line 954.

Error: Failed to generate runnable configure script

Yeah, that was broken by design. :-(

Some offline discussion later, it turned out that the reason for the 
trouble was mismatch between cygwin-32 and cygwin-64. I've solved this 
by generating two different packages, for cygwin-32 and cygwin-64. I've 
also made some other minor fixes to the build script, e.g. using perl 
instead of sed due to the braindead sed being available on solaris. :(


Updated webrev:
http://cr.openjdk.java.net/~ihse/JDK-8195689-remove-generated-configure/webrev.02/

/Magnus




/Erik


On 2018-01-18 05:28, Magnus Ihse Bursie wrote:
Currently, we require all developers who modify the configure script 
to run autoconf locally, to update the generated-configure.sh script, 
which is then checked in. This is the only instance of checked in 
"compiled" code in OpenJDK, and this has brought along several problems:


* Only a specific version of autoconf, 2.69, can be used, to avoid 
large code changes in the generated file. Unfortunately, Ubuntu ships 
a version of autoconf that claims to be 2.69 but is actually heavily 
patched. This requires all Ubuntu users to compiler their own 
autoconf from source.


* The Oracle JDK closed sources has a closed version that needs to be 
updated. In practice, this has meant that all non-Oracle developers, 
need an Oracle sponsor for patches modifying the configure script.


* If the configure script is not properly updated, the build will 
fail. The same happens on the Oracle side if the closed version is 
not in sync with the open version. It is easy to miss re-generating 
the script after the last fix of a typo in the comments in an .m4 
file...


* Merging between two changes containing configure modifications is 
almost impossible. In practice, the entire generated-configure.sh 
needs to be thrown away and regenerated.


The entire benefit of having the file in the repo is to save 
first-time developers the hassle of installing autoconf. On most 
platforms, this is a no-brainer (like "apt install autoconf"), and 
the requirement is similar to other open source projects using 
autoconf and "./configure". It's just not worth it.


Bug: https://bugs.openjdk.java.net/browse/JDK-8195689
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8195689-remove-generated-configure/webrev.01


/Magnus

Re: RFR: JDK-8195689 Remove generated-configure.sh and instead use autoconf

[Martin Buchholz]

Alright folks, you've convinced me.

The "builder interface" is the same "bash configure && make"
(aside: having configure be non-executable is super-annoying, because it
changes an interface burned into my fingers)
and yes, users already have to install many dependencies.
I was thinking you would force users to run some other command like
autoconf; that was the path I recall taken by some other projects.

On debian-based systems, there's "sudo apt-get build-dep openjdk-N" for
some N close to whatever you're actually building.



On Thu, Jan 18, 2018 at 11:49 PM, Magnus Ihse Bursie <
magnus.ihse.bur...@oracle.com> wrote:

> On 2018-01-19 08:08, Erik Helin wrote:
>
>> On 01/19/2018 07:18 AM, Martin Buchholz wrote:
>>
>>> Differing projects have come to different conclusions about whether to
>>> include a generated configure.
>>>
>>> But the standard seems to be to include one. The mantra is: "./configure
>>> &&
>>> make" without an autoconf step.
>>>
>>
>> And this is still the mantra (except we don't have an executable
>> configure file in the repo so you have to run `bash configure && make`).
>> The only thing we are discussing is whether the script "configure" should
>> depend on the program "autoconf" or not.
>>
>> If I'm downloading a .tar.gz source code bundle of a project (like the
>> ones usually generated via `make dist`), then it seems more common that
>> "configure" will not depend on autoconf. However, if I'm cloning a project
>> from source, then I'm used to having autoconf being run for me (or
>> sometimes having to run it myself).
>>
> Good point. If we were to start shipping source code bundles, it would
> certainly make sense to include a generated configure script for it. I'm
> all for it. The problem here is that the current model presupposes a
> working, generated configure script for every separate commit.
>
> I'll admit that I was part of introducing this model some five (or more?)
> years ago. I didn't really like it then either, but at that time I
> perceived it to be a quite massive resistance amongst the JDK developers
> (perhaps mostly inside Oracle) for any kind of change in the environment,
> so adding a new build requirement seemed like a huge war that was needed to
> be fought. (Just removing the old build system was enough effort...)
> Nowadays, we're using jib inside Oracle, so a new requirement is virtually
> undetectable by most developers. And for all others, it's just an "sudo apt
> install autoconf" (or brew, or whatever) away, in the unlikely case that
> you're a developer without autoconf installed already.
>
> /Magnus
>
>
>> The number of people building openjdk is
>>> much larger than the number of people patching configure.  So I agree
>>> with
>>> David that we should stick with the status quo.
>>>
>>
>> I disagree, I don't think depending on autoconf will make it harder to
>> build OpenJDK. Remember that the build steps are still:
>>
>> $ bash configure
>> $ make
>>
>> the only difference now is that configure will use autoconf to first
>> generate .build/generated-configure.sh. As noted in the documentation,
>> installing autoconf is trivial on essentially any system today (the
>> configure script will also provide a useful help message for your platform
>> if you are missing autoconf).
>>
>> So for me, this patch gets +1. I'll leave the actual Makefile changes and
>> details for Erik J to review though ;)
>>
>> Thanks,
>> Erik
>>
>> On Thu, Jan 18, 2018 at 6:14 PM, David Holmes 
>>> wrote:
>>>
>>> On 18/01/2018 11:28 PM, Magnus Ihse Bursie wrote:

 Currently, we require all developers who modify the configure script to
> run autoconf locally, to update the generated-configure.sh script,
> which is
> then checked in. This is the only instance of checked in "compiled"
> code in
> OpenJDK, and this has brought along several problems:
>
> * Only a specific version of autoconf, 2.69, can be used, to avoid
> large
> code changes in the generated file. Unfortunately, Ubuntu ships a
> version
> of autoconf that claims to be 2.69 but is actually heavily patched.
> This
> requires all Ubuntu users to compiler their own autoconf from source.
>
> * The Oracle JDK closed sources has a closed version that needs to be
> updated. In practice, this has meant that all non-Oracle developers,
> need
> an Oracle sponsor for patches modifying the configure script.
>
> * If the configure script is not properly updated, the build will fail.
> The same happens on the Oracle side if the closed version is not in
> sync
> with the open version. It is easy to miss re-generating the script
> after
> the last fix of a typo in the comments in an .m4 file...
>
> * Merging between two changes containing configure modifications is
> almost impossible. In practice, the entire generated-configure.sh
> needs to
> be thrown away and regenerated.
>
> The entire benef

Re: RFR: JDK-8195689 Remove generated-configure.sh and instead use autoconf

[Magnus Ihse Bursie]

On 2018-01-19 08:08, Erik Helin wrote:

On 01/19/2018 07:18 AM, Martin Buchholz wrote:

Differing projects have come to different conclusions about whether to
include a generated configure.

But the standard seems to be to include one. The mantra is: 
"./configure &&

make" without an autoconf step.


And this is still the mantra (except we don't have an executable 
configure file in the repo so you have to run `bash configure && 
make`). The only thing we are discussing is whether the script 
"configure" should depend on the program "autoconf" or not.


If I'm downloading a .tar.gz source code bundle of a project (like the 
ones usually generated via `make dist`), then it seems more common 
that "configure" will not depend on autoconf. However, if I'm cloning 
a project from source, then I'm used to having autoconf being run for 
me (or sometimes having to run it myself).
Good point. If we were to start shipping source code bundles, it would 
certainly make sense to include a generated configure script for it. I'm 
all for it. The problem here is that the current model presupposes a 
working, generated configure script for every separate commit.


I'll admit that I was part of introducing this model some five (or 
more?) years ago. I didn't really like it then either, but at that time 
I perceived it to be a quite massive resistance amongst the JDK 
developers (perhaps mostly inside Oracle) for any kind of change in the 
environment, so adding a new build requirement seemed like a huge war 
that was needed to be fought. (Just removing the old build system was 
enough effort...) Nowadays, we're using jib inside Oracle, so a new 
requirement is virtually undetectable by most developers. And for all 
others, it's just an "sudo apt install autoconf" (or brew, or whatever) 
away, in the unlikely case that you're a developer without autoconf 
installed already.


/Magnus



The number of people building openjdk is
much larger than the number of people patching configure.  So I agree 
with

David that we should stick with the status quo.


I disagree, I don't think depending on autoconf will make it harder to 
build OpenJDK. Remember that the build steps are still:


$ bash configure
$ make

the only difference now is that configure will use autoconf to first 
generate .build/generated-configure.sh. As noted in the documentation, 
installing autoconf is trivial on essentially any system today (the 
configure script will also provide a useful help message for your 
platform if you are missing autoconf).


So for me, this patch gets +1. I'll leave the actual Makefile changes 
and details for Erik J to review though ;)


Thanks,
Erik


On Thu, Jan 18, 2018 at 6:14 PM, David Holmes 
wrote:


On 18/01/2018 11:28 PM, Magnus Ihse Bursie wrote:

Currently, we require all developers who modify the configure 
script to
run autoconf locally, to update the generated-configure.sh script, 
which is
then checked in. This is the only instance of checked in "compiled" 
code in

OpenJDK, and this has brought along several problems:

* Only a specific version of autoconf, 2.69, can be used, to avoid 
large
code changes in the generated file. Unfortunately, Ubuntu ships a 
version
of autoconf that claims to be 2.69 but is actually heavily patched. 
This

requires all Ubuntu users to compiler their own autoconf from source.

* The Oracle JDK closed sources has a closed version that needs to be
updated. In practice, this has meant that all non-Oracle 
developers, need

an Oracle sponsor for patches modifying the configure script.

* If the configure script is not properly updated, the build will 
fail.
The same happens on the Oracle side if the closed version is not in 
sync
with the open version. It is easy to miss re-generating the script 
after

the last fix of a typo in the comments in an .m4 file...

* Merging between two changes containing configure modifications is
almost impossible. In practice, the entire generated-configure.sh 
needs to

be thrown away and regenerated.

The entire benefit of having the file in the repo is to save 
first-time
developers the hassle of installing autoconf. On most platforms, 
this is a
no-brainer (like "apt install autoconf"), and the requirement is 
similar to
other open source projects using autoconf and "./configure". It's 
just not

worth it.



I'm not convinced just by you saying it is so - sorry. This seems to 
make
an already complex build process even more complex for every single 
person
who wants to build OpenJDK, for the benefit of a handful of people 
who may

want to modify configure options and whom already work closely with the
build team and so there's really little hardship in getting a 
sponsor, or

just someone with access to autoconf.

It introduces a new point of failure in the build for everyone.

Has this been beta-tested with external contributors? I'd be happier
knowing we've put this through its paces with people developing on a 
wide

range of platforms, before mak

Re: RFR: JDK-8195689 Remove generated-configure.sh and instead use autoconf

[Magnus Ihse Bursie]

On 2018-01-19 07:18, Martin Buchholz wrote:
Differing projects have come to different conclusions about whether to 
include a generated configure.


But the standard seems to be to include one. The mantra is: 
"./configure && make" without an autoconf step. The number of people 
building openjdk is much larger than the number of people patching 
configure.  So I agree with David that we should stick with the status 
quo.


This patch will not introduce a separate autoconf step. Please see the 
updated build readme in my patch. The only difference is an added build 
requirement.


If someone wants to contribute to OpenJDK, installing external 
dependencies for the build can hardly be the biggest hurdle. If we 
*really* thought that was an issue, then we should start bundling much 
more external dependencies. Like freetype, which is a *real* problem for 
all Windows users, for instance.


I don't think that's the right way to go, though. And in general, the 
direction of OpenJDK for the last few years has been in the opposite 
direction: less bundling, more external dependencies. The community has 
requested configure arguments for using the system provided versions of 
libraries that we also bundle. We've stopped linking statically with 
glibc. We've removed bundled source code and introduced a new dependency 
for fontconfig.


/Magnus


On Thu, Jan 18, 2018 at 6:14 PM, David Holmes > wrote:


On 18/01/2018 11:28 PM, Magnus Ihse Bursie wrote:

Currently, we require all developers who modify the configure
script to run autoconf locally, to update the
generated-configure.sh script, which is then checked in. This
is the only instance of checked in "compiled" code in OpenJDK,
and this has brought along several problems:

* Only a specific version of autoconf, 2.69, can be used, to
avoid large code changes in the generated file. Unfortunately,
Ubuntu ships a version of autoconf that claims to be 2.69 but
is actually heavily patched. This requires all Ubuntu users to
compiler their own autoconf from source.

* The Oracle JDK closed sources has a closed version that
needs to be updated. In practice, this has meant that all
non-Oracle developers, need an Oracle sponsor for patches
modifying the configure script.

* If the configure script is not properly updated, the build
will fail. The same happens on the Oracle side if the closed
version is not in sync with the open version. It is easy to
miss re-generating the script after the last fix of a typo in
the comments in an .m4 file...

* Merging between two changes containing configure
modifications is almost impossible. In practice, the entire
generated-configure.sh needs to be thrown away and regenerated.

The entire benefit of having the file in the repo is to save
first-time developers the hassle of installing autoconf. On
most platforms, this is a no-brainer (like "apt install
autoconf"), and the requirement is similar to other open
source projects using autoconf and "./configure". It's just
not worth it.


I'm not convinced just by you saying it is so - sorry. This seems
to make an already complex build process even more complex for
every single person who wants to build OpenJDK, for the benefit of
a handful of people who may want to modify configure options and
whom already work closely with the build team and so there's
really little hardship in getting a sponsor, or just someone with
access to autoconf.

It introduces a new point of failure in the build for everyone.

Has this been beta-tested with external contributors? I'd be
happier knowing we've put this through its paces with people
developing on a wide range of platforms, before making it the default.

Have the devkits been updated so I can try this out myself?

Thanks,
David


Bug: https://bugs.openjdk.java.net/browse/JDK-8195689

WebRev:

http://cr.openjdk.java.net/~ihse/JDK-8195689-remove-generated-configure/webrev.01






/Magnus

Re: RFR: JDK-8195689 Remove generated-configure.sh and instead use autoconf

[Erik Helin]

On 01/19/2018 07:18 AM, Martin Buchholz wrote:

Differing projects have come to different conclusions about whether to
include a generated configure.

But the standard seems to be to include one. The mantra is: "./configure &&
make" without an autoconf step.


And this is still the mantra (except we don't have an executable 
configure file in the repo so you have to run `bash configure && make`). 
The only thing we are discussing is whether the script "configure" 
should depend on the program "autoconf" or not.


If I'm downloading a .tar.gz source code bundle of a project (like the 
ones usually generated via `make dist`), then it seems more common that 
"configure" will not depend on autoconf. However, if I'm cloning a 
project from source, then I'm used to having autoconf being run for me 
(or sometimes having to run it myself).



The number of people building openjdk is
much larger than the number of people patching configure.  So I agree with
David that we should stick with the status quo.


I disagree, I don't think depending on autoconf will make it harder to 
build OpenJDK. Remember that the build steps are still:


$ bash configure
$ make

the only difference now is that configure will use autoconf to first 
generate .build/generated-configure.sh. As noted in the documentation, 
installing autoconf is trivial on essentially any system today (the 
configure script will also provide a useful help message for your 
platform if you are missing autoconf).


So for me, this patch gets +1. I'll leave the actual Makefile changes 
and details for Erik J to review though ;)


Thanks,
Erik


On Thu, Jan 18, 2018 at 6:14 PM, David Holmes 
wrote:


On 18/01/2018 11:28 PM, Magnus Ihse Bursie wrote:


Currently, we require all developers who modify the configure script to
run autoconf locally, to update the generated-configure.sh script, which is
then checked in. This is the only instance of checked in "compiled" code in
OpenJDK, and this has brought along several problems:

* Only a specific version of autoconf, 2.69, can be used, to avoid large
code changes in the generated file. Unfortunately, Ubuntu ships a version
of autoconf that claims to be 2.69 but is actually heavily patched. This
requires all Ubuntu users to compiler their own autoconf from source.

* The Oracle JDK closed sources has a closed version that needs to be
updated. In practice, this has meant that all non-Oracle developers, need
an Oracle sponsor for patches modifying the configure script.

* If the configure script is not properly updated, the build will fail.
The same happens on the Oracle side if the closed version is not in sync
with the open version. It is easy to miss re-generating the script after
the last fix of a typo in the comments in an .m4 file...

* Merging between two changes containing configure modifications is
almost impossible. In practice, the entire generated-configure.sh needs to
be thrown away and regenerated.

The entire benefit of having the file in the repo is to save first-time
developers the hassle of installing autoconf. On most platforms, this is a
no-brainer (like "apt install autoconf"), and the requirement is similar to
other open source projects using autoconf and "./configure". It's just not
worth it.



I'm not convinced just by you saying it is so - sorry. This seems to make
an already complex build process even more complex for every single person
who wants to build OpenJDK, for the benefit of a handful of people who may
want to modify configure options and whom already work closely with the
build team and so there's really little hardship in getting a sponsor, or
just someone with access to autoconf.

It introduces a new point of failure in the build for everyone.

Has this been beta-tested with external contributors? I'd be happier
knowing we've put this through its paces with people developing on a wide
range of platforms, before making it the default.

Have the devkits been updated so I can try this out myself?

Thanks,
David


Bug: https://bugs.openjdk.java.net/browse/JDK-8195689

WebRev: http://cr.openjdk.java.net/~ihse/JDK-8195689-remove-generate
d-configure/webrev.01





/Magnus

Re: RFR: JDK-8195689 Remove generated-configure.sh and instead use autoconf

[Martin Buchholz]

Another possibility is implementing the invariant that configure is
generated via autoconf 2.69 by a mercurial commit hook.

On Thu, Jan 18, 2018 at 10:18 PM, Martin Buchholz 
wrote:

> Differing projects have come to different conclusions about whether to
> include a generated configure.
>
> But the standard seems to be to include one. The mantra is: "./configure
> && make" without an autoconf step.  The number of people building openjdk
> is much larger than the number of people patching configure.  So I agree
> with David that we should stick with the status quo.
>
> On Thu, Jan 18, 2018 at 6:14 PM, David Holmes 
> wrote:
>
>> On 18/01/2018 11:28 PM, Magnus Ihse Bursie wrote:
>>
>>> Currently, we require all developers who modify the configure script to
>>> run autoconf locally, to update the generated-configure.sh script, which is
>>> then checked in. This is the only instance of checked in "compiled" code in
>>> OpenJDK, and this has brought along several problems:
>>>
>>> * Only a specific version of autoconf, 2.69, can be used, to avoid large
>>> code changes in the generated file. Unfortunately, Ubuntu ships a version
>>> of autoconf that claims to be 2.69 but is actually heavily patched. This
>>> requires all Ubuntu users to compiler their own autoconf from source.
>>>
>>> * The Oracle JDK closed sources has a closed version that needs to be
>>> updated. In practice, this has meant that all non-Oracle developers, need
>>> an Oracle sponsor for patches modifying the configure script.
>>>
>>> * If the configure script is not properly updated, the build will fail.
>>> The same happens on the Oracle side if the closed version is not in sync
>>> with the open version. It is easy to miss re-generating the script after
>>> the last fix of a typo in the comments in an .m4 file...
>>>
>>> * Merging between two changes containing configure modifications is
>>> almost impossible. In practice, the entire generated-configure.sh needs to
>>> be thrown away and regenerated.
>>>
>>> The entire benefit of having the file in the repo is to save first-time
>>> developers the hassle of installing autoconf. On most platforms, this is a
>>> no-brainer (like "apt install autoconf"), and the requirement is similar to
>>> other open source projects using autoconf and "./configure". It's just not
>>> worth it.
>>>
>>
>> I'm not convinced just by you saying it is so - sorry. This seems to make
>> an already complex build process even more complex for every single person
>> who wants to build OpenJDK, for the benefit of a handful of people who may
>> want to modify configure options and whom already work closely with the
>> build team and so there's really little hardship in getting a sponsor, or
>> just someone with access to autoconf.
>>
>> It introduces a new point of failure in the build for everyone.
>>
>> Has this been beta-tested with external contributors? I'd be happier
>> knowing we've put this through its paces with people developing on a wide
>> range of platforms, before making it the default.
>>
>> Have the devkits been updated so I can try this out myself?
>>
>> Thanks,
>> David
>>
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8195689
>>> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8195689-remove-generate
>>> d-configure/webrev.01
>>>
>>
>>
>>> /Magnus
>>>
>>
>

Re: RFR: JDK-8195689 Remove generated-configure.sh and instead use autoconf

[Martin Buchholz]

Differing projects have come to different conclusions about whether to
include a generated configure.

But the standard seems to be to include one. The mantra is: "./configure &&
make" without an autoconf step.  The number of people building openjdk is
much larger than the number of people patching configure.  So I agree with
David that we should stick with the status quo.

On Thu, Jan 18, 2018 at 6:14 PM, David Holmes 
wrote:

> On 18/01/2018 11:28 PM, Magnus Ihse Bursie wrote:
>
>> Currently, we require all developers who modify the configure script to
>> run autoconf locally, to update the generated-configure.sh script, which is
>> then checked in. This is the only instance of checked in "compiled" code in
>> OpenJDK, and this has brought along several problems:
>>
>> * Only a specific version of autoconf, 2.69, can be used, to avoid large
>> code changes in the generated file. Unfortunately, Ubuntu ships a version
>> of autoconf that claims to be 2.69 but is actually heavily patched. This
>> requires all Ubuntu users to compiler their own autoconf from source.
>>
>> * The Oracle JDK closed sources has a closed version that needs to be
>> updated. In practice, this has meant that all non-Oracle developers, need
>> an Oracle sponsor for patches modifying the configure script.
>>
>> * If the configure script is not properly updated, the build will fail.
>> The same happens on the Oracle side if the closed version is not in sync
>> with the open version. It is easy to miss re-generating the script after
>> the last fix of a typo in the comments in an .m4 file...
>>
>> * Merging between two changes containing configure modifications is
>> almost impossible. In practice, the entire generated-configure.sh needs to
>> be thrown away and regenerated.
>>
>> The entire benefit of having the file in the repo is to save first-time
>> developers the hassle of installing autoconf. On most platforms, this is a
>> no-brainer (like "apt install autoconf"), and the requirement is similar to
>> other open source projects using autoconf and "./configure". It's just not
>> worth it.
>>
>
> I'm not convinced just by you saying it is so - sorry. This seems to make
> an already complex build process even more complex for every single person
> who wants to build OpenJDK, for the benefit of a handful of people who may
> want to modify configure options and whom already work closely with the
> build team and so there's really little hardship in getting a sponsor, or
> just someone with access to autoconf.
>
> It introduces a new point of failure in the build for everyone.
>
> Has this been beta-tested with external contributors? I'd be happier
> knowing we've put this through its paces with people developing on a wide
> range of platforms, before making it the default.
>
> Have the devkits been updated so I can try this out myself?
>
> Thanks,
> David
>
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8195689
>> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8195689-remove-generate
>> d-configure/webrev.01
>>
>
>
>> /Magnus
>>
>

Re: RFR: JDK-8195689 Remove generated-configure.sh and instead use autoconf

[Erik Joelsson]

Just apply the patch and you can try it. The current version of the patch is 
somewhat broken on windows though, needs another iteration.
/Erik

⁣Sent from BlueMail ​

On Jan 18, 2018, 18:14, at 18:14, David Holmes  wrote:
>On 18/01/2018 11:28 PM, Magnus Ihse Bursie wrote:
>> Currently, we require all developers who modify the configure script
>to
>> run autoconf locally, to update the generated-configure.sh script,
>which
>> is then checked in. This is the only instance of checked in
>"compiled"
>> code in OpenJDK, and this has brought along several problems:
>>
>> * Only a specific version of autoconf, 2.69, can be used, to avoid
>large
>> code changes in the generated file. Unfortunately, Ubuntu ships a
>> version of autoconf that claims to be 2.69 but is actually heavily
>> patched. This requires all Ubuntu users to compiler their own
>autoconf
>> from source.
>>
>> * The Oracle JDK closed sources has a closed version that needs to be
>
>> updated. In practice, this has meant that all non-Oracle developers,
>> need an Oracle sponsor for patches modifying the configure script.
>>
>> * If the configure script is not properly updated, the build will
>fail.
>> The same happens on the Oracle side if the closed version is not in
>sync
>> with the open version. It is easy to miss re-generating the script
>after
>> the last fix of a typo in the comments in an .m4 file...
>>
>> * Merging between two changes containing configure modifications is
>> almost impossible. In practice, the entire generated-configure.sh
>needs
>> to be thrown away and regenerated.
>>
>> The entire benefit of having the file in the repo is to save
>first-time
>> developers the hassle of installing autoconf. On most platforms, this
>is
>> a no-brainer (like "apt install autoconf"), and the requirement is
>> similar to other open source projects using autoconf and
>"./configure".
>> It's just not worth it.
>
>I'm not convinced just by you saying it is so - sorry. This seems to
>make an already complex build process even more complex for every
>single
>person who wants to build OpenJDK, for the benefit of a handful of
>people who may want to modify configure options and whom already work
>closely with the build team and so there's really little hardship in
>getting a sponsor, or just someone with access to autoconf.
>
>It introduces a new point of failure in the build for everyone.
>
>Has this been beta-tested with external contributors? I'd be happier
>knowing we've put this through its paces with people developing on a
>wide range of platforms, before making it the default.
>
>Have the devkits been updated so I can try this out myself?
>
>Thanks,
>David
>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8195689
>> WebRev:
>>
>http://cr.openjdk.java.net/~ihse/JDK-8195689-remove-generated-configure/webrev.01
>
>
>>
>> /Magnus

Re: RFR: JDK-8195689 Remove generated-configure.sh and instead use autoconf

[David Holmes]

On 18/01/2018 11:28 PM, Magnus Ihse Bursie wrote:
Currently, we require all developers who modify the configure script to 
run autoconf locally, to update the generated-configure.sh script, which 
is then checked in. This is the only instance of checked in "compiled" 
code in OpenJDK, and this has brought along several problems:


* Only a specific version of autoconf, 2.69, can be used, to avoid large 
code changes in the generated file. Unfortunately, Ubuntu ships a 
version of autoconf that claims to be 2.69 but is actually heavily 
patched. This requires all Ubuntu users to compiler their own autoconf 
from source.


* The Oracle JDK closed sources has a closed version that needs to be 
updated. In practice, this has meant that all non-Oracle developers, 
need an Oracle sponsor for patches modifying the configure script.


* If the configure script is not properly updated, the build will fail. 
The same happens on the Oracle side if the closed version is not in sync 
with the open version. It is easy to miss re-generating the script after 
the last fix of a typo in the comments in an .m4 file...


* Merging between two changes containing configure modifications is 
almost impossible. In practice, the entire generated-configure.sh needs 
to be thrown away and regenerated.


The entire benefit of having the file in the repo is to save first-time 
developers the hassle of installing autoconf. On most platforms, this is 
a no-brainer (like "apt install autoconf"), and the requirement is 
similar to other open source projects using autoconf and "./configure". 
It's just not worth it.


I'm not convinced just by you saying it is so - sorry. This seems to 
make an already complex build process even more complex for every single 
person who wants to build OpenJDK, for the benefit of a handful of 
people who may want to modify configure options and whom already work 
closely with the build team and so there's really little hardship in 
getting a sponsor, or just someone with access to autoconf.


It introduces a new point of failure in the build for everyone.

Has this been beta-tested with external contributors? I'd be happier 
knowing we've put this through its paces with people developing on a 
wide range of platforms, before making it the default.


Have the devkits been updated so I can try this out myself?

Thanks,
David


Bug: https://bugs.openjdk.java.net/browse/JDK-8195689
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8195689-remove-generated-configure/webrev.01 




/Magnus

Re: RFR: JDK-8195689 Remove generated-configure.sh and instead use autoconf

[Erik Joelsson]

Hello Magnus,

Nice to see this finally happening!

In building.md, when getting autoconf for Cygwin, I believe you also 
need the autoconf wrapper scripts package. Perhaps it's also worth 
mentioning that you can download the autoconf src and build/install from 
there?


In jib-profiles.js, you need to use build_platform instead of 
target_platform when creating the module name for the dependency. I 
would also like if you added the dependency on the other main profiles 
as well. At least linux-x86, linux-arm64 and windows-x86 are still 
possible to build and should work fine with this. For solaris-x64 you 
would need to generate the package for it to work.


The TMPDIR logic looks weird. Are you unconditionally setting it to 
/cygdrive/t/...? That's a valid dir in our build farm, but not on my 
local windows box:


mkdir: cannot create directory ‘/cygdrive/t’: No such file or directory
autom4te: cannot create 
/cygdrive/t/workspace/.build/tmpdir/am4t7009.7040: No such file or directory
 at 
/cygdrive/c/cygwin64/var/tmp/jib-erik/install/jpg/infra/builddeps/autoconf-windows_x64/2.69+1.0.1/autoconf-windows_x64-2.69+1.0.1.tar.gz/usr/bin/autom4te 
line 954.

Error: Failed to generate runnable configure script

/Erik


On 2018-01-18 05:28, Magnus Ihse Bursie wrote:
Currently, we require all developers who modify the configure script 
to run autoconf locally, to update the generated-configure.sh script, 
which is then checked in. This is the only instance of checked in 
"compiled" code in OpenJDK, and this has brought along several problems:


* Only a specific version of autoconf, 2.69, can be used, to avoid 
large code changes in the generated file. Unfortunately, Ubuntu ships 
a version of autoconf that claims to be 2.69 but is actually heavily 
patched. This requires all Ubuntu users to compiler their own autoconf 
from source.


* The Oracle JDK closed sources has a closed version that needs to be 
updated. In practice, this has meant that all non-Oracle developers, 
need an Oracle sponsor for patches modifying the configure script.


* If the configure script is not properly updated, the build will 
fail. The same happens on the Oracle side if the closed version is not 
in sync with the open version. It is easy to miss re-generating the 
script after the last fix of a typo in the comments in an .m4 file...


* Merging between two changes containing configure modifications is 
almost impossible. In practice, the entire generated-configure.sh 
needs to be thrown away and regenerated.


The entire benefit of having the file in the repo is to save 
first-time developers the hassle of installing autoconf. On most 
platforms, this is a no-brainer (like "apt install autoconf"), and the 
requirement is similar to other open source projects using autoconf 
and "./configure". It's just not worth it.


Bug: https://bugs.openjdk.java.net/browse/JDK-8195689
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8195689-remove-generated-configure/webrev.01


/Magnus

Re: RFR: JDK-8195689 Remove generated-configure.sh and instead use autoconf

[Jonathan Gibbons]

On 1/18/18 8:39 AM, Erik Joelsson wrote:



On 2018-01-18 07:42, Jonathan Gibbons wrote:

On 1/18/18 5:28 AM, Magnus Ihse Bursie wrote:



* Only a specific version of autoconf, 2.69, can be used, to avoid 
large code changes in the generated file. Unfortunately, Ubuntu 
ships a version of autoconf that claims to be 2.69 but is actually 
heavily patched. This requires all Ubuntu users to compiler their 
own autoconf from source. 
Can you provide more details here?  My recent experience is that some 
older versions of Ubuntu are OK, with respect to autoconf 2.69.  Is 
there are list somewhere of which versions of Ubuntu are good and 
which are bad?


I know 16.04 is bad and 14.04 is good. Likely other non LTS versions 
are bad as well. The point is though that with this change, we won't 
need to care anymore. The autoconf in Ubuntu still works fine for 
building the product. We just don't like it now because it introduces 
spurious changes in the history for generated-configure.sh.


/Erik


OK, thanks for the clarification.

-- Jon

Re: RFR: JDK-8195689 Remove generated-configure.sh and instead use autoconf

[Erik Joelsson]

On 2018-01-18 07:42, Jonathan Gibbons wrote:

On 1/18/18 5:28 AM, Magnus Ihse Bursie wrote:



* Only a specific version of autoconf, 2.69, can be used, to avoid 
large code changes in the generated file. Unfortunately, Ubuntu ships 
a version of autoconf that claims to be 2.69 but is actually heavily 
patched. This requires all Ubuntu users to compiler their own 
autoconf from source. 
Can you provide more details here?  My recent experience is that some 
older versions of Ubuntu are OK, with respect to autoconf 2.69.  Is 
there are list somewhere of which versions of Ubuntu are good and 
which are bad?


I know 16.04 is bad and 14.04 is good. Likely other non LTS versions are 
bad as well. The point is though that with this change, we won't need to 
care anymore. The autoconf in Ubuntu still works fine for building the 
product. We just don't like it now because it introduces spurious 
changes in the history for generated-configure.sh.


/Erik

Re: RFR: JDK-8195689 Remove generated-configure.sh and instead use autoconf

[Jonathan Gibbons]

On 1/18/18 5:28 AM, Magnus Ihse Bursie wrote:



* Only a specific version of autoconf, 2.69, can be used, to avoid 
large code changes in the generated file. Unfortunately, Ubuntu ships 
a version of autoconf that claims to be 2.69 but is actually heavily 
patched. This requires all Ubuntu users to compiler their own autoconf 
from source. 
Can you provide more details here?  My recent experience is that some 
older versions of Ubuntu are OK, with respect to autoconf 2.69.  Is 
there are list somewhere of which versions of Ubuntu are good and which 
are bad?


-- Jon

Re: RFR: JDK-8195689 Remove generated-configure.sh and instead use autoconf

[Severin Gehwolf]

On Thu, 2018-01-18 at 14:28 +0100, Magnus Ihse Bursie wrote:
> Currently, we require all developers who modify the configure script to 
> run autoconf locally, to update the generated-configure.sh script, which 
> is then checked in. This is the only instance of checked in "compiled" 
> code in OpenJDK, and this has brought along several problems:
> 
> * Only a specific version of autoconf, 2.69, can be used, to avoid large 
> code changes in the generated file. Unfortunately, Ubuntu ships a 
> version of autoconf that claims to be 2.69 but is actually heavily 
> patched. This requires all Ubuntu users to compiler their own autoconf 
> from source.
> 
> * The Oracle JDK closed sources has a closed version that needs to be 
> updated. In practice, this has meant that all non-Oracle developers, 
> need an Oracle sponsor for patches modifying the configure script.
> 
> * If the configure script is not properly updated, the build will fail. 
> The same happens on the Oracle side if the closed version is not in sync 
> with the open version. It is easy to miss re-generating the script after 
> the last fix of a typo in the comments in an .m4 file...
> 
> * Merging between two changes containing configure modifications is 
> almost impossible. In practice, the entire generated-configure.sh needs 
> to be thrown away and regenerated.
> 
> The entire benefit of having the file in the repo is to save first-time 
> developers the hassle of installing autoconf. On most platforms, this is 
> a no-brainer (like "apt install autoconf"), and the requirement is 
> similar to other open source projects using autoconf and "./configure". 
> It's just not worth it.
> 
> Bug: https://bugs.openjdk.java.net/browse/JDK-8195689
> WebRev: 
> http://cr.openjdk.java.net/~ihse/JDK-8195689-remove-generated-configure/webrev.01

Very cool! +1

Note: I haven't really reviewed the patch. This change in approach is
very welcome, though.

Thanks,
Severin

RFR: JDK-8195689 Remove generated-configure.sh and instead use autoconf

[Magnus Ihse Bursie]

Currently, we require all developers who modify the configure script to 
run autoconf locally, to update the generated-configure.sh script, which 
is then checked in. This is the only instance of checked in "compiled" 
code in OpenJDK, and this has brought along several problems:


* Only a specific version of autoconf, 2.69, can be used, to avoid large 
code changes in the generated file. Unfortunately, Ubuntu ships a 
version of autoconf that claims to be 2.69 but is actually heavily 
patched. This requires all Ubuntu users to compiler their own autoconf 
from source.


* The Oracle JDK closed sources has a closed version that needs to be 
updated. In practice, this has meant that all non-Oracle developers, 
need an Oracle sponsor for patches modifying the configure script.


* If the configure script is not properly updated, the build will fail. 
The same happens on the Oracle side if the closed version is not in sync 
with the open version. It is easy to miss re-generating the script after 
the last fix of a typo in the comments in an .m4 file...


* Merging between two changes containing configure modifications is 
almost impossible. In practice, the entire generated-configure.sh needs 
to be thrown away and regenerated.


The entire benefit of having the file in the repo is to save first-time 
developers the hassle of installing autoconf. On most platforms, this is 
a no-brainer (like "apt install autoconf"), and the requirement is 
similar to other open source projects using autoconf and "./configure". 
It's just not worth it.


Bug: https://bugs.openjdk.java.net/browse/JDK-8195689
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8195689-remove-generated-configure/webrev.01


/Magnus