Re: RFR: JDK-8196356: Changes to m4 files don't trigger autoconf execution at build time

[Magnus Ihse Bursie]

On 2018-02-10 12:29, Thomas Stüfe wrote:

On Sat, Feb 10, 2018 at 9:12 AM, Alan Bateman 
wrote:


On 08/02/2018 17:49, Erik Joelsson wrote:


The check for when to generate the generated configure script is not
working quite as expected. It currently only compares timestamps if the
local repository has any local changes in the make/autoconf directory. This
used to make sense when we had a committed generated script, but now we
actually do need to regenerate any time an input file is newer than the
generated script.

Bug: https://bugs.openjdk.java.net/browse/JDK-8196356


In addition to `hg status` showing no changes, I think it will continue to
confuse people to generate it into a hidden directory. Was there any
consideration to generating into a regular directory?



I agree. Also, we still generate the configure.sh into the source tree even
if the output directory is somewhere else. I always keep my output
directories separate from the source tree. Sometimes my source directory is
even on a read-only share. I would prefer and also expect any temporary
files to be placed in the output directory resp. the current directory, not
in the source tree. Would that be possible?
As discussed in another thread, the current behaviour is no good when 
running configure outside the source tree root. We should definitely fix 
that (I thought Volker were going to publish a RFR but that didn't 
happen; I'll pick up the ball and submit a RFR). With that in place, the 
behaviour will in principle be that the output is in the current directory.


/Magnus




Thanks and Kind Regards, Thomas



-Alan

Re: RFR: JDK-8196356: Changes to m4 files don't trigger autoconf execution at build time

[Thomas Stüfe]

Hi Magnus,

On Mon, Feb 12, 2018 at 12:50 PM, Magnus Ihse Bursie <
magnus.ihse.bur...@oracle.com> wrote:

> On 2018-02-10 23:39, Martin Buchholz wrote:
>
>> I agree.  Once you make something lazy-initted you have a concurrency
>> problem.  And there's no CAS or lock on the filesystem.  What happens if
>> two configure processes run at exactly the same time, perhaps even with
>> different versions of autoconf?  If you lazy-generate configure, it must
>> be
>> written outside the source tree.
>>
> That is of course a problem with the entire build system, and is certainly
> not unique to this problem. What happens if you run "make" concurrently in
> the same directory? Catastrophe! Or if you run "configure" at the same
> time? Disaster! Creating the file outside the source tree will not help in
> the slightest regard; you can just as well create a race condition in any
> place you select. This does not sound like a real-world problem that one
> has to safe-guard against.
>
>
Until now it was perfectly fine to run N makes in parallel from the same
source tree. I do this all the time: I have a source directory and a number
of output directories side by side for various build configurations (debug,
fastdebug, release, 32bit, zero...); in the morning I pull all changes and
start a number of scratch builds. We also do this nightly. We share source
trees for multiple builds (among other reasons it makes comparing builds
easier).

But since now the configuration is written by the build into the source
tree, parallel builds can overwrite each others generated-configure shell
script.

As for your other mail, this issue here (generating stuff into the source
tree) is for me a far greater pain than the fact that the directory is
hidden; the latter is just aesthetics, and as you say, we already do this
in other cases.

Best Regards, Thomas




> /Magnus
>
>
>
>> On Sat, Feb 10, 2018 at 3:29 AM, Thomas Stüfe 
>> wrote:
>>
>> On Sat, Feb 10, 2018 at 9:12 AM, Alan Bateman 
>>> wrote:
>>>
>>> On 08/02/2018 17:49, Erik Joelsson wrote:

 The check for when to generate the generated configure script is not
> working quite as expected. It currently only compares timestamps if the
> local repository has any local changes in the make/autoconf directory.
>
 This
>>>
 used to make sense when we had a committed generated script, but now we
> actually do need to regenerate any time an input file is newer than the
> generated script.
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8196356
>
> In addition to `hg status` showing no changes, I think it will continue

>>> to
>>>
 confuse people to generate it into a hidden directory. Was there any
 consideration to generating into a regular directory?


 I agree. Also, we still generate the configure.sh into the source tree
>>> even
>>> if the output directory is somewhere else. I always keep my output
>>> directories separate from the source tree. Sometimes my source directory
>>> is
>>> even on a read-only share. I would prefer and also expect any temporary
>>> files to be placed in the output directory resp. the current directory,
>>> not
>>> in the source tree. Would that be possible?
>>>
>>> Thanks and Kind Regards, Thomas
>>>
>>>
>>> -Alan


>

Re: RFR: JDK-8196356: Changes to m4 files don't trigger autoconf execution at build time

[Alan Bateman]

On 12/02/2018 11:42, Magnus Ihse Bursie wrote:


Why would hidden files be confusing? Or, rephrased, how is this any 
more confusing than other hidden directories, such as .idea or .jib? I 
modelled the behaviour on jib. In fact, I was considering using the 
same directory (.jib), but since the name was so specialized, it felt 
like misusing it. That's why I gave it a more general name. (In fact, 
I believe, if .build had been present when jib was created, it would 
have used that instead of creating the specialized .jib directory).
I'm aware of at least 10 people that ran into problems in the latter 
part of last week because they didn't know about the .build directory. I 
think most people are used to blowing away the build directory, the 
.build directory is new so we need to learn to blow away that too 
(irrespective of whether the build re-generated when it seems the time 
stamp is old). All I'm saying is that hidden directories have a tenancy 
to confuse. I can't comment on .jib as that is not in OpenJDK.


-Alan

Re: RFR: JDK-8196356: Changes to m4 files don't trigger autoconf execution at build time

[Magnus Ihse Bursie]

On 2018-02-10 23:39, Martin Buchholz wrote:

I agree.  Once you make something lazy-initted you have a concurrency
problem.  And there's no CAS or lock on the filesystem.  What happens if
two configure processes run at exactly the same time, perhaps even with
different versions of autoconf?  If you lazy-generate configure, it must be
written outside the source tree.
That is of course a problem with the entire build system, and is 
certainly not unique to this problem. What happens if you run "make" 
concurrently in the same directory? Catastrophe! Or if you run 
"configure" at the same time? Disaster! Creating the file outside the 
source tree will not help in the slightest regard; you can just as well 
create a race condition in any place you select. This does not sound 
like a real-world problem that one has to safe-guard against.


/Magnus



On Sat, Feb 10, 2018 at 3:29 AM, Thomas Stüfe 
wrote:


On Sat, Feb 10, 2018 at 9:12 AM, Alan Bateman 
wrote:


On 08/02/2018 17:49, Erik Joelsson wrote:


The check for when to generate the generated configure script is not
working quite as expected. It currently only compares timestamps if the
local repository has any local changes in the make/autoconf directory.

This

used to make sense when we had a committed generated script, but now we
actually do need to regenerate any time an input file is newer than the
generated script.

Bug: https://bugs.openjdk.java.net/browse/JDK-8196356


In addition to `hg status` showing no changes, I think it will continue

to

confuse people to generate it into a hidden directory. Was there any
consideration to generating into a regular directory?



I agree. Also, we still generate the configure.sh into the source tree even
if the output directory is somewhere else. I always keep my output
directories separate from the source tree. Sometimes my source directory is
even on a read-only share. I would prefer and also expect any temporary
files to be placed in the output directory resp. the current directory, not
in the source tree. Would that be possible?

Thanks and Kind Regards, Thomas



-Alan

Re: RFR: JDK-8196356: Changes to m4 files don't trigger autoconf execution at build time

[Magnus Ihse Bursie]

On 2018-02-10 09:12, Alan Bateman wrote:

On 08/02/2018 17:49, Erik Joelsson wrote:
The check for when to generate the generated configure script is not 
working quite as expected. It currently only compares timestamps if 
the local repository has any local changes in the make/autoconf 
directory. This used to make sense when we had a committed generated 
script, but now we actually do need to regenerate any time an input 
file is newer than the generated script.


Bug: https://bugs.openjdk.java.net/browse/JDK-8196356
In addition to `hg status` showing no changes, I think it will 
continue to confuse people to generate it into a hidden directory. Was 
there any consideration to generating into a regular directory?


Why would hidden files be confusing? Or, rephrased, how is this any more 
confusing than other hidden directories, such as .idea or .jib? I 
modelled the behaviour on jib. In fact, I was considering using the same 
directory (.jib), but since the name was so specialized, it felt like 
misusing it. That's why I gave it a more general name. (In fact, I 
believe, if .build had been present when jib was created, it would have 
used that instead of creating the specialized .jib directory).


/Magnus

Re: RFR: JDK-8196356: Changes to m4 files don't trigger autoconf execution at build time

[Martin Buchholz]

I agree.  Once you make something lazy-initted you have a concurrency
problem.  And there's no CAS or lock on the filesystem.  What happens if
two configure processes run at exactly the same time, perhaps even with
different versions of autoconf?  If you lazy-generate configure, it must be
written outside the source tree.

On Sat, Feb 10, 2018 at 3:29 AM, Thomas Stüfe 
wrote:

> On Sat, Feb 10, 2018 at 9:12 AM, Alan Bateman 
> wrote:
>
> > On 08/02/2018 17:49, Erik Joelsson wrote:
> >
> >> The check for when to generate the generated configure script is not
> >> working quite as expected. It currently only compares timestamps if the
> >> local repository has any local changes in the make/autoconf directory.
> This
> >> used to make sense when we had a committed generated script, but now we
> >> actually do need to regenerate any time an input file is newer than the
> >> generated script.
> >>
> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8196356
> >>
> > In addition to `hg status` showing no changes, I think it will continue
> to
> > confuse people to generate it into a hidden directory. Was there any
> > consideration to generating into a regular directory?
> >
> >
> I agree. Also, we still generate the configure.sh into the source tree even
> if the output directory is somewhere else. I always keep my output
> directories separate from the source tree. Sometimes my source directory is
> even on a read-only share. I would prefer and also expect any temporary
> files to be placed in the output directory resp. the current directory, not
> in the source tree. Would that be possible?
>
> Thanks and Kind Regards, Thomas
>
>
> > -Alan
> >
>

Re: RFR: JDK-8196356: Changes to m4 files don't trigger autoconf execution at build time

[Thomas Stüfe]

On Sat, Feb 10, 2018 at 9:12 AM, Alan Bateman 
wrote:

> On 08/02/2018 17:49, Erik Joelsson wrote:
>
>> The check for when to generate the generated configure script is not
>> working quite as expected. It currently only compares timestamps if the
>> local repository has any local changes in the make/autoconf directory. This
>> used to make sense when we had a committed generated script, but now we
>> actually do need to regenerate any time an input file is newer than the
>> generated script.
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8196356
>>
> In addition to `hg status` showing no changes, I think it will continue to
> confuse people to generate it into a hidden directory. Was there any
> consideration to generating into a regular directory?
>
>
I agree. Also, we still generate the configure.sh into the source tree even
if the output directory is somewhere else. I always keep my output
directories separate from the source tree. Sometimes my source directory is
even on a read-only share. I would prefer and also expect any temporary
files to be placed in the output directory resp. the current directory, not
in the source tree. Would that be possible?

Thanks and Kind Regards, Thomas


> -Alan
>

Re: RFR: JDK-8196356: Changes to m4 files don't trigger autoconf execution at build time

[Alan Bateman]

On 08/02/2018 17:49, Erik Joelsson wrote:
The check for when to generate the generated configure script is not 
working quite as expected. It currently only compares timestamps if 
the local repository has any local changes in the make/autoconf 
directory. This used to make sense when we had a committed generated 
script, but now we actually do need to regenerate any time an input 
file is newer than the generated script.


Bug: https://bugs.openjdk.java.net/browse/JDK-8196356
In addition to `hg status` showing no changes, I think it will continue 
to confuse people to generate it into a hidden directory. Was there any 
consideration to generating into a regular directory?


-Alan

Re: RFR: JDK-8196356: Changes to m4 files don't trigger autoconf execution at build time

[Tim Bell]

Erik:


The check for when to generate the generated configure script is not
working quite as expected. It currently only compares timestamps if the
local repository has any local changes in the make/autoconf directory.
This used to make sense when we had a committed generated script, but
now we actually do need to regenerate any time an input file is newer
than the generated script.

Bug: https://bugs.openjdk.java.net/browse/JDK-8196356

Webrev: http://cr.openjdk.java.net/~erikj/8196356/webrev.01/


Yes, that makes sense.  Looks good to me.

/Tim

RFR: JDK-8196356: Changes to m4 files don't trigger autoconf execution at build time

[Erik Joelsson]

The check for when to generate the generated configure script is not 
working quite as expected. It currently only compares timestamps if the 
local repository has any local changes in the make/autoconf directory. 
This used to make sense when we had a committed generated script, but 
now we actually do need to regenerate any time an input file is newer 
than the generated script.


Bug: https://bugs.openjdk.java.net/browse/JDK-8196356

Webrev: http://cr.openjdk.java.net/~erikj/8196356/webrev.01/

/Erik