Re: Variable Assignment Consistency

2012-12-06 Thread Chris Penev 
I just found my own answer by reading the relevant part of the gnu make
manual

http://www.gnu.org/software/make/manual/html_node/Shell-Function.html

Sorry for the bother. It seems like make is doing what is documented.

On Thu, Dec 6, 2012 at 2:12 PM, Chris Penev  wrote:

> Dear gnu-make gurus,
>
> I have the following c program (garble.c)
>
> #include 
>
> int main() {
>
> unsigned char buf[256];
> int   off;
>
> for(off = 0; off < sizeof buf; off++)
> buf[off] = off;
>
> return write(1, buf, sizeof buf) != off;
> }
>
>
> and the following makefile (makefile)
>
> E?= $(shell ./garble | tr -d \\000)
> $(error $(E))
>
> Running the following commands, give me result which seem inconsistent to
> me.
>
> $ make 2>&1 | md5sum
> 6bfaed01fc4f24828ce7014b81c4edf1  -
>
> $ make E="$(./garble | tr -d \\000)" 2>&1 | md5sum
> 59d743d40047b58aa7829fd38870f288  -
>
> I would expect there to be no difference between the two hashes. My
> environment is
>
> $ sed q /etc/issue; uname -r; make --version
> Debian GNU/Linux wheezy/sid \n \l
> 3.2.0-4-amd64
> GNU Make 3.81
> Copyright (C) 2006  Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.
> There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
> PARTICULAR PURPOSE.
>
> This program built for x86_64-pc-linux-gnu
>
>
> If instead of piping the output of make to md5sum I convert the output to
> hex, I see that ...
>
> In one case (the first one)
>
>- make converts the newline character to a space.
>
> In the other case (the second one)
>
>- make does not convert the newline character to a space
>- make deletes the characters \044 \045
>   - leading me to think make tried to expand $% as a variable.
>
>
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: timestamp bug when files are created just before make is run

2012-12-06 Thread Mikulas Patocka 


On Thu, 6 Dec 2012, Philip Guenther wrote:

> On Thu, Dec 6, 2012 at 2:02 PM, Mikulas Patocka
>  wrote:
> > The apparent problem is that after make rebuilds b, it compares b's time
> > with a's time, finds that the times are equal (because a was touched just
> > before make was run) and doesn't rebuild a.
> >
> > I think it is a bug - when b is finished, make should find out whether the
> > rule for b modified the file b (if b's time is greater or equal than the
> > time when the rule for b was started, then b may have been modified) and
> > rebuild a in this case.
> 
> That's not the rule that make has always followed and, indeed, is
> required by the standard to follow.  To quote POSIX:
> 
> The make utility attempts to perform the actions required to ensure
> that the specified targets are up-to-date. A target is considered
> out-of-date if it is older than any of its prerequisites or if
> it does not
> exist. The make utility shall treat all prerequisites as targets
> themselves and recursively ensure that they are up-to-date, processing
> them in the order in which they appear in the rule. The make utility
> shall use the modification times of files to determine whether the
> corresponding targets are out-of-date.
> 
> The fact that the timestamp on a prerequisite has changed do *not*
> mean that it's newer than the target; it may have been copied from
> some other file which, while newer, is still older than the target
> file.  So the change to the make algorithm described in the paragraph
> above that starts "I think it is a bug" would not be correct;
> something more precise about what timestamps would be considered
> "newer" would be necessary.
> 
> 
> Note that this problem doesn't arise on systems with high precision
> file timestamps.  Many systems have provided those since the mid 90's;
> I'm appalled that the modern system that process the involved shell
> commands fast enough for this to regularly be a problem don't provide
> microsecond or better timestamp precision.

Yes, but it is very counterintuitive to remake file "b" and not remake 
file "a" that depends on "b" when asked to make file "a". It may be ok 
according to the standard, but not what the user wants.

> > This bug is causing real-world problems in automake-generated Makefiles.
> > This is a simplified piece of Makefile from automake:
> > all : config.h
> > $(MAKE) all-am
> >
> > config.h : stamp-h1
> > echo build config.h
> >
> > stamp-h1 : config.h.in
> > echo build stamp-h1
> > rm -f stamp-h1
> > touch config.h
> > touch stamp-h1
> >
> > config.h.in : am__configure_deps
> > echo build config.h.in
> > rm -f stamp-h1
> > touch config.h.in
> >
> > all-am :
> > echo ALL-AM
> >
> > Now, if you run this script, you trigger the bug:
> >
> > touch config.h.in;sleep 1;touch am__configure_deps;sleep 1;touch config.h 
> > stamp-h1;make
> >
> > - you see "build config.h.in", but the other files are not rebuild and
> > all-am is executed
> > (the essential thing to trigger the bug is that make is run in the same
> > second in which config.h and stamp-h1 were created)
> ...
> > Another possible solution for this bug would be to remove rm -f stamp-h1
> > from config.h.in rule, but there is some complex explanation in
> > /usr/local/share/automake-1.12/am/remake-hdr.am why rm -f stamp-h1 is
> > there so it would likely fix one bug and trigger another one.
> 
> Hmm, adding a "sleep 1" after that rm -f and before the touch
> config.h.in would work without reintroducing the issue described in
> that explanation, no?
> 
> 
> Philip Guenther

Yes it fixes it, you can add "sleep 1" to automake there, that fixes the 
bug.

Mikulas

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: timestamp bug when files are created just before make is run

2012-12-06 Thread Mikulas Patocka 


On Thu, 6 Dec 2012, Philip Guenther wrote:

> Note that this problem doesn't arise on systems with high precision
> file timestamps.  Many systems have provided those since the mid 90's;
> I'm appalled that the modern system that process the involved shell
> commands fast enough for this to regularly be a problem don't provide
> microsecond or better timestamp precision.

BTW. on Linux, high precision timestamps have really kernel-tick 
precision, not nanosecond precision.

So the first example that I posted doesn't always work correctly on Linux 
with 100HZ tick - with sufficiently fast computer, touch "a" and the rule 
to make "b" is executed in the same tick - so "a" isn't rebuilt.

Mikulas

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

[bug #37878] index out of bounds when handling a file contains '(' and not ends with ')'

2012-12-06 Thread junhee cho 
Follow-up Comment #1, bug #37878 (project make):


read.c:3023
if (! (flags & PARSEFS_NOAR)
&& tp == tmpbuf && tp[0] != '(' && tp[nlen-1] != ')')
  ^^
   should be '=='


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/


___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

[bug #37878] index out of bounds when handling a file contains '(' and not ends with ')'

2012-12-06 Thread anonymous 
URL:
  

 Summary: index out of bounds when handling a file contains
'('  and not ends with ')'
 Project: make
Submitted by: None
Submitted on: 2012년 12월 07일 (금) 오전 02시 34분 54초
Severity: 3 - Normal
  Item Group: Bug
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any
   Component Version: 3.82
Operating System: POSIX-Based
   Fixed Release: None
   Triage Status: None

___

Details:

-
read.c:3023

if (! (flags & PARSEFS_NOAR)
&& tp == tmpbuf && tp[0] != '(' && tp[nlen=1] != ')')
  ^^
  should be '=='
-

It occurs indexing out of bounds when handling the following prerequisite
detected. Above if condition becomes true 
even if the file is not a form of archive member reference.


target: this_is_a_target_not_archive_memeber_reference(blah).jpeg
..recipe..


read.c: 3034
e = next_token (e);  // At here, index out of bounds occurs

I've used electric fence (http://perens.com/FreeSoftware/ElectricFence/) for
detecting this.













___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/


___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: timestamp bug when files are created just before make is run

2012-12-06 Thread Philip Guenther 
On Thu, Dec 6, 2012 at 2:02 PM, Mikulas Patocka
 wrote:
> The apparent problem is that after make rebuilds b, it compares b's time
> with a's time, finds that the times are equal (because a was touched just
> before make was run) and doesn't rebuild a.
>
> I think it is a bug - when b is finished, make should find out whether the
> rule for b modified the file b (if b's time is greater or equal than the
> time when the rule for b was started, then b may have been modified) and
> rebuild a in this case.

That's not the rule that make has always followed and, indeed, is
required by the standard to follow.  To quote POSIX:

The make utility attempts to perform the actions required to ensure
that the specified targets are up-to-date. A target is considered
out-of-date if it is older than any of its prerequisites or if
it does not
exist. The make utility shall treat all prerequisites as targets
themselves and recursively ensure that they are up-to-date, processing
them in the order in which they appear in the rule. The make utility
shall use the modification times of files to determine whether the
corresponding targets are out-of-date.

The fact that the timestamp on a prerequisite has changed do *not*
mean that it's newer than the target; it may have been copied from
some other file which, while newer, is still older than the target
file.  So the change to the make algorithm described in the paragraph
above that starts "I think it is a bug" would not be correct;
something more precise about what timestamps would be considered
"newer" would be necessary.


Note that this problem doesn't arise on systems with high precision
file timestamps.  Many systems have provided those since the mid 90's;
I'm appalled that the modern system that process the involved shell
commands fast enough for this to regularly be a problem don't provide
microsecond or better timestamp precision.


> This bug is causing real-world problems in automake-generated Makefiles.
> This is a simplified piece of Makefile from automake:
> all : config.h
> $(MAKE) all-am
>
> config.h : stamp-h1
> echo build config.h
>
> stamp-h1 : config.h.in
> echo build stamp-h1
> rm -f stamp-h1
> touch config.h
> touch stamp-h1
>
> config.h.in : am__configure_deps
> echo build config.h.in
> rm -f stamp-h1
> touch config.h.in
>
> all-am :
> echo ALL-AM
>
> Now, if you run this script, you trigger the bug:
>
> touch config.h.in;sleep 1;touch am__configure_deps;sleep 1;touch config.h 
> stamp-h1;make
>
> - you see "build config.h.in", but the other files are not rebuild and
> all-am is executed
> (the essential thing to trigger the bug is that make is run in the same
> second in which config.h and stamp-h1 were created)
...
> Another possible solution for this bug would be to remove rm -f stamp-h1
> from config.h.in rule, but there is some complex explanation in
> /usr/local/share/automake-1.12/am/remake-hdr.am why rm -f stamp-h1 is
> there so it would likely fix one bug and trigger another one.

Hmm, adding a "sleep 1" after that rm -f and before the touch
config.h.in would work without reintroducing the issue described in
that explanation, no?


Philip Guenther

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: Variable Assignment Consistency

2012-12-06 Thread Paul Smith 
On Thu, 2012-12-06 at 14:12 -0800, Chris Penev wrote:
> I would expect there to be no difference between the two hashes.

That expectation would not be correct, clearly :-).  Make has its own
escaping and expanding procedures that it follows, in addition to /
aside from what the shell does.

> If instead of piping the output of make to md5sum I convert the output
> to hex, I see that ...
> 
> In one case (the first one) 
>   * make converts the newline character to a space.
> In the other case (the second one) 
>   * make does not convert the newline character to a space 
>   * make deletes the characters \044 \045 
>   * leading me to think make tried to expand $% as
> a variable. 

You have perfectly captured the differences.  They are easily
explainable.

>From the GNU make manual description of the $(shell ...) function:

   The `shell' function performs the same function that backquotes
(``') perform in most shells: it does "command expansion".  This means
that it takes as an argument a shell command and evaluates to the
output of the command.  The only processing `make' does on the result
is to convert each newline (or carriage-return / newline pair) to a
single space.  If there is a trailing (carriage-return and) newline it
will simply be removed.

In the second case, you are assigning a value to a variable (that is,
make sees just an assignment like "E=abcedfg$%hijk" or whatever, and
values are always expanded.


___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Variable Assignment Consistency

2012-12-06 Thread Chris Penev 
Dear gnu-make gurus,

I have the following c program (garble.c)

#include 

int main() {

unsigned char buf[256];
int   off;

for(off = 0; off < sizeof buf; off++)
buf[off] = off;

return write(1, buf, sizeof buf) != off;
}


and the following makefile (makefile)

E?= $(shell ./garble | tr -d \\000)
$(error $(E))

Running the following commands, give me result which seem inconsistent to
me.

$ make 2>&1 | md5sum
6bfaed01fc4f24828ce7014b81c4edf1  -

$ make E="$(./garble | tr -d \\000)" 2>&1 | md5sum
59d743d40047b58aa7829fd38870f288  -

I would expect there to be no difference between the two hashes. My
environment is

$ sed q /etc/issue; uname -r; make --version
Debian GNU/Linux wheezy/sid \n \l
3.2.0-4-amd64
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for x86_64-pc-linux-gnu


If instead of piping the output of make to md5sum I convert the output to
hex, I see that ...

In one case (the first one)

   - make converts the newline character to a space.

In the other case (the second one)

   - make does not convert the newline character to a space
   - make deletes the characters \044 \045
  - leading me to think make tried to expand $% as a variable.
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

timestamp bug when files are created just before make is run

2012-12-06 Thread Mikulas Patocka 
Hi

Try this Makefile:
---
a : b
echo build a
touch a

b : c
echo build b
touch b
---
and run this script:
touch b;sleep 1;touch a c;make

You see
"echo build b
build b
touch b"
but it doesn't remake a.

The apparent problem is that after make rebuilds b, it compares b's time 
with a's time, finds that the times are equal (because a was touched just 
before make was run) and doesn't rebuild a.

I think it is a bug - when b is finished, make should find out whether the 
rule for b modified the file b (if b's time is greater or equal than the 
time when the rule for b was started, then b may have been modified) and 
rebuild a in this case.

---

This bug is causing real-world problems in automake-generated Makefiles. 
This is a simplified piece of Makefile from automake:
all : config.h
$(MAKE) all-am

config.h : stamp-h1
echo build config.h

stamp-h1 : config.h.in
echo build stamp-h1
rm -f stamp-h1
touch config.h
touch stamp-h1

config.h.in : am__configure_deps
echo build config.h.in
rm -f stamp-h1
touch config.h.in

all-am :
echo ALL-AM

Now, if you run this script, you trigger the bug:

touch config.h.in;sleep 1;touch am__configure_deps;sleep 1;touch config.h 
stamp-h1;make

- you see "build config.h.in", but the other files are not rebuild and 
all-am is executed
(the essential thing to trigger the bug is that make is run in the same 
second in which config.h and stamp-h1 were created)


The problem that really happens in a real build:

* The user runs autoheader && aclocal && automake && autoconf && ./configure && 
make -j4
* Configure runs ./config.status, ./config.status writes config.h and 
stamp-h1
* Make sees that am__configure_deps is newer than config.h.in
* Make runs the rule for config.h.in. It sets the new timestamp for 
config.h.in and deletes stamp-h1
* Now make sees that config.h.in has the same time as stamp-h1 (the 
timestamp is read from make's cache even if stamp-h1 no longer exists)
* Because of the same timestamp, make doesn't run the commands for 
stamp-h1 and config.h
* Make executes a subprocess to build the rule all-am

* The subprocess doesn't have the file cache of the parent process, so the 
subprocess knows that stamp-h1 is missing
* The subprocess sees a dependency config.h->stamp-h1, stamp-h1 doesn't 
exist
* The subprocess executes the rule for stamp-h1 which regenerates config.h 
- the problem is that this rule is executed in parallel with other rules 
that build C files that include config.h - changing config.h while it is 
being used results in build failure

Another possible solution for this bug would be to remove rm -f stamp-h1 
from config.h.in rule, but there is some complex explanation in 
/usr/local/share/automake-1.12/am/remake-hdr.am why rm -f stamp-h1 is 
there so it would likely fix one bug and trigger another one.

So it would be better to fix this in make.

Mikulas

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make