Re: use of math.h / libmath discouraged?

[Edward Welbourne]

On Wed, 2018-07-25 at 14:25 -0600, Brian Vandenberg wrote:
>> # note: the space before the word TEXT is helpful for readability but causes 
>> a problem
>> $(info $(call F1, TEXT))

Paul Smith (28 July 2018 14:48) replied:
> To me this seems like a bug.  In GNU make generally the rule is
> (intended to be?) that leading whitespace is always ignored and
> trailing whitespace is always preserved.
>
> However if leading whitespace is currently preserved in call functions,
> even unintentionally, I'll need to consider whether that can be fixed
> or not due to backward-compatibility issues.

I'm fairly confident this is how call functions have been since at least
2006.  I would be in favour of applying the rule you just described
instead - I tripped over this repeatedly, back when I spent more time
maintaining make-files, and working round it felt unnatural and made
code less readable,

Eddy.

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: use of math.h / libmath discouraged?

[Paul Smith]

On Wed, 2018-07-25 at 14:25 -0600, Brian Vandenberg wrote:
> > However, I'm not sure that this function is sufficiently generic to
> > be added as a built-in function.  Can you provide use-cases for it?
> 
> I suppose that depends on the definition of "sufficiently generic". 
> The two definitions that come to mind:

Have you considered creating a loadable module for this?  That seems
like it would be a straightforward and flexible way to manage this.

Docs: https://www.gnu.org/software/make/manual/html_node/Loading-Objects.html

> # note: the space before the word TEXT is helpful for readability but causes 
> a problem
> $(info $(call F1, TEXT))

To me this seems like a bug.  In GNU make generally the rule is
(intended to be?) that leading whitespace is always ignored and
trailing whitespace is always preserved.

However if leading whitespace is currently preserved in call functions,
even unintentionally, I'll need to consider whether that can be fixed
or not due to backward-compatibility issues.

Cheers!



___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: use of math.h / libmath discouraged?

[Eddy Petrișor]

joi, 26 iul. 2018, 19:42 Brian Vandenberg  a
scris:

> Wouldn't profiling information help?
>> See details and link to code with 3.81 that does this at:
>>
>
> Yes, it would.  I was unaware of that issue.
>
> It looks as though that feature never made it in [unfortunately].  If it
> were added, was present in a default build and there were a way to
> enable/disable it dynamically then what I'm proposing here wouldn't be much
> of an improvement (if any).
>

Yes, unfortunately, I did not get any feedback from the community except
the initial one suggesting some unusable, IMO, output, while I was focusing
on the usability of the feature without involving a long chain of external
tools (raw output and LibreOffice Calc were enough in both my proposals).

There were since other minor portability issue which I was planning to
address once the main design was agreed, but that did not happened, stated
using another build tool which had profiling info and I lost interest in
improving GNU make.

The enabling of the profiling feature was triggered by the passing of the
command line parameter and was done at rule level and start and end of the
make process. Finer granularity was not considered.


>
> When I say "enable it dynamically" I mean something along these lines:
>
> $(profile_options ...formatting/level/etc goes here...)
>> # or
>> .PROFILE_OPTIONS := ...ditto...
>> $(enable profiling)
>> ...some expressions not at recipe scope...
>> $(disable profiling)
>> $(profile some expression)
>> .RECIPEPREFIX := >
>> some_target:
>> > $(profile ${CC} ${FLAGS} ${<} -c -o ${@})
>
>
> In lieu of that, what I'm proposing is worth adding in that it's not
> difficult to use, its footprint is small and can give insight that's
> otherwise difficult to glean.
>

I have no problems with being able to profile a build, I know the pain of
managing a large build system and having no insight into the performance
bottle necks.
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: use of math.h / libmath discouraged?

[Brian Vandenberg]

>
> Wouldn't profiling information help?
> See details and link to code with 3.81 that does this at:
>

Yes, it would.  I was unaware of that issue.

It looks as though that feature never made it in [unfortunately].  If it
were added, was present in a default build and there were a way to
enable/disable it dynamically then what I'm proposing here wouldn't be much
of an improvement (if any).

When I say "enable it dynamically" I mean something along these lines:

$(profile_options ...formatting/level/etc goes here...)
> # or
> .PROFILE_OPTIONS := ...ditto...
> $(enable profiling)
> ...some expressions not at recipe scope...
> $(disable profiling)
> $(profile some expression)
> .RECIPEPREFIX := >
> some_target:
> > $(profile ${CC} ${FLAGS} ${<} -c -o ${@})


In lieu of that, what I'm proposing is worth adding in that it's not
difficult to use, its footprint is small and can give insight that's
otherwise difficult to glean.

-brian
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: use of math.h / libmath discouraged?

[Eddy Petrișor]

Eddy

mie., 25 iul. 2018, 23:26 Brian Vandenberg  a
scris:

> However, I'm not sure that this function is sufficiently generic to be
>>
> added as a built-in function.  Can you provide use-cases for it?
>>
>
> I suppose that depends on the definition of "sufficiently generic".  The
> two definitions that come to mind:
>
>
>1. Useful for many people who do build maintenance
>2. Useful for people trying to improve performance on large builds or
>on makefile "libraries" (eg gmsl  or
>libmakefile )
>
>
Wouldn't profiling information help?
See details and link to code with 3.81 that does this at:

https://savannah.gnu.org/bugs/?40639

Eddy
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make

Re: use of math.h / libmath discouraged?

[Brian Vandenberg]

>
> However, I'm not sure that this function is sufficiently generic to be
>
added as a built-in function.  Can you provide use-cases for it?
>

I suppose that depends on the definition of "sufficiently generic".  The
two definitions that come to mind:


   1. Useful for many people who do build maintenance
   2. Useful for people trying to improve performance on large builds or on
   makefile "libraries" (eg gmsl  or
   libmakefile )


If the chosen definition is (1) then you're right, but I think it's safe to
say most people in (2) would find it useful (or would have).

I'll try to give some good examples, though I came up with this years ago
and I'm just getting around to contributing it.

---

One of the more annoying problems I've encountered when writing canned
expressions is whitespace creeping in.  Here's a contrived example with a
potential solution:

$ cat makefile
> F1 = ${${1}}
> F2 = $(call ${0}_,$(strip ${1}))
> F2_ = ${${1}}
> TEXT = something
> # note: the space before the word *TEXT* is helpful for readability but
> causes a problem
> $(info $(call F1, TEXT))
> $(warning $(call F2, TEXT))
> $ make --warn-undefined-variables
> makefile:6: warning: undefined variable ' TEXT'
> makefile:7: something


Adding that extra level of indirection makes it easier to write the
expression without worrying about unexpected whitespace but there's a cost
that's easy to ignore if you don't have a straightforward way to measure it.

---

One of the bottlenecks I uncovered in our build looked something like this:

.RECIPEPREFIX := >
some_target:
> ${CC} *${SOMETHING_EXPENSIVE}* ${OTHER_FLAGS} ${^} -o ${@}

The documentation says it but I didn't fully appreciate it until diving
into this problem: the entire recipe undergoes expansion before a subshell
gets created (either that or it occurs after vfork() and before execve(),
but the effect is still the same).  The non-obvious [to me] consequence:
expensive operations in a recipe can artificially limit the number of jobs
make can spawn/reap since time spent on recipe expansion is time not spent
reaping/spawning jobs.  The impact isn't as bad if the build is recursive,
but that doesn't apply to us.

---

After I created the *timeit* function I spent a few days coming up with /
looking for awful contrivances to help me better understand what effect
various expressions might have on time spent parsing them.  Here's a few I
remember, though I cannot guarantee these are functional as written -- I'm
writing from memory without testing them:

# Recursion:
> F1 = $(strip $(if $(filter 16,${3}),${2},$(call ${0},${1},${${1}}
> ${2},$(words ${2}
> TEST = something
> # prints the word "something" 16 times
> $(info $(call F1,TEST,,))


# Automatically generate a target
> define MAKE_TARGET
> $(eval ${1}.o := $(addsuffix .o, $(basename $(wildcard ${2}
> # evals needed because of *${${1}.o}*
> $(eval ${1} : ${${1}.o})
> # ... whatever other strange contrivance comes to mind ...
> endef


# recursively expanded variable (B) with partial expansion.
> A = a
> B = ${A}
> $(eval B=$(value B) ${B})
> # prints: ${A} a
> $(info $(value B))


# variable reflection
> THING = $(call ${0}_,$(filter $(firstword $(notdir $(basename
> ${1}))).$(strip ${2}),${.VARIABLES}),$(strip ${2}))
> THING_ = ${$(strip $(if ${1},${1},${2}))}
> # Depending on its existence either *bar.CFLAGS* or *CFLAGS* gets
> evaluated
> bar : ; ${CC} $(call THING, ${@}, CFLAGS) ${^} -o ${@}


# Conditional evaluation of macro arguments; *ifdef/ifndef* is from our
> plugin

F = $(ifndef 3,$(error ${0} requires 3 arguments))...stuff...


Testing the effect they have on performance could be tricky.  Without a way
to evaluate timing of specific expressions you're limited to:


   1. Build make / attach a profiler / figure out how to attribute
   performance changes to expressions used in the makefile
   2. Use *time* (or something similar) / establish a baseline by removing
   the code in question
   3. Cook up some makefile hackery that in some way measures time spent
   evaluating the expression


(1) is probably not realistic.  With (2): if expression A causes expression
B to have performance problems, but B is in your baseline then the
performance loss will be attributed to A whereas the problem may lie in B.

Without a builtin or plugin function, (3) amounts to something like this:

NOW = $(shell date +%s%N)
> before := ${NOW}
> ${EXPRESSION}
> after := ${NOW}
> $(info difference: $(shell expr ${after} - ${before}))


... which has limitations:


   - Relies on *$(shell)* which may skew/bias the results (though to be
   fair a builtin/plugin function could be added that returns a high
   resolution timestamp)
   - Would be more complicated to apply this to recipe parsing,
   *.SECONDEXPANSION* and other situations where temporary variables may be
   an issue, though you could get around it with something like: *$(info
   

Re: use of math.h / libmath discouraged?

[Paul Smith]

On Tue, 2018-07-24 at 16:04 -0600, Brian Vandenberg wrote:
> In a plugin I wrote I created a function with the following form /
> function:
> 
> $ cat makefile
> ASDF := something
> $(timeit 100, ${ASDF})
> $ make
> makefile:2 Time taken: mean: 232, stdev: 81.719765
> 
> make: *** No targets.  Stop.
> 
> ... where "timeit" is implemented using the online mean/variance
> estimate from Knuth's TAoCP.  It evaluates the 2nd argument a number
> of times equal to what the 1st argument evaluates to then spits out a
> mean / std deviation across those runs.
> 
> I want to contribute this function back to the make project, however
> there's a problem if it gets accepted: calculating the standard
> deviation requires calling sqrt from math.h / libmath.
> 
> Am I safe to assume that adding a dependence on math.h / libmath
> isn't an option?

IIRC, math.h / sqrt() is defined in the ISO C90 standard and so it
should be portable.

However, I'm not sure that this function is sufficiently generic to be
added as a built-in function.  Can you provide use-cases for it?

Cheers!

___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make