Re: [gentoo-dev] Re: cmake + ninja vs autotools

[Peter Stuge]

William L. Thomson Jr. wrote:
> Or uber minimal, can't get much smaller still 5.20s
> https://travis-ci.org/Obsidian-StudiosInc/asspr#L165
> https://github.com/Obsidian-StudiosInc/asspr/blob/master/configure.ac

Consider

AM_INIT_AUTOMAKE([foreign no-define no-installinfo no-installman])

foreign in particular if you would like to skip the various UPPERCASE
files in the project root.


//Peter

Re: [gentoo-dev] Re: cmake + ninja vs autotools

[Peter Stuge]

William L. Thomson Jr. wrote:
> Or uber minimal, can't get much smaller still 5.20s
> https://travis-ci.org/Obsidian-StudiosInc/asspr#L165
> https://github.com/Obsidian-StudiosInc/asspr/blob/master/configure.ac

That takes 2.0s here, on quite old hardware, though with primed cache.


> #secondsmatter :)

Yeah! :)


> The dependency aspect I agree with 100%. I think even cmake has more.

cmake itself is the dependency. ;)


> Or who cares about end users, its all about saving devs time, no clue.
> #mesonbandwagon

End users generally consume behind a distribution build process, so
yes, it's all optimization for development time, which makes some sense.


//Peter

Re: [gentoo-dev] Re: cmake + ninja vs autotools

[William L. Thomson Jr.]

On Sat, 25 Nov 2017 00:20:38 +
Peter Stuge  wrote:

> William L. Thomson Jr. wrote:
> > I cannot understand why systems get faster, yet configure seems to
> > take the same amount of time and is super slow.  
> 
> The generated configure scripts can be fork intensive, which is still
> fairly expensive.
> 
> But I think the problem is more with poorly written configure source,
> which is the argument about mastering..

Not sure what can be done for minimal ones.

Meson 1.20s
https://travis-ci.org/Obsidian-StudiosInc/entrance/builds/291848344#L1033

Before I switched to meson
Configure/autotools 5.52s
https://travis-ci.org/Obsidian-StudiosInc/entrance/builds/270985567#L963

Original configure I dropped for meson
https://git.enlightenment.org/misc/entrance.git/tree/configure.ac


Or uber minimal, can't get much smaller still 5.20s
https://travis-ci.org/Obsidian-StudiosInc/asspr#L165
https://github.com/Obsidian-StudiosInc/asspr/blob/master/configure.ac

#secondsmatter :)

> > On small projects configure can take longer than compile...
> > Configure is my main gripe against make/autotools. Plus all the
> > other stuff, auto-reconf, autogen, etc.  
> 
> configure having zero dependencies is the killer feature compared
> to all other options. The tight integration between configure and
> cross-toolchains is also a very strong point.

The dependency aspect I agree with 100%. I think even cmake has more.
Meson requires python, so that alone has a big dependency chain. Which
some what surprises me so many with libraries are going that route. I
guess they figure it is not core, python likely to be present.

Or who cares about end users, its all about saving devs time, no clue.
#mesonbandwagon

> > The larger the project, the slower configure can be.  
> 
> Doesn't have to be, but it's easy to write poor configure source and
> difficult to write good source.

I would assume then most are poorly written, as I have yet to see a
fast configure. Beyond say the one for asspr.

-- 
William L. Thomson Jr.


pgp7QIqKdPlHt.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: cmake + ninja vs autotools

[Peter Stuge]

William L. Thomson Jr. wrote:
> I cannot understand why systems get faster, yet configure seems to
> take the same amount of time and is super slow.

The generated configure scripts can be fork intensive, which is still
fairly expensive.

But I think the problem is more with poorly written configure source,
which is the argument about mastering..


> On small projects configure can take longer than compile... Configure
> is my main gripe against make/autotools. Plus all the other stuff,
> auto-reconf, autogen, etc.

configure having zero dependencies is the killer feature compared
to all other options. The tight integration between configure and
cross-toolchains is also a very strong point.


> The larger the project, the slower configure can be.

Doesn't have to be, but it's easy to write poor configure source and
difficult to write good source.


//Peter

Re: [gentoo-dev] Re: cmake + ninja vs autotools

[William L. Thomson Jr.]

On Thu, 23 Nov 2017 16:36:07 +
Peter Stuge  wrote:
>
> It's arguably a bug that a projects gets huge.

Sometimes huge projects are split into many internally. Imagine this
was using autotools. I doubt it could use a master configure for all
sub projects, but maybe.
https://github.com/apache/incubator-netbeans
 
> The simplicity of configure+make is difficult to beat, but I also
> agree that it's difficult or at least tedious to master autotools.

The syntax is nasty, the files get big quick. But that is not my main
gripe with what I call as a suite autotools. Make is not so much of an
issue, but configure I absolutely  hate. I cannot understand why
systems get faster, yet configure seems to take the same amount of time
and is super slow.

On small projects configure can take longer than compile... Configure
is my main gripe against make/autotools. Plus all the other stuff,
auto-reconf, autogen, etc.

> That is arguably reason enough to choose meson, but I think I will
> stay with autotools for a while..

Its likely to remain for sometime. I am not on the meson bandwagon
entirely as I like cmake+cpack. But I am surprised how many projects
are migrating to meson. Seems to be the current trend, and a
considerable amount moving to meson.

Meson vs cmake configure is not that big of a difference. maybe like
make vs ninja. But Meson or Cmake vs configure, HUGE difference...
The larger the project, the slower configure can be.

-- 
William L. Thomson Jr.

Re: [gentoo-dev] Re: cmake + ninja vs autotools

[Peter Stuge]

Martin Vaeth wrote:
> > 1.  Doing a full clean build [..] the speed of make or ninja is hugely
> > offset by the compilation speed, and their overhead is negligible.
> 
> It depends on the definition of negligible. For huge projects like
> boost or chromium it is several minutes

It's arguably a bug that a projects gets huge.

The simplicity of configure+make is difficult to beat, but I also
agree that it's difficult or at least tedious to master autotools.

That is arguably reason enough to choose meson, but I think I will
stay with autotools for a while..


//Peter

[gentoo-dev] Re: cmake + ninja vs autotools

[Martin Vaeth]

Georg Rudoy <0xd34df...@gmail.com> wrote:
> 1. Doing a full clean build [..]
> the speed of make or ninja is hugely offset by the compilation speed,
> and their overhead is negligible.

It depends on the definition of negligible. For huge projects like
boost or chromium it is several minutes: That's why these projects
switched to ninja. Admittedly, for eix the difference is probably
only a few seconds on most systems.

Re: [gentoo-dev] Re: cmake + ninja vs autotools

[Michael Orlitzky]

On 11/19/2017 01:00 PM, William L. Thomson Jr. wrote:
>>
>> This is broken: Static metadata like DEPEND must not depend
>> on dynamic data like environment variables which are supposed
>> to change at emerge time.
> 
> I wondered about that. I guess adding to DEPEND via eclass is bad then.
> 

So long as you add to DEPEND unconditionally, the issue that Martin
pointed out does not apply. If you need to fudge the conditional, you
can put it behind a USE flag; for example,

  DEPEND+=" some_use_expand_flag_foo? ( foo )"

in the eclass. That's still adding to DEPEND unconditionally, even
though its effect on the dependency tree (at "emerge" time) is
conditional on the USE flag.

Re: [gentoo-dev] Re: cmake + ninja vs autotools

[Georg Rudoy]

On 19.11.17 at 16:49 user Martin Vaeth  wrote:
>> Ninja is most of the speed of meson less configure time savings
> 
> ++
> For eix, the main motivation to support meson as an
> alternative build system was to be able to use ninja...

As somebody having a big C++/Qt/cmake project: there are basically two corner 
cases for the build scenario:

1. Doing a full clean build — that's what's relevant for Gentoo, and in this 
case the speed of make or ninja is hugely offset by the compilation speed, and 
their overhead is negligible.
2. Doing an incremental build after changing a couple of files — that's what 
relevant during normal development, and arguably not that relevant for Gentoo. 
And, ninja doesn't have that much of a speed advantage lately in this case. In 
fact, make turns out to be faster in the latter if the project is using automoc 
in cmake.

Just my two cents.


-- 
  Georg Rudoy

Re: [gentoo-dev] Re: cmake + ninja vs autotools

[William L. Thomson Jr.]

On Sun, 19 Nov 2017 16:49:51 + (UTC)
Martin Vaeth  wrote:

> William L. Thomson Jr.  wrote:
> >
> > case ${CMAKE_MAKEFILE_GENERATOR} in
> > emake)
> > DEPEND="sys-devel/make"
> > ;;
> > ninja)
> > DEPEND="dev-util/ninja"
> > ;;  
> 
> This is broken: Static metadata like DEPEND must not depend
> on dynamic data like environment variables which are supposed
> to change at emerge time.

I wondered about that. I guess adding to DEPEND via eclass is bad then.

> > Only 2 thus far does not sound like most things would have issues.  
> 
> In fact, almost nothing has issues.

That has been my experience why I brought it up.

>I am using
> CMAKE_MAKEFILE_GENERATOR=ninja in my make.conf
> since years, and the total list of packages which had
> ever failed here (out of currently ~1500 installed)
> is this:
> 
> dev-util/cmake
> kde-apps/kate
> kde-apps/gwenview
> media-libs/avidemux-core
> media-libs/avidemux-plugins
> media-video/avidemux
> media-video/kaffeine
> media-video/kmplayer
> net-vpn/kvpnc
> sci-mathematics/reduce
> 
> This list might appear long, but note that

That is not that long and seems to favor heavily in ninjas favor. Seems
considerably more have no issues with ninja than make.

Thanks for that information!
 
> > Ninja is most of the speed of meson less configure time savings  
> 
> ++
> For eix, the main motivation to support meson as an
> alternative build system was to be able to use ninja...

For new projects I do not want a deb or rpm I like meson +
ninja. For all other stuff I prefer cmake + ninja. Kinda best of both
worlds. At least till meson can spit out deb and rpm, not just rpm spec.

I also like how cmake updates po files. Not sure about pot file, meson
does have something there, and can update them. Just a separate
process. cmake updates po all the time, I like that :)

But either way meson or cmake, ninja is the main speed for compiling.
Though I do like the cmake make output formatting and color, etc.

-- 
William L. Thomson Jr.


pgpsyIoAkVBmh.pgp
Description: OpenPGP digital signature

[gentoo-dev] Re: cmake + ninja vs autotools

[Martin Vaeth]

William L. Thomson Jr.  wrote:
>
> case ${CMAKE_MAKEFILE_GENERATOR} in
> emake)
> DEPEND="sys-devel/make"
> ;;
> ninja)
> DEPEND="dev-util/ninja"
> ;;

This is broken: Static metadata like DEPEND must not depend
on dynamic data like environment variables which are supposed
to change at emerge time.

> Only 2 thus far does not sound like most things would have issues.

In fact, almost nothing has issues. I am using
CMAKE_MAKEFILE_GENERATOR=ninja in my make.conf
since years, and the total list of packages which had
ever failed here (out of currently ~1500 installed)
is this:

dev-util/cmake
kde-apps/kate
kde-apps/gwenview
media-libs/avidemux-core
media-libs/avidemux-plugins
media-video/avidemux
media-video/kaffeine
media-video/kmplayer
net-vpn/kvpnc
sci-mathematics/reduce

This list might appear long, but note that

1. */avidemux* is essentially only 1 package
2. net-vpn/kvpnc is ancient and masked
3. sci-mathematics/reduce is ancient and masked

and moreover, perhaps some other entries of this list are even
obsolete (I do not have installed KDE 5, so I cannot try quickly;
when I had added the packages to the list, KDE 4 was rather new...)

> Ninja is most of the speed of meson less configure time savings

++
For eix, the main motivation to support meson as an
alternative build system was to be able to use ninja...

Re: [gentoo-dev] Re: cmake + ninja vs autotools

[William L. Thomson Jr.]

On Sat, 18 Nov 2017 02:52:33 +0100
Francesco Riosa  wrote:
>
> In my user opinion this has no place in a ebuild unless upstream
> clearly say to use (or evidently use) ninja as it main generator.

I think Gentoo deviates from upstreams fairly considerably at times. I
see this as case where Gentoo can help facility things to upstream.
Maybe they haven't the time to test, etc.

Current example
https://sourceforge.net/p/firebird/mailman/firebird-devel/thread/assp.04935012e0.20171116111219.18e86899%40wlt.obsidian-studios.com/#msg36117925

> In cases where ninja is considered (by upstream) the best option,
> Michael suggestion to make it overridable is a very wise one.
> In that case, please also remember to depend on ninja

I do not think that is necessary unless it bypasses this

case ${CMAKE_MAKEFILE_GENERATOR} in
emake)
DEPEND="sys-devel/make"
;;
ninja)
DEPEND="dev-util/ninja"
;;


> Since other emails (by Christoph and Brian) in this thread make
> evident that it's not a drop in replacement,  fixing it in the eclass
> seem a bad idea, but that probably should be reconsidered when ninja
> become more capable.

Only 2 thus far does not sound like most things would have issues.
Maybe worth a bug to track stuff that builds fine and things that fail.
Could use the math alone to make a final call. More packages fail,
stick with make. if only a few, switch to ninja and have those stick
with the slow make.

Either way up to others. I am just passing on whats going on in many
other FOSS projects. Ninja is most of the speed of meson less configure
time savings.

-- 
William L. Thomson Jr.


pgpbJ6TtMkfy0.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: cmake + ninja vs autotools

[Francesco Riosa]

On 18/11/2017 00:35, Michael Palimaka wrote:
> On 11/16/2017 02:27 PM, William L. Thomson Jr. wrote:
>> It maybe worth considering switching the default generator in the
>> cmake-utils.eclass from the default of emake to ninja.
>>
>> - : ${CMAKE_MAKEFILE_GENERATOR:=emake}
>> + : ${CMAKE_MAKEFILE_GENERATOR:=ninja}
>>
>> For those with cmake ebuilds you can test this out now via 
>>
>> CMAKE_MAKEFILE_GENERATOR="ninja"
>> inherit cmake-utils
> If testing this, please use : ${CMAKE_MAKEFILE_GENERATOR:=ninja} in
> ebuilds unless the emake generator is explicitly known not to work. This
> will preserve user choice if they want to avoid ninja for some reason.
>
In my user opinion this has no place in a ebuild unless upstream clearly
say to use (or evidently use) ninja as it main generator.
In cases where ninja is considered (by upstream) the best option,
Michael suggestion to make it overridable is a very wise one.
In that case, please also remember to depend on ninja

Since other emails (by Christoph and Brian) in this thread make evident
that it's not a drop in replacement,  fixing it in the eclass seem a bad
idea, but that probably should be reconsidered when ninja become more
capable.

[gentoo-dev] Re: cmake + ninja vs autotools

[Michael Palimaka]

On 11/16/2017 02:27 PM, William L. Thomson Jr. wrote:
> It maybe worth considering switching the default generator in the
> cmake-utils.eclass from the default of emake to ninja.
> 
> - : ${CMAKE_MAKEFILE_GENERATOR:=emake}
> + : ${CMAKE_MAKEFILE_GENERATOR:=ninja}
> 
> For those with cmake ebuilds you can test this out now via 
> 
> CMAKE_MAKEFILE_GENERATOR="ninja"
> inherit cmake-utils

If testing this, please use : ${CMAKE_MAKEFILE_GENERATOR:=ninja} in
ebuilds unless the emake generator is explicitly known not to work. This
will preserve user choice if they want to avoid ninja for some reason.