On Wed, Sep 24, 2014 at 09:03:11 +0200, Freddie Chopin wrote:
> On 09/24/2014 02:40 AM, Mike Shal wrote:
> > Build time increased by over 100% compared to what?
>
> Full rebuild WITH ^c^ takes 69s / 63s (first / second run). The same
> full rebuild without this flag takes half that time - 31s / 29s. This is
> a very specific project - it has 724 commands to run, most of the source
> files are really small (someone has strange vision about helping linker
> with dead code removal), but on the other hand it is the project that
> made me start using tup (; It uses recursive Makefiles and the build was
> so bad that a no-op build on Windows took about 30s (I don't remember
> exactly now), and a full rebuild - about 3 minutes. With tup a no-op
> build was down to about 10s (some parts of the Makefile based build
> ["context generation"] were left, as 10s was acceptable, and the amount
> of work to full convert to tup would be huge... - from these 10s 99% was
> Makefile, so no worries, tup is still fast (; ), and a full rebuild to
> ~90s. Out of curiosity I tried the recursive Makefile build of this
> project on Linux, and it's actually a bit faster in compilation than tup
> with ^c^ enabled... Full rebuild takes about 55s, while the almost-no-op
> build (the Makefiles are so damn wrong that the executable get's
> re-linked every time) takes 3-5s.
I have a project for which there are 114 source files, 10k lines of C++
and 45 build trees worth of no-op builds took .5s with Ninja.
> The difference between non-chroot rebuild and chroot rebuild is not so
> significant with a more "normal" (and a bit smaller) project:
> - with ^c^ - ~10s,
> - without ^c^ - ~8s.
>
> In the first project most of the time is "wasted" on starting processes
> (and doing chroots), as each compilation step takes about 0.1s, while in
> the second one compilation of most files takes about 0.5s each (there
> are 50 commands in total). And the second one has Tupfiles written in lua (;
>
> Oh - while I still remember - I think that automatic dependency tracking
> of tup ("correctness") and it's simplicity* should be emphasized more
> than its speed - from several talks with people I think that not many of
> them care about speed that much, as not many of us have projects with
> several thousands of source files... As I understand that tup can get
> popular when some major (and big) project adopts it, I guess that a
> middle ground would be more appropriate - emphasize correctness,
> simplicity and speed equally (; In my opinion tup's website puts the
> most emphasis on the speed.
Would you mind testing Ninja (with CMake, gn, gyp, or hand-written) as
well? I'm curious (and it could give a good baseline for how fast the
build could be) as to the difference.
> * - for me tup's syntax (the original one) was super simple, especially
> compared to other systems like make, cmake. The lua syntax in tup is not
> that simple (especially with lack of good documentation), but still
> pretty easy to use.
>
> > Ahh, yeah - right now I think the tupjob number is just the tupid,
> > which is essentially the row that gets added to the database. I don't
> > see why we couldn't just use a job slot number (1 - #cpus) and map
> > that to the tupid instead. Though if we end up just using a namespace
> > the problem may go away anyway.
>
> It seems that namespaces are enabled in Arch Linux now (last time
> someone on this list mentioned that Arch didn't have them):
FWIW, Fedora 20 has them as well.
> > $ zgrep CONFIG_NAMESPACES /proc/config.gz
> > CONFIG_NAMESPACES=y
>
> At least I hope that it's the feature, as there is some discussion about
> "user namespaces" being explicitly NOT enabled...
The CONFIG_.*_NS options are also important. CONFIG_NAMESPACES is just
namespaces in general; there are multiple kinds.
> > $ zgrep CONFIG_USER_NS /proc/config.gz
> > # CONFIG_USER_NS is not set
>
> https://bugs.archlinux.org/task/36969
>
> Anyway - with "slot" number the problem of debugging would be solvable
> within IDE (or GDB) by simple path mapping - adding as many entries as
> there are CPUs available is reasonable - I would definetely go that way
> instead of playing with suid (;
Yes, distro builders (I imagine) won't be happy with suid binaries being
necessary for a build on their build machines. Especially if Tup's
permissions dropping isn't airtight.
--Ben
--
--
tup-users mailing list
email: [email protected]
unsubscribe: [email protected]
options: http://groups.google.com/group/tup-users?hl=en
---
You received this message because you are subscribed to the Google Groups
"tup-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
