Re: LLVM port(s) take very long time to package

[Alexey Dokuchaev]

On Fri, Apr 14, 2017 at 04:42:35PM +0200, Michael Gmelin wrote:
> > On 14. Apr 2017, at 16:22, Alexey Dokuchaev  wrote:
> > On Wed, Apr 12, 2017 at 12:06:42PM +0200, Michael Gmelin wrote:
> >>> On 12. Apr 2017, at 11:37, Alexey Dokuchaev  wrote:
> >>> ...
> >>> Other alternative to PKG_NOCOMPRESS=1 could be PKGSUFFIX=.tbz
> >>> (pkg-static create -f tbz ...), will play with that as well.
> >> 
> >> I don't think bzip will buy you much in terms of performance,
> > 
> > Surprisingly, it actually did:
> > 
> > $ time make package PKG_SUFX=.tbz
> > ===>  Building package for llvm40-4.0.0_2
> > 565.215u 5.316s 9:34.18 99.3%   7899+502k 1+3020io 0pf+0w
> > ^^^
> > Single-threaded bzip2 was faster than multi-threaded xz [...]
> > of larger package of course:
> 
> Interesting, I'll tried that on our high-frequency builders.

Cool, please share your numbers with us once you have them.

Meanwhile, I've patched my tinderbox to package with PKG_NOCOMPRESS=1 and
then use `archivers/pbzip2' to compress the file.  This cut down the time
to package `devel/llvm40' from previous ~50 minutes to just three without
touching any libarchive(3) or pkg(8) code.  I guess I can live with that
until libarchive(3) and/or pkg(8) gain proper support of multicore-aware
efficient modern compression algorithm(s).

./danfe
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: LLVM port(s) take very long time to package

[Michael Gmelin]

> On 14. Apr 2017, at 16:22, Alexey Dokuchaev  wrote:
> 
> On Wed, Apr 12, 2017 at 12:06:42PM +0200, Michael Gmelin wrote:
>>> On 12. Apr 2017, at 11:37, Alexey Dokuchaev  wrote:
>>> ...
>>> Other alternative to PKG_NOCOMPRESS=1 could be PKGSUFFIX=.tbz (pkg-static
>>> create -f tbz ...), will play with that as well.
>> 
>> I don't think bzip will buy you much in terms of performance,
> 
> Surprisingly, it actually did:
> 
> $ time make package PKG_SUFX=.tbz
> ===>  Building package for llvm40-4.0.0_2
> 565.215u 5.316s 9:34.18 99.3%   7899+502k 1+3020io 0pf+0w
>^^^
> Single-threaded bzip2 was faster than multi-threaded xz, at the expense
> of larger package of course:
> 
> $ ls -s1 llvm40-4.0.0_2.t*
> 386752 llvm40-4.0.0_2.tbz
> 286816 llvm40-4.0.0_2.txz
>> 

Interesting, I'll tried that on our high-frequency builders.

-m

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: LLVM port(s) take very long time to package

[Alexey Dokuchaev]

On Wed, Apr 12, 2017 at 12:06:42PM +0200, Michael Gmelin wrote:
> > On 12. Apr 2017, at 11:37, Alexey Dokuchaev  wrote:
> > ...
> > Other alternative to PKG_NOCOMPRESS=1 could be PKGSUFFIX=.tbz (pkg-static
> > create -f tbz ...), will play with that as well.
> 
> I don't think bzip will buy you much in terms of performance,

Surprisingly, it actually did:

$ time make package PKG_SUFX=.tbz
===>  Building package for llvm40-4.0.0_2
565.215u 5.316s 9:34.18 99.3%   7899+502k 1+3020io 0pf+0w
^^^
Single-threaded bzip2 was faster than multi-threaded xz, at the expense
of larger package of course:

$ ls -s1 llvm40-4.0.0_2.t*
386752 llvm40-4.0.0_2.tbz
286816 llvm40-4.0.0_2.txz

> supporting something like lz4 would be useful.

Or, like vsevolod@ had mentioned, Zstandard.  But I'm afraid it would
take some time before we'd arrive there.

./danfe
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: LLVM port(s) take very long time to package

[Alexey Dokuchaev]

On Wed, Apr 12, 2017 at 04:12:45PM +0200, Jan Beich wrote:
> Michael Gmelin  writes:
> > It's probably busy compressing the package. Try setting
> > PKG_NOCOMPRESS=1 in /etc/make.conf (works ok when just building a
> > package with make, didn't test with poudriere[0]). When creating
> > packages, the current compression algorithm can't take advantage of
> > multiple cores, so it's kind of a bottleneck on modern machines.
> 
> Nah, libarchive just defaults to 1 thread instead of using hw.ncpu.
> The tricky part is propagating the fix into old releases or setting
> compression options from within pkg(8).

Nice, I've rebuilt pkg(8) against patched libarchive, and run `make
package' for devel/llvm40:

$ make
...
$ time make package
===>  Building package for llvm40-4.0.0_2
2029.111u 6.614s 11:03.25 306.9%7884+502k 4+2240io 1pf+0w
$ xzcat -t llvm40-4.0.0_2.txz ; echo $?
0

It went down from ~50 minutes to eleven on a 4-core APU.  I've filed an
issue on GitHub:

https://github.com/freebsd/pkg/issues/1564

./danfe
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: LLVM port(s) take very long time to package

[Walter Schwarzenfeld]

Sorry, seems it was my error.

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: LLVM port(s) take very long time to package

[Walter Schwarzenfeld]

The patch does not apply.

Only the first file is patched, the two others are rejected.

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: LLVM port(s) take very long time to package

[Jan Beich]

Michael Gmelin  writes:

>> On 12. Apr 2017, at 05:43, Alexey Dokuchaev  wrote:
>> 
>> Hi there porters,
>> 
>> It had been annoying me for a while, so I figured I try asking here:
>> LLVM ports (particularly, llvm39-3.9.1) take abnormally long time to
>> package, e.g. right now it took 43 minutes (tmpfs-backed build area,
>> pkg-1.10.0_2.txz, AMD A8 APU @1900 MHz, plenty of RAM, no swap).
>> 
>> pkg was eating 100% (of a single core) during that time, so it looks
>> like it's actually busy (not waiting for I/O or something), but with
>> what?  And can it be fixed?  LLVM ports are real PITA to (re)build,
>> and while ccache helps to greatly reduce the build times, 3/4-hour
>> package time still cripples it.
>> 
>
> It's probably busy compressing the package. Try setting
> PKG_NOCOMPRESS=1 in /etc/make.conf (works ok when just building a
> package with make, didn't test with poudriere[0]). When creating
> packages, the current compression algorithm can't take advantage of
> multiple cores, so it's kind of a bottleneck on modern machines.

Nah, libarchive just defaults to 1 thread instead of using hw.ncpu.
The tricky part is propagating the fix into old releases or setting 
compression options from within pkg(8).

Index: lib/libarchive/config_freebsd.h
===
--- lib/libarchive/config_freebsd.h (revision 316719)
+++ lib/libarchive/config_freebsd.h (working copy)
@@ -139,6 +139,7 @@
 #define HAVE_LONG_LONG_INT 1
 #define HAVE_LSTAT 1
 #define HAVE_LUTIMES 1
+#define HAVE_LZMA_STREAM_ENCODER_MT 1
 #define HAVE_MBRTOWC 1
 #define HAVE_MEMMOVE 1
 #define HAVE_MEMORY_H 1
Index: contrib/libarchive/libarchive/archive_write_add_filter_xz.c
===
--- contrib/libarchive/libarchive/archive_write_add_filter_xz.c (revision 
316719)
+++ contrib/libarchive/libarchive/archive_write_add_filter_xz.c (working copy)
@@ -152,7 +152,11 @@ common_setup(struct archive_write_filter *f)
}
f->data = data;
data->compression_level = LZMA_PRESET_DEFAULT;
+#ifdef HAVE_LZMA_STREAM_ENCODER_MT
+   data->threads = lzma_cputhreads();
+#else
data->threads = 1;
+#endif
f->open = _compressor_xz_open;
f->close = archive_compressor_xz_close;
f->free = archive_compressor_xz_free;
Index: contrib/libarchive/libarchive/archive_write_set_format_xar.c
===
--- contrib/libarchive/libarchive/archive_write_set_format_xar.c
(revision 316719)
+++ contrib/libarchive/libarchive/archive_write_set_format_xar.c
(working copy)
@@ -384,7 +384,11 @@ archive_write_set_format_xar(struct archive *_a)
/* Set default compression type, level, and number of threads. */
xar->opt_compression = GZIP;
xar->opt_compression_level = 6;
+#ifdef HAVE_LZMA_STREAM_ENCODER_MT
+   xar->opt_threads = lzma_cputhreads();
+#else
xar->opt_threads = 1;
+#endif
 
a->format_data = xar;
 


signature.asc
Description: PGP signature

Re: LLVM port(s) take very long time to package

[Michael Gmelin]

> On 12. Apr 2017, at 11:37, Alexey Dokuchaev  wrote:
> 
> On Wed, Apr 12, 2017 at 10:43:14AM +0200, Michael Gmelin wrote:
>>> On 12. Apr 2017, at 05:43, Alexey Dokuchaev  wrote:
>>> ...
>>> pkg was eating 100% (of a single core) during that time, so it looks
>>> like it's actually busy (not waiting for I/O or something), but with
>>> what?  And can it be fixed?  LLVM ports are real PITA to (re)build,
>>> and while ccache helps to greatly reduce the build times, 3/4-hour
>>> package time still cripples it.
>> 
>> It's probably busy compressing the package. Try setting PKG_NOCOMPRESS=1
>> in /etc/make.conf (works ok when just building a package with make,
>> didn't test with poudriere[0]). When creating packages, the current
>> compression algorithm can't take advantage of multiple cores, so it's
>> kind of a bottleneck on modern machines.
> 
> That is my current best guest as well, but I could not confirm it because
> /usr/local/sbin/pkg-static is stripped by default and I could not see any
> function names when I attached gdb(1) to it.  Will rebuild pkg(8) shortly
> WITH_DEBUG to get more information.
> 
> Other alternative to PKG_NOCOMPRESS=1 could be PKGSUFFIX=.tbz (pkg-static
> create -f tbz ...), will play with that as well.

I don't think bzip will buy you much in terms of performance, supporting 
something like lz4 would be useful.

-m


___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: LLVM port(s) take very long time to package

[Alexey Dokuchaev]

On Wed, Apr 12, 2017 at 10:43:14AM +0200, Michael Gmelin wrote:
> > On 12. Apr 2017, at 05:43, Alexey Dokuchaev  wrote:
> > ...
> > pkg was eating 100% (of a single core) during that time, so it looks
> > like it's actually busy (not waiting for I/O or something), but with
> > what?  And can it be fixed?  LLVM ports are real PITA to (re)build,
> > and while ccache helps to greatly reduce the build times, 3/4-hour
> > package time still cripples it.
> 
> It's probably busy compressing the package. Try setting PKG_NOCOMPRESS=1
> in /etc/make.conf (works ok when just building a package with make,
> didn't test with poudriere[0]). When creating packages, the current
> compression algorithm can't take advantage of multiple cores, so it's
> kind of a bottleneck on modern machines.

That is my current best guest as well, but I could not confirm it because
/usr/local/sbin/pkg-static is stripped by default and I could not see any
function names when I attached gdb(1) to it.  Will rebuild pkg(8) shortly
WITH_DEBUG to get more information.

Other alternative to PKG_NOCOMPRESS=1 could be PKGSUFFIX=.tbz (pkg-static
create -f tbz ...), will play with that as well.  Thanks,

./danfe
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: LLVM port(s) take very long time to package

[Michael Gmelin]

> On 12. Apr 2017, at 05:43, Alexey Dokuchaev  wrote:
> 
> Hi there porters,
> 
> It had been annoying me for a while, so I figured I try asking here:
> LLVM ports (particularly, llvm39-3.9.1) take abnormally long time to
> package, e.g. right now it took 43 minutes (tmpfs-backed build area,
> pkg-1.10.0_2.txz, AMD A8 APU @1900 MHz, plenty of RAM, no swap).
> 
> pkg was eating 100% (of a single core) during that time, so it looks
> like it's actually busy (not waiting for I/O or something), but with
> what?  And can it be fixed?  LLVM ports are real PITA to (re)build,
> and while ccache helps to greatly reduce the build times, 3/4-hour
> package time still cripples it.
> 

It's probably busy compressing the package. Try setting PKG_NOCOMPRESS=1 in 
/etc/make.conf (works ok when just building a package with make, didn't test 
with poudriere[0]). When creating packages, the current compression algorithm 
can't take advantage of multiple cores, so it's kind of a bottleneck on modern 
machines.

-m

[0] probably not, but it seems like bdrewery is working on this 
https://github.com/freebsd/poudriere/issues/459
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"