Re: svn commit: r316978 - in head: contrib/zstd etc/mtree lib lib/libzstd share/mk usr.bin usr.bin/zstd

[Ian Lepore]

On Sun, 2017-04-23 at 01:27 +0200, Baptiste Daroussin wrote:
> On Sat, Apr 22, 2017 at 04:05:28PM -0600, Ian Lepore wrote:
> > 
> > On Sun, 2017-04-16 at 09:35 +0200, Baptiste Daroussin wrote:
> > > 
> > > On Sat, Apr 15, 2017 at 04:37:02PM -0700, Rodney W. Grimes wrote:
> > > > 
> > > > 
> > > > And we need this in base for what great purpose???
> > > > 
> > > zstandard is the successor to lz4, there are attempts to
> > > integrate
> > > zstandard in
> > > ZFS, Allan Jude is working on that, I am working on integrating
> > > in
> > > libstand to
> > > allow to boot a kernel compressed with zstandard or a mfsroot
> > > with
> > > zstandard,
> > > which would benefit lots of pxe users.
> > > 
> > > I have also added it to newsyslog(8) so users can benefit
> > > zstandard
> > > compression
> > > for the log files, which will consume less CPUs for greater
> > > compression that
> > > bzip2. On low ended boxes with low storage this is actually very
> > > useful as well.
> > > 
> > > I haven't checked yet the performances on my armv6 boxes, but on
> > > the
> > > low ended
> > > recent atom laptops I can tell this is already very useful.
> > > 
> > > Bapt
> > Does it also need to be added to the toolkit in /rescue?  The other
> > major compression tools in base are there.
> I'm not sure it is worth it, is it?
> 
> Bapt

I'm not sure what "worth it" means.  The other compression tools are in
there, I assume because there are recovery scenarios that require them.
 If this new compression gets used in the base system, then perhaps it
also would need to be in there.

For example, if logs are compressed this new way, and you get into a
recovery situation, then you might need to view the logs to figure out
how to recover the system.

-- Ian

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316978 - in head: contrib/zstd etc/mtree lib lib/libzstd share/mk usr.bin usr.bin/zstd

[Baptiste Daroussin]

On Sat, Apr 22, 2017 at 04:05:28PM -0600, Ian Lepore wrote:
> On Sun, 2017-04-16 at 09:35 +0200, Baptiste Daroussin wrote:
> > On Sat, Apr 15, 2017 at 04:37:02PM -0700, Rodney W. Grimes wrote:
> > > 
> > > And we need this in base for what great purpose???
> > > 
> > zstandard is the successor to lz4, there are attempts to integrate
> > zstandard in
> > ZFS, Allan Jude is working on that, I am working on integrating in
> > libstand to
> > allow to boot a kernel compressed with zstandard or a mfsroot with
> > zstandard,
> > which would benefit lots of pxe users.
> > 
> > I have also added it to newsyslog(8) so users can benefit zstandard
> > compression
> > for the log files, which will consume less CPUs for greater
> > compression that
> > bzip2. On low ended boxes with low storage this is actually very
> > useful as well.
> > 
> > I haven't checked yet the performances on my armv6 boxes, but on the
> > low ended
> > recent atom laptops I can tell this is already very useful.
> > 
> > Bapt
> 
> Does it also need to be added to the toolkit in /rescue?  The other
> major compression tools in base are there.

I'm not sure it is worth it, is it?

Bapt


signature.asc
Description: PGP signature

Re: svn commit: r316978 - in head: contrib/zstd etc/mtree lib lib/libzstd share/mk usr.bin usr.bin/zstd

[Ian Lepore]

On Sun, 2017-04-16 at 09:35 +0200, Baptiste Daroussin wrote:
> On Sat, Apr 15, 2017 at 04:37:02PM -0700, Rodney W. Grimes wrote:
> > 
> > And we need this in base for what great purpose???
> > 
> zstandard is the successor to lz4, there are attempts to integrate
> zstandard in
> ZFS, Allan Jude is working on that, I am working on integrating in
> libstand to
> allow to boot a kernel compressed with zstandard or a mfsroot with
> zstandard,
> which would benefit lots of pxe users.
> 
> I have also added it to newsyslog(8) so users can benefit zstandard
> compression
> for the log files, which will consume less CPUs for greater
> compression that
> bzip2. On low ended boxes with low storage this is actually very
> useful as well.
> 
> I haven't checked yet the performances on my armv6 boxes, but on the
> low ended
> recent atom laptops I can tell this is already very useful.
> 
> Bapt

Does it also need to be added to the toolkit in /rescue?  The other
major compression tools in base are there.

-- Ian

___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316978 - in head: contrib/zstd etc/mtree lib lib/libzstd share/mk usr.bin usr.bin/zstd

[Ed Maste]

On 19 April 2017 at 11:22, Ruslan Bukin  wrote:
>
> I committed. Please upstream that.

As jhb pointed out we should probably add __bswapdi2 to compiler-rt
(it's already there for ARM). I would assume that libgcc also has it,
and this is a workaround for our slightly unusual case of GCC w/ a
compiler-rt patched for RISC-V. If so this should not be sent
upstream.
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316978 - in head: contrib/zstd etc/mtree lib lib/libzstd share/mk usr.bin usr.bin/zstd

[Ruslan Bukin]

On Wed, Apr 19, 2017 at 04:53:25PM +0200, Baptiste Daroussin wrote:
> On Wed, Apr 19, 2017 at 02:21:06PM +, Ruslan Bukin wrote:
> > This break RISC-V world build:
> > /home/br/obj//riscv.riscv64/usr/home/br/dev/freebsd-head/tmp/usr/lib/libprivatezstd.so:
> >  undefined reference to `__bswap
> > si2'
> > /home/br/obj//riscv.riscv64/usr/home/br/dev/freebsd-head/tmp/usr/lib/libprivatezstd.so:
> >  undefined reference to `__bswap
> > di2'
> > 
> > Here is patch that makes it compilable:
> > 
> > diff --git a/contrib/zstd/lib/common/xxhash.c 
> > b/contrib/zstd/lib/common/xxhash.c
> > index eb44222c5fc..d59a22a0638 100644
> > --- a/contrib/zstd/lib/common/xxhash.c
> > +++ b/contrib/zstd/lib/common/xxhash.c
> > @@ -206,7 +206,7 @@ static U64 XXH_read64(const void* memPtr)
> >  #if defined(_MSC_VER) /* Visual Studio */
> >  #  define XXH_swap32 _byteswap_ulong
> >  #  define XXH_swap64 _byteswap_uint64
> > -#elif GCC_VERSION >= 403
> > +#elif (GCC_VERSION >= 403 && !defined(__riscv__))
> >  #  define XXH_swap32 __builtin_bswap32
> >  #  define XXH_swap64 __builtin_bswap64
> >  #else
> > 
> > Ruslan
> 
> Thank you, can you please commit it? Have you upstreamed it, do you want me to
> do it? (I have already signed the facebook agreement)
> 

I committed. Please upstream that.

thanks

Ruslan
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316978 - in head: contrib/zstd etc/mtree lib lib/libzstd share/mk usr.bin usr.bin/zstd

[Baptiste Daroussin]

On Wed, Apr 19, 2017 at 02:21:06PM +, Ruslan Bukin wrote:
> This break RISC-V world build:
> /home/br/obj//riscv.riscv64/usr/home/br/dev/freebsd-head/tmp/usr/lib/libprivatezstd.so:
>  undefined reference to `__bswap
> si2'
> /home/br/obj//riscv.riscv64/usr/home/br/dev/freebsd-head/tmp/usr/lib/libprivatezstd.so:
>  undefined reference to `__bswap
> di2'
> 
> Here is patch that makes it compilable:
> 
> diff --git a/contrib/zstd/lib/common/xxhash.c 
> b/contrib/zstd/lib/common/xxhash.c
> index eb44222c5fc..d59a22a0638 100644
> --- a/contrib/zstd/lib/common/xxhash.c
> +++ b/contrib/zstd/lib/common/xxhash.c
> @@ -206,7 +206,7 @@ static U64 XXH_read64(const void* memPtr)
>  #if defined(_MSC_VER) /* Visual Studio */
>  #  define XXH_swap32 _byteswap_ulong
>  #  define XXH_swap64 _byteswap_uint64
> -#elif GCC_VERSION >= 403
> +#elif (GCC_VERSION >= 403 && !defined(__riscv__))
>  #  define XXH_swap32 __builtin_bswap32
>  #  define XXH_swap64 __builtin_bswap64
>  #else
> 
> Ruslan

Thank you, can you please commit it? Have you upstreamed it, do you want me to
do it? (I have already signed the facebook agreement)

Best regards,
Bapt


signature.asc
Description: PGP signature

Re: svn commit: r316978 - in head: contrib/zstd etc/mtree lib lib/libzstd share/mk usr.bin usr.bin/zstd

[Ruslan Bukin]

This break RISC-V world build:
/home/br/obj//riscv.riscv64/usr/home/br/dev/freebsd-head/tmp/usr/lib/libprivatezstd.so:
 undefined reference to `__bswap
si2'
/home/br/obj//riscv.riscv64/usr/home/br/dev/freebsd-head/tmp/usr/lib/libprivatezstd.so:
 undefined reference to `__bswap
di2'

Here is patch that makes it compilable:

diff --git a/contrib/zstd/lib/common/xxhash.c b/contrib/zstd/lib/common/xxhash.c
index eb44222c5fc..d59a22a0638 100644
--- a/contrib/zstd/lib/common/xxhash.c
+++ b/contrib/zstd/lib/common/xxhash.c
@@ -206,7 +206,7 @@ static U64 XXH_read64(const void* memPtr)
 #if defined(_MSC_VER) /* Visual Studio */
 #  define XXH_swap32 _byteswap_ulong
 #  define XXH_swap64 _byteswap_uint64
-#elif GCC_VERSION >= 403
+#elif (GCC_VERSION >= 403 && !defined(__riscv__))
 #  define XXH_swap32 __builtin_bswap32
 #  define XXH_swap64 __builtin_bswap64
 #else

Ruslan

On Sat, Apr 15, 2017 at 08:05:22PM +, Baptiste Daroussin wrote:
> Author: bapt
> Date: Sat Apr 15 20:05:22 2017
> New Revision: 316978
> URL: https://svnweb.freebsd.org/changeset/base/316978
> 
> Log:
>   Import zstandard 1.1.4 in base
>   
>   zstandard is a new compression library/tool which is very fast at
>   compression/decompression
>   
>   For now import as a private library
> 
> Added:
>   head/contrib/zstd/
>  - copied from r316976, vendor/zstd/dist/
>   head/lib/libzstd/
>   head/lib/libzstd/Makefile   (contents, props changed)
>   head/usr.bin/zstd/
>   head/usr.bin/zstd/Makefile   (contents, props changed)
> Modified:
>   head/etc/mtree/BSD.usr.dist
>   head/lib/Makefile
>   head/share/mk/src.libnames.mk
>   head/usr.bin/Makefile
> 
> Modified: head/etc/mtree/BSD.usr.dist
> ==
> --- head/etc/mtree/BSD.usr.dist   Sat Apr 15 20:03:50 2017
> (r316977)
> +++ head/etc/mtree/BSD.usr.dist   Sat Apr 15 20:05:22 2017
> (r316978)
> @@ -17,6 +17,8 @@
>  ..
>  ucl
>  ..
> +zstd
> +..
>  ..
>  ..
>  lib
> 
> Modified: head/lib/Makefile
> ==
> --- head/lib/Makefile Sat Apr 15 20:03:50 2017(r316977)
> +++ head/lib/Makefile Sat Apr 15 20:05:22 2017(r316978)
> @@ -90,6 +90,7 @@ SUBDIR= ${SUBDIR_BOOTSTRAP} \
>   libxo \
>   liby \
>   libz \
> + libzstd \
>   ncurses
>  
>  # Inter-library dependencies.  When the makefile for a library contains LDADD
> 
> Added: head/lib/libzstd/Makefile
> ==
> --- /dev/null 00:00:00 1970   (empty, because file is newly added)
> +++ head/lib/libzstd/Makefile Sat Apr 15 20:05:22 2017(r316978)
> @@ -0,0 +1,34 @@
> +# $FreeBSD$
> +
> +LIB= zstd
> +SRCS=entropy_common.c \
> + error_private.c \
> + fse_decompress.c \
> + pool.c \
> + threading.c \
> + xxhash.c \
> + zstd_common.c \
> + fse_compress.c \
> + huf_compress.c \
> + zstd_compress.c \
> + zstdmt_compress.c \
> + huf_decompress.c \
> + zstd_decompress.c \
> + zbuff_common.c \
> + zbuff_compress.c \
> + zbuff_decompress.c \
> + cover.c \
> + divsufsort.c \
> + zdict.c
> +WARNS=   2
> +INCS=zstd.h
> +CFLAGS+= -I${ZSTDDIR}/lib -I${ZSTDDIR}/lib/common -DXXH_NAMESPACE=ZSTD_
> +
> +PRIVATELIB=  yes
> +
> +ZSTDDIR= ${SRCTOP}/contrib/zstd
> +.PATH:   ${ZSTDDIR}/lib/common ${ZSTDDIR}/lib/compress \
> + ${ZSTDDIR}/lib/decompress ${ZSTDDIR}/lib/deprecated \
> + ${ZSTDDIR}/lib/dictBuilder ${ZSTDDIR}/lib
> +
> +.include 
> 
> Modified: head/share/mk/src.libnames.mk
> ==
> --- head/share/mk/src.libnames.mk Sat Apr 15 20:03:50 2017
> (r316977)
> +++ head/share/mk/src.libnames.mk Sat Apr 15 20:05:22 2017
> (r316978)
> @@ -24,7 +24,8 @@ _PRIVATELIBS=   \
>   sqlite3 \
>   ssh \
>   ucl \
> - unbound
> + unbound \
> + zstd
>  
>  _INTERNALLIBS=   \
>   amu \
> 
> Modified: head/usr.bin/Makefile
> ==
> --- head/usr.bin/Makefile Sat Apr 15 20:03:50 2017(r316977)
> +++ head/usr.bin/Makefile Sat Apr 15 20:05:22 2017(r316978)
> @@ -189,7 +189,8 @@ SUBDIR=   alias \
>   xo \
>   xz \
>   xzdec \
> - yes
> + yes \
> + zstd
>  
>  # NB: keep these sorted by MK_* knobs
>  
> 
> Added: head/usr.bin/zstd/Makefile
> ==
> --- /dev/null 00:00:00 1970   (empty, because file is newly added)
> +++ head/usr.bin/zstd/MakefileSat Apr 15 20:05:22 2017
> (r316978)
> @@ -0,0 +1,21 @@
> +# $FreeBSD$
> +
> +PROG=   

Re: svn commit: r316978 - in head: contrib/zstd etc/mtree lib lib/libzstd share/mk usr.bin usr.bin/zstd

[Matthew Ahrens]

On Sun, Apr 16, 2017 at 12:35 AM, Baptiste Daroussin 
wrote:

>
> zstandard is the successor to lz4,


That's not my understanding.  zstandard is an improvement in every respect
over zlib (gzip), but lz4 is faster, especially at decompression[*].  If
you have data to the contrary I'd be happy to be wrong, but AFAIK, lz4 will
retain its usefulness for some time, at least for ZFS.

there are attempts to integrate zstandard in
> ZFS, Allan Jude is working on that,


That's great, I look forward to it!

--matt

[*] According to the zstandard github page, lz4 is 1.5x faster at
compression and 3x faster at decompression. https://github.com/facebook/zstd
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316978 - in head: contrib/zstd etc/mtree lib lib/libzstd share/mk usr.bin usr.bin/zstd

[Mark Linimon]

On Sat, Apr 15, 2017 at 05:01:10PM -0700, Conrad Meyer wrote:
> Can you tone down the condescension, Rod?  Being rude doesn't help you
> make your case.

I'm going to have to +1 this.

We're a huge community now.  Comments that might be appropriate among a
handful of people aren't as appropriate now.  Add in all the different
(human) languages and cultures, and we have a hard enough time commmunicating
as it is.

mcl
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316978 - in head: contrib/zstd etc/mtree lib lib/libzstd share/mk usr.bin usr.bin/zstd

[Baptiste Daroussin]

On Sat, Apr 15, 2017 at 04:37:02PM -0700, Rodney W. Grimes wrote:
> And we need this in base for what great purpose???
> 

zstandard is the successor to lz4, there are attempts to integrate zstandard in
ZFS, Allan Jude is working on that, I am working on integrating in libstand to
allow to boot a kernel compressed with zstandard or a mfsroot with zstandard,
which would benefit lots of pxe users.

I have also added it to newsyslog(8) so users can benefit zstandard compression
for the log files, which will consume less CPUs for greater compression that
bzip2. On low ended boxes with low storage this is actually very useful as well.

I haven't checked yet the performances on my armv6 boxes, but on the low ended
recent atom laptops I can tell this is already very useful.

Bapt


signature.asc
Description: PGP signature

Re: svn commit: r316978 - in head: contrib/zstd etc/mtree lib lib/libzstd share/mk usr.bin usr.bin/zstd

[Allan Jude]

On 2017-04-15 19:37, Rodney W. Grimes wrote:
> And we need this in base for what great purpose???
> 

It is a new faster compression algorithm from the creator of LZ4, Yann
Collet.

It offers compression ratios similar or great than gzip, but 4-6x faster
compression, and 2-3x faster decompression (in excess of 1 gigabyte per
second per core).

It will be used in ZFS shortly, and is expected to supplant gzip/bzip2
for compressed kernel/memdisk in the loader for pxeboot / iscsi boot etc.


> 
>> Author: bapt
>> Date: Sat Apr 15 20:05:22 2017
>> New Revision: 316978
>> URL: https://svnweb.freebsd.org/changeset/base/316978
>>
>> Log:
>>   Import zstandard 1.1.4 in base
>>   
>>   zstandard is a new compression library/tool which is very fast at
>>   compression/decompression
>>   
>>   For now import as a private library
>>


-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

Re: svn commit: r316978 - in head: contrib/zstd etc/mtree lib lib/libzstd share/mk usr.bin usr.bin/zstd

[Conrad Meyer]

On Sat, Apr 15, 2017 at 4:37 PM, Rodney W. Grimes
 wrote:
> And we need this in base for what great purpose???

Can you tone down the condescension, Rod?  Being rude doesn't help you
make your case.

We already have zlib, bzip2, and xz in base.  zstd is just one more
that fills an important niche.

If you're not familiar with it, zstd is a relatively recent new
compression algorithm that provides a better compression/performance
point relative to zlib at any point along the -0/-9 curve.  It
performs favorably to bzip2 and xz as well, although it's not dominant
at all points relative to those two.  See the last figure in
https://clearlinux.org/blogs/linux-os-data-compression-options-comparing-behavior
for a better idea of what niche it fills.

Thanks,
Conrad
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r316978 - in head: contrib/zstd etc/mtree lib lib/libzstd share/mk usr.bin usr.bin/zstd

[Rodney W. Grimes]

And we need this in base for what great purpose???


> Author: bapt
> Date: Sat Apr 15 20:05:22 2017
> New Revision: 316978
> URL: https://svnweb.freebsd.org/changeset/base/316978
> 
> Log:
>   Import zstandard 1.1.4 in base
>   
>   zstandard is a new compression library/tool which is very fast at
>   compression/decompression
>   
>   For now import as a private library
> 
> Added:
>   head/contrib/zstd/
>  - copied from r316976, vendor/zstd/dist/
>   head/lib/libzstd/
>   head/lib/libzstd/Makefile   (contents, props changed)
>   head/usr.bin/zstd/
>   head/usr.bin/zstd/Makefile   (contents, props changed)
> Modified:
>   head/etc/mtree/BSD.usr.dist
>   head/lib/Makefile
>   head/share/mk/src.libnames.mk
>   head/usr.bin/Makefile
> 
> Modified: head/etc/mtree/BSD.usr.dist
> ==
> --- head/etc/mtree/BSD.usr.dist   Sat Apr 15 20:03:50 2017
> (r316977)
> +++ head/etc/mtree/BSD.usr.dist   Sat Apr 15 20:05:22 2017
> (r316978)
> @@ -17,6 +17,8 @@
>  ..
>  ucl
>  ..
> +zstd
> +..
>  ..
>  ..
>  lib
> 
> Modified: head/lib/Makefile
> ==
> --- head/lib/Makefile Sat Apr 15 20:03:50 2017(r316977)
> +++ head/lib/Makefile Sat Apr 15 20:05:22 2017(r316978)
> @@ -90,6 +90,7 @@ SUBDIR= ${SUBDIR_BOOTSTRAP} \
>   libxo \
>   liby \
>   libz \
> + libzstd \
>   ncurses
>  
>  # Inter-library dependencies.  When the makefile for a library contains LDADD
> 
> Added: head/lib/libzstd/Makefile
> ==
> --- /dev/null 00:00:00 1970   (empty, because file is newly added)
> +++ head/lib/libzstd/Makefile Sat Apr 15 20:05:22 2017(r316978)
> @@ -0,0 +1,34 @@
> +# $FreeBSD$
> +
> +LIB= zstd
> +SRCS=entropy_common.c \
> + error_private.c \
> + fse_decompress.c \
> + pool.c \
> + threading.c \
> + xxhash.c \
> + zstd_common.c \
> + fse_compress.c \
> + huf_compress.c \
> + zstd_compress.c \
> + zstdmt_compress.c \
> + huf_decompress.c \
> + zstd_decompress.c \
> + zbuff_common.c \
> + zbuff_compress.c \
> + zbuff_decompress.c \
> + cover.c \
> + divsufsort.c \
> + zdict.c
> +WARNS=   2
> +INCS=zstd.h
> +CFLAGS+= -I${ZSTDDIR}/lib -I${ZSTDDIR}/lib/common -DXXH_NAMESPACE=ZSTD_
> +
> +PRIVATELIB=  yes
> +
> +ZSTDDIR= ${SRCTOP}/contrib/zstd
> +.PATH:   ${ZSTDDIR}/lib/common ${ZSTDDIR}/lib/compress \
> + ${ZSTDDIR}/lib/decompress ${ZSTDDIR}/lib/deprecated \
> + ${ZSTDDIR}/lib/dictBuilder ${ZSTDDIR}/lib
> +
> +.include 
> 
> Modified: head/share/mk/src.libnames.mk
> ==
> --- head/share/mk/src.libnames.mk Sat Apr 15 20:03:50 2017
> (r316977)
> +++ head/share/mk/src.libnames.mk Sat Apr 15 20:05:22 2017
> (r316978)
> @@ -24,7 +24,8 @@ _PRIVATELIBS=   \
>   sqlite3 \
>   ssh \
>   ucl \
> - unbound
> + unbound \
> + zstd
>  
>  _INTERNALLIBS=   \
>   amu \
> 
> Modified: head/usr.bin/Makefile
> ==
> --- head/usr.bin/Makefile Sat Apr 15 20:03:50 2017(r316977)
> +++ head/usr.bin/Makefile Sat Apr 15 20:05:22 2017(r316978)
> @@ -189,7 +189,8 @@ SUBDIR=   alias \
>   xo \
>   xz \
>   xzdec \
> - yes
> + yes \
> + zstd
>  
>  # NB: keep these sorted by MK_* knobs
>  
> 
> Added: head/usr.bin/zstd/Makefile
> ==
> --- /dev/null 00:00:00 1970   (empty, because file is newly added)
> +++ head/usr.bin/zstd/MakefileSat Apr 15 20:05:22 2017
> (r316978)
> @@ -0,0 +1,21 @@
> +# $FreeBSD$
> +
> +PROG=zstd
> +SRCS=bench.c \
> + datagen.c \
> + dibio.c \
> + fileio.c \
> + zstdcli.c
> +
> +CFLAGS+= -I${SRCTOP}/contrib/zstd/programs \
> + -I${SRCTOP}/contrib/zstd/lib/common \
> + -I${SRCTOP}/contrib/zstd/lib/compress \
> + -I${SRCTOP}/contrib/zstd/lib/dictBuilder \
> + -I${SRCTOP}/contrib/zstd/lib \
> + -DXXH_NAMESPACE=ZSTD_
> +
> +WARNS=   2
> +LIBADD=  zstd
> +.PATH: ${SRCTOP}/contrib/zstd/programs
> +
> +.include 
> 
> 

-- 
Rod Grimes rgri...@freebsd.org
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r316978 - in head: contrib/zstd etc/mtree lib lib/libzstd share/mk usr.bin usr.bin/zstd

[Baptiste Daroussin]

Author: bapt
Date: Sat Apr 15 20:05:22 2017
New Revision: 316978
URL: https://svnweb.freebsd.org/changeset/base/316978

Log:
  Import zstandard 1.1.4 in base
  
  zstandard is a new compression library/tool which is very fast at
  compression/decompression
  
  For now import as a private library

Added:
  head/contrib/zstd/
 - copied from r316976, vendor/zstd/dist/
  head/lib/libzstd/
  head/lib/libzstd/Makefile   (contents, props changed)
  head/usr.bin/zstd/
  head/usr.bin/zstd/Makefile   (contents, props changed)
Modified:
  head/etc/mtree/BSD.usr.dist
  head/lib/Makefile
  head/share/mk/src.libnames.mk
  head/usr.bin/Makefile

Modified: head/etc/mtree/BSD.usr.dist
==
--- head/etc/mtree/BSD.usr.dist Sat Apr 15 20:03:50 2017(r316977)
+++ head/etc/mtree/BSD.usr.dist Sat Apr 15 20:05:22 2017(r316978)
@@ -17,6 +17,8 @@
 ..
 ucl
 ..
+zstd
+..
 ..
 ..
 lib

Modified: head/lib/Makefile
==
--- head/lib/Makefile   Sat Apr 15 20:03:50 2017(r316977)
+++ head/lib/Makefile   Sat Apr 15 20:05:22 2017(r316978)
@@ -90,6 +90,7 @@ SUBDIR=   ${SUBDIR_BOOTSTRAP} \
libxo \
liby \
libz \
+   libzstd \
ncurses
 
 # Inter-library dependencies.  When the makefile for a library contains LDADD

Added: head/lib/libzstd/Makefile
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ head/lib/libzstd/Makefile   Sat Apr 15 20:05:22 2017(r316978)
@@ -0,0 +1,34 @@
+# $FreeBSD$
+
+LIB=   zstd
+SRCS=  entropy_common.c \
+   error_private.c \
+   fse_decompress.c \
+   pool.c \
+   threading.c \
+   xxhash.c \
+   zstd_common.c \
+   fse_compress.c \
+   huf_compress.c \
+   zstd_compress.c \
+   zstdmt_compress.c \
+   huf_decompress.c \
+   zstd_decompress.c \
+   zbuff_common.c \
+   zbuff_compress.c \
+   zbuff_decompress.c \
+   cover.c \
+   divsufsort.c \
+   zdict.c
+WARNS= 2
+INCS=  zstd.h
+CFLAGS+=   -I${ZSTDDIR}/lib -I${ZSTDDIR}/lib/common -DXXH_NAMESPACE=ZSTD_
+
+PRIVATELIB=yes
+
+ZSTDDIR=   ${SRCTOP}/contrib/zstd
+.PATH: ${ZSTDDIR}/lib/common ${ZSTDDIR}/lib/compress \
+   ${ZSTDDIR}/lib/decompress ${ZSTDDIR}/lib/deprecated \
+   ${ZSTDDIR}/lib/dictBuilder ${ZSTDDIR}/lib
+
+.include 

Modified: head/share/mk/src.libnames.mk
==
--- head/share/mk/src.libnames.mk   Sat Apr 15 20:03:50 2017
(r316977)
+++ head/share/mk/src.libnames.mk   Sat Apr 15 20:05:22 2017
(r316978)
@@ -24,7 +24,8 @@ _PRIVATELIBS= \
sqlite3 \
ssh \
ucl \
-   unbound
+   unbound \
+   zstd
 
 _INTERNALLIBS= \
amu \

Modified: head/usr.bin/Makefile
==
--- head/usr.bin/Makefile   Sat Apr 15 20:03:50 2017(r316977)
+++ head/usr.bin/Makefile   Sat Apr 15 20:05:22 2017(r316978)
@@ -189,7 +189,8 @@ SUBDIR= alias \
xo \
xz \
xzdec \
-   yes
+   yes \
+   zstd
 
 # NB: keep these sorted by MK_* knobs
 

Added: head/usr.bin/zstd/Makefile
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ head/usr.bin/zstd/Makefile  Sat Apr 15 20:05:22 2017(r316978)
@@ -0,0 +1,21 @@
+# $FreeBSD$
+
+PROG=  zstd
+SRCS=  bench.c \
+   datagen.c \
+   dibio.c \
+   fileio.c \
+   zstdcli.c
+
+CFLAGS+=   -I${SRCTOP}/contrib/zstd/programs \
+   -I${SRCTOP}/contrib/zstd/lib/common \
+   -I${SRCTOP}/contrib/zstd/lib/compress \
+   -I${SRCTOP}/contrib/zstd/lib/dictBuilder \
+   -I${SRCTOP}/contrib/zstd/lib \
+   -DXXH_NAMESPACE=ZSTD_
+
+WARNS= 2
+LIBADD=zstd
+.PATH: ${SRCTOP}/contrib/zstd/programs
+
+.include 
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"