Re: svn commit: r316980 - head/contrib/zstd/programs
On Tue, Dec 05, 2017 at 08:42:11PM -0500, Allan Jude wrote: > On 2017-12-05 20:17, Conrad Meyer wrote: > > Ping. Please revert this change. If you want zstd to behave this > > way, pursue it upstream first. > > > > Thanks, > > Conrad > > > > On Wed, Nov 15, 2017 at 7:38 PM, Conrad Meyerwrote: > >> Please revert this change. > >> > >> First, it introduces the POLA-violating behavior that zstdcat deletes > >> its source files. This is not how zcat/bzcat behaves. > >> > >> Second, it introduces a needless behavioral difference between FreeBSD > >> zstd and the rest of the world's zstd. The zstd documentation we ship > >> continues to claim that zstd preserves source files by default, yet > >> this change makes that documentation exactly backwards. While we can > >> change FreeBSD's documentation to accommodate the change, we can't > >> change Google results. > >> > >> Thanks, > >> Conrad > >> > >> On Sat, Apr 15, 2017 at 1:15 PM, Baptiste Daroussin > >> wrote: > >>> Author: bapt > >>> Date: Sat Apr 15 20:15:44 2017 > >>> New Revision: 316980 > >>> URL: https://svnweb.freebsd.org/changeset/base/316980 > >>> > >>> Log: > >>> Change some default to make zstd a dropin replacement for gzip,bzip etc > >>> in most cases > >>> > >>> Changes ares: > >>> - quiet by default > >>> - remove the source files one compression completion by default > >>> > >>> Modified: > >>> head/contrib/zstd/programs/fileio.c > >>> head/contrib/zstd/programs/zstdcli.c > >>> > >>> Modified: head/contrib/zstd/programs/fileio.c > >>> == > >>> --- head/contrib/zstd/programs/fileio.c Sat Apr 15 20:06:24 2017 > >>> (r316979) > >>> +++ head/contrib/zstd/programs/fileio.c Sat Apr 15 20:15:44 2017 > >>> (r316980) > >>> @@ -138,7 +138,7 @@ static U32 g_dictIDFlag = 1; > >>> void FIO_setDictIDFlag(unsigned dictIDFlag) { g_dictIDFlag = dictIDFlag; > >>> } > >>> static U32 g_checksumFlag = 1; > >>> void FIO_setChecksumFlag(unsigned checksumFlag) { g_checksumFlag = > >>> checksumFlag; } > >>> -static U32 g_removeSrcFile = 0; > >>> +static U32 g_removeSrcFile = 1; > >>> void FIO_setRemoveSrcFile(unsigned flag) { g_removeSrcFile = (flag>0); } > >>> static U32 g_memLimit = 0; > >>> void FIO_setMemLimit(unsigned memLimit) { g_memLimit = memLimit; } > >>> > >>> Modified: head/contrib/zstd/programs/zstdcli.c > >>> == > >>> --- head/contrib/zstd/programs/zstdcli.cSat Apr 15 20:06:24 2017 > >>> (r316979) > >>> +++ head/contrib/zstd/programs/zstdcli.cSat Apr 15 20:15:44 2017 > >>> (r316980) > >>> @@ -61,7 +61,7 @@ > >>> #define MB *(1 <<20) > >>> #define GB *(1U<<30) > >>> > >>> -#define DEFAULT_DISPLAY_LEVEL 2 > >>> +#define DEFAULT_DISPLAY_LEVEL 1 > >>> > >>> static const char*g_defaultDictName = "dictionary"; > >>> static const unsigned g_defaultMaxDictSize = 110 KB; > >>> > > > > Upstream recommends we create an additional hardlink, called 'zz' (like > xz but zstd), to do such compatibility, and keep 'zstd' the same as zstd > on every other distro. > Actually I would prefer not to do it and simply adapt newsyslog to be more flexible. it is the only consumer of the zstd command. Bapt signature.asc Description: PGP signature
Re: svn commit: r316980 - head/contrib/zstd/programs
On Tue, Dec 05, 2017 at 05:17:22PM -0800, Conrad Meyer wrote: > Ping. Please revert this change. If you want zstd to behave this > way, pursue it upstream first. This is planned, for now I first need to modify newsyslog to accept more arguments for the compression program, then I'll switch back zstd to a vanilla. I am not pushing into the direction of yet another hardlink > > Thanks, > Conrad > > On Wed, Nov 15, 2017 at 7:38 PM, Conrad Meyerwrote: > > Please revert this change. > > > > First, it introduces the POLA-violating behavior that zstdcat deletes > > its source files. This is not how zcat/bzcat behaves. > > > > Second, it introduces a needless behavioral difference between FreeBSD > > zstd and the rest of the world's zstd. The zstd documentation we ship > > continues to claim that zstd preserves source files by default, yet > > this change makes that documentation exactly backwards. While we can > > change FreeBSD's documentation to accommodate the change, we can't > > change Google results. > > > > Thanks, > > Conrad > > > > On Sat, Apr 15, 2017 at 1:15 PM, Baptiste Daroussin > > wrote: > >> Author: bapt > >> Date: Sat Apr 15 20:15:44 2017 > >> New Revision: 316980 > >> URL: https://svnweb.freebsd.org/changeset/base/316980 > >> > >> Log: > >> Change some default to make zstd a dropin replacement for gzip,bzip etc > >> in most cases > >> > >> Changes ares: > >> - quiet by default > >> - remove the source files one compression completion by default > >> > >> Modified: > >> head/contrib/zstd/programs/fileio.c > >> head/contrib/zstd/programs/zstdcli.c > >> > >> Modified: head/contrib/zstd/programs/fileio.c > >> == > >> --- head/contrib/zstd/programs/fileio.c Sat Apr 15 20:06:24 2017 > >> (r316979) > >> +++ head/contrib/zstd/programs/fileio.c Sat Apr 15 20:15:44 2017 > >> (r316980) > >> @@ -138,7 +138,7 @@ static U32 g_dictIDFlag = 1; > >> void FIO_setDictIDFlag(unsigned dictIDFlag) { g_dictIDFlag = dictIDFlag; } > >> static U32 g_checksumFlag = 1; > >> void FIO_setChecksumFlag(unsigned checksumFlag) { g_checksumFlag = > >> checksumFlag; } > >> -static U32 g_removeSrcFile = 0; > >> +static U32 g_removeSrcFile = 1; > >> void FIO_setRemoveSrcFile(unsigned flag) { g_removeSrcFile = (flag>0); } > >> static U32 g_memLimit = 0; > >> void FIO_setMemLimit(unsigned memLimit) { g_memLimit = memLimit; } > >> > >> Modified: head/contrib/zstd/programs/zstdcli.c > >> == > >> --- head/contrib/zstd/programs/zstdcli.cSat Apr 15 20:06:24 2017 > >> (r316979) > >> +++ head/contrib/zstd/programs/zstdcli.cSat Apr 15 20:15:44 2017 > >> (r316980) > >> @@ -61,7 +61,7 @@ > >> #define MB *(1 <<20) > >> #define GB *(1U<<30) > >> > >> -#define DEFAULT_DISPLAY_LEVEL 2 > >> +#define DEFAULT_DISPLAY_LEVEL 1 > >> > >> static const char*g_defaultDictName = "dictionary"; > >> static const unsigned g_defaultMaxDictSize = 110 KB; > >> signature.asc Description: PGP signature
Re: svn commit: r316980 - head/contrib/zstd/programs
On 2017-12-05 20:17, Conrad Meyer wrote: > Ping. Please revert this change. If you want zstd to behave this > way, pursue it upstream first. > > Thanks, > Conrad > > On Wed, Nov 15, 2017 at 7:38 PM, Conrad Meyerwrote: >> Please revert this change. >> >> First, it introduces the POLA-violating behavior that zstdcat deletes >> its source files. This is not how zcat/bzcat behaves. >> >> Second, it introduces a needless behavioral difference between FreeBSD >> zstd and the rest of the world's zstd. The zstd documentation we ship >> continues to claim that zstd preserves source files by default, yet >> this change makes that documentation exactly backwards. While we can >> change FreeBSD's documentation to accommodate the change, we can't >> change Google results. >> >> Thanks, >> Conrad >> >> On Sat, Apr 15, 2017 at 1:15 PM, Baptiste Daroussin wrote: >>> Author: bapt >>> Date: Sat Apr 15 20:15:44 2017 >>> New Revision: 316980 >>> URL: https://svnweb.freebsd.org/changeset/base/316980 >>> >>> Log: >>> Change some default to make zstd a dropin replacement for gzip,bzip etc >>> in most cases >>> >>> Changes ares: >>> - quiet by default >>> - remove the source files one compression completion by default >>> >>> Modified: >>> head/contrib/zstd/programs/fileio.c >>> head/contrib/zstd/programs/zstdcli.c >>> >>> Modified: head/contrib/zstd/programs/fileio.c >>> == >>> --- head/contrib/zstd/programs/fileio.c Sat Apr 15 20:06:24 2017 >>> (r316979) >>> +++ head/contrib/zstd/programs/fileio.c Sat Apr 15 20:15:44 2017 >>> (r316980) >>> @@ -138,7 +138,7 @@ static U32 g_dictIDFlag = 1; >>> void FIO_setDictIDFlag(unsigned dictIDFlag) { g_dictIDFlag = dictIDFlag; } >>> static U32 g_checksumFlag = 1; >>> void FIO_setChecksumFlag(unsigned checksumFlag) { g_checksumFlag = >>> checksumFlag; } >>> -static U32 g_removeSrcFile = 0; >>> +static U32 g_removeSrcFile = 1; >>> void FIO_setRemoveSrcFile(unsigned flag) { g_removeSrcFile = (flag>0); } >>> static U32 g_memLimit = 0; >>> void FIO_setMemLimit(unsigned memLimit) { g_memLimit = memLimit; } >>> >>> Modified: head/contrib/zstd/programs/zstdcli.c >>> == >>> --- head/contrib/zstd/programs/zstdcli.cSat Apr 15 20:06:24 2017 >>> (r316979) >>> +++ head/contrib/zstd/programs/zstdcli.cSat Apr 15 20:15:44 2017 >>> (r316980) >>> @@ -61,7 +61,7 @@ >>> #define MB *(1 <<20) >>> #define GB *(1U<<30) >>> >>> -#define DEFAULT_DISPLAY_LEVEL 2 >>> +#define DEFAULT_DISPLAY_LEVEL 1 >>> >>> static const char*g_defaultDictName = "dictionary"; >>> static const unsigned g_defaultMaxDictSize = 110 KB; >>> > Upstream recommends we create an additional hardlink, called 'zz' (like xz but zstd), to do such compatibility, and keep 'zstd' the same as zstd on every other distro. -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: svn commit: r316980 - head/contrib/zstd/programs
Ping. Please revert this change. If you want zstd to behave this way, pursue it upstream first. Thanks, Conrad On Wed, Nov 15, 2017 at 7:38 PM, Conrad Meyerwrote: > Please revert this change. > > First, it introduces the POLA-violating behavior that zstdcat deletes > its source files. This is not how zcat/bzcat behaves. > > Second, it introduces a needless behavioral difference between FreeBSD > zstd and the rest of the world's zstd. The zstd documentation we ship > continues to claim that zstd preserves source files by default, yet > this change makes that documentation exactly backwards. While we can > change FreeBSD's documentation to accommodate the change, we can't > change Google results. > > Thanks, > Conrad > > On Sat, Apr 15, 2017 at 1:15 PM, Baptiste Daroussin wrote: >> Author: bapt >> Date: Sat Apr 15 20:15:44 2017 >> New Revision: 316980 >> URL: https://svnweb.freebsd.org/changeset/base/316980 >> >> Log: >> Change some default to make zstd a dropin replacement for gzip,bzip etc >> in most cases >> >> Changes ares: >> - quiet by default >> - remove the source files one compression completion by default >> >> Modified: >> head/contrib/zstd/programs/fileio.c >> head/contrib/zstd/programs/zstdcli.c >> >> Modified: head/contrib/zstd/programs/fileio.c >> == >> --- head/contrib/zstd/programs/fileio.c Sat Apr 15 20:06:24 2017 >> (r316979) >> +++ head/contrib/zstd/programs/fileio.c Sat Apr 15 20:15:44 2017 >> (r316980) >> @@ -138,7 +138,7 @@ static U32 g_dictIDFlag = 1; >> void FIO_setDictIDFlag(unsigned dictIDFlag) { g_dictIDFlag = dictIDFlag; } >> static U32 g_checksumFlag = 1; >> void FIO_setChecksumFlag(unsigned checksumFlag) { g_checksumFlag = >> checksumFlag; } >> -static U32 g_removeSrcFile = 0; >> +static U32 g_removeSrcFile = 1; >> void FIO_setRemoveSrcFile(unsigned flag) { g_removeSrcFile = (flag>0); } >> static U32 g_memLimit = 0; >> void FIO_setMemLimit(unsigned memLimit) { g_memLimit = memLimit; } >> >> Modified: head/contrib/zstd/programs/zstdcli.c >> == >> --- head/contrib/zstd/programs/zstdcli.cSat Apr 15 20:06:24 2017 >>(r316979) >> +++ head/contrib/zstd/programs/zstdcli.cSat Apr 15 20:15:44 2017 >>(r316980) >> @@ -61,7 +61,7 @@ >> #define MB *(1 <<20) >> #define GB *(1U<<30) >> >> -#define DEFAULT_DISPLAY_LEVEL 2 >> +#define DEFAULT_DISPLAY_LEVEL 1 >> >> static const char*g_defaultDictName = "dictionary"; >> static const unsigned g_defaultMaxDictSize = 110 KB; >> ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r316980 - head/contrib/zstd/programs
On Fri, Nov 17, 2017 at 1:34 AM, Baptiste Daroussinwrote: > On Wed, Nov 15, 2017 at 07:38:13PM -0800, Conrad Meyer wrote: >> Please revert this change. >> >> First, it introduces the POLA-violating behavior that zstdcat deletes >> its source files. This is not how zcat/bzcat behaves. > > I have modified zstdcat to behave like zcat/bzcat. > > The commit you stated is exactly to ensure the zstd(1) command is behaving > like > xz, gzip, etc (to the exception of zstdcat which was buggy) this commit is > needed to have those tools a drop-in replacement for other compression > programs. > (which is necessary for example to have newsyslog being able to use zstd.) > > I committed a change needed in base and I will start a discussion with > upstream > > Best regards, > Bapt Hi Bapt, I don't think that's a good enough reason to differ from upstream. Furthermore, the change isn't documented. For compatibility with gzip/xz, you could simply add a new FreeBSD-specific zstd frontend with the behavior you want — instead of changing every other frontend. That way the behavior and documentation would match both the documentation we ship and the upstream documentation and behavior. No surprises for anyone. I really want to emphasize that *deleting user files when we claim we will not* is an awful design choice to make. I think this change should be reverted until at minimum our documentation is updated to inform users we do not --keep by default. (I think we should stay with upstream regardless, but if we're going to make a major change like this it MUST be documented.) Best, Conrad ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
Re: svn commit: r316980 - head/contrib/zstd/programs
On Wed, Nov 15, 2017 at 07:38:13PM -0800, Conrad Meyer wrote: > Please revert this change. > > First, it introduces the POLA-violating behavior that zstdcat deletes > its source files. This is not how zcat/bzcat behaves. I have modified zstdcat to behave like zcat/bzcat. The commit you stated is exactly to ensure the zstd(1) command is behaving like xz, gzip, etc (to the exception of zstdcat which was buggy) this commit is needed to have those tools a drop-in replacement for other compression programs. (which is necessary for example to have newsyslog being able to use zstd.) I committed a change needed in base and I will start a discussion with upstream Best regards, Bapt signature.asc Description: PGP signature
Re: svn commit: r316980 - head/contrib/zstd/programs
On 2017-11-16 04:04, Baptiste Daroussin wrote: > On Wed, Nov 15, 2017 at 07:38:13PM -0800, Conrad Meyer wrote: >> Please revert this change. >> >> First, it introduces the POLA-violating behavior that zstdcat deletes >> its source files. This is not how zcat/bzcat behaves. > > This is not a POLA-violating behavior, this is a bug! that I introduced. >> >> Second, it introduces a needless behavioral difference between FreeBSD >> zstd and the rest of the world's zstd. The zstd documentation we ship >> continues to claim that zstd preserves source files by default, yet >> this change makes that documentation exactly backwards. While we can >> change FreeBSD's documentation to accommodate the change, we can't >> change Google results. >> > > The difference has been made so that zstd follow by default the same behaviour > has gzip/bzip2/xz so it can be a dropped in replacement. > > The argument about the documentation is however a good one. Let me do first > some > tests to ensure restoring the initial behaviour does not break existing usage > in > base > > Best regards, > Bapt > I think in this case, it is safer to surprise the user by NOT deleting a file, than to surprise a user by DELETING a file. If you really want to modify the behaviour (I suggest we don't, and stick closer to upstream zstd), then you'll want to set the 'delete' flag for the specific invocation cases inside zstdcli.c, rather than modifying the default as was done in this commit. I think we can deal with changing the default verbosity level. I think if we want to compromise, we make 2 additional hard links, zzip and zunzip that maintain the gzip like behaviour, since those will not conflict with the documentation that exists in the rest of the world for zstd(1) -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: svn commit: r316980 - head/contrib/zstd/programs
On Wed, Nov 15, 2017 at 07:38:13PM -0800, Conrad Meyer wrote: > Please revert this change. > > First, it introduces the POLA-violating behavior that zstdcat deletes > its source files. This is not how zcat/bzcat behaves. This is not a POLA-violating behavior, this is a bug! that I introduced. > > Second, it introduces a needless behavioral difference between FreeBSD > zstd and the rest of the world's zstd. The zstd documentation we ship > continues to claim that zstd preserves source files by default, yet > this change makes that documentation exactly backwards. While we can > change FreeBSD's documentation to accommodate the change, we can't > change Google results. > The difference has been made so that zstd follow by default the same behaviour has gzip/bzip2/xz so it can be a dropped in replacement. The argument about the documentation is however a good one. Let me do first some tests to ensure restoring the initial behaviour does not break existing usage in base Best regards, Bapt signature.asc Description: PGP signature
Re: svn commit: r316980 - head/contrib/zstd/programs
Please revert this change. First, it introduces the POLA-violating behavior that zstdcat deletes its source files. This is not how zcat/bzcat behaves. Second, it introduces a needless behavioral difference between FreeBSD zstd and the rest of the world's zstd. The zstd documentation we ship continues to claim that zstd preserves source files by default, yet this change makes that documentation exactly backwards. While we can change FreeBSD's documentation to accommodate the change, we can't change Google results. Thanks, Conrad On Sat, Apr 15, 2017 at 1:15 PM, Baptiste Daroussinwrote: > Author: bapt > Date: Sat Apr 15 20:15:44 2017 > New Revision: 316980 > URL: https://svnweb.freebsd.org/changeset/base/316980 > > Log: > Change some default to make zstd a dropin replacement for gzip,bzip etc > in most cases > > Changes ares: > - quiet by default > - remove the source files one compression completion by default > > Modified: > head/contrib/zstd/programs/fileio.c > head/contrib/zstd/programs/zstdcli.c > > Modified: head/contrib/zstd/programs/fileio.c > == > --- head/contrib/zstd/programs/fileio.c Sat Apr 15 20:06:24 2017 > (r316979) > +++ head/contrib/zstd/programs/fileio.c Sat Apr 15 20:15:44 2017 > (r316980) > @@ -138,7 +138,7 @@ static U32 g_dictIDFlag = 1; > void FIO_setDictIDFlag(unsigned dictIDFlag) { g_dictIDFlag = dictIDFlag; } > static U32 g_checksumFlag = 1; > void FIO_setChecksumFlag(unsigned checksumFlag) { g_checksumFlag = > checksumFlag; } > -static U32 g_removeSrcFile = 0; > +static U32 g_removeSrcFile = 1; > void FIO_setRemoveSrcFile(unsigned flag) { g_removeSrcFile = (flag>0); } > static U32 g_memLimit = 0; > void FIO_setMemLimit(unsigned memLimit) { g_memLimit = memLimit; } > > Modified: head/contrib/zstd/programs/zstdcli.c > == > --- head/contrib/zstd/programs/zstdcli.cSat Apr 15 20:06:24 2017 > (r316979) > +++ head/contrib/zstd/programs/zstdcli.cSat Apr 15 20:15:44 2017 > (r316980) > @@ -61,7 +61,7 @@ > #define MB *(1 <<20) > #define GB *(1U<<30) > > -#define DEFAULT_DISPLAY_LEVEL 2 > +#define DEFAULT_DISPLAY_LEVEL 1 > > static const char*g_defaultDictName = "dictionary"; > static const unsigned g_defaultMaxDictSize = 110 KB; > ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
svn commit: r316980 - head/contrib/zstd/programs
Author: bapt Date: Sat Apr 15 20:15:44 2017 New Revision: 316980 URL: https://svnweb.freebsd.org/changeset/base/316980 Log: Change some default to make zstd a dropin replacement for gzip,bzip etc in most cases Changes ares: - quiet by default - remove the source files one compression completion by default Modified: head/contrib/zstd/programs/fileio.c head/contrib/zstd/programs/zstdcli.c Modified: head/contrib/zstd/programs/fileio.c == --- head/contrib/zstd/programs/fileio.c Sat Apr 15 20:06:24 2017 (r316979) +++ head/contrib/zstd/programs/fileio.c Sat Apr 15 20:15:44 2017 (r316980) @@ -138,7 +138,7 @@ static U32 g_dictIDFlag = 1; void FIO_setDictIDFlag(unsigned dictIDFlag) { g_dictIDFlag = dictIDFlag; } static U32 g_checksumFlag = 1; void FIO_setChecksumFlag(unsigned checksumFlag) { g_checksumFlag = checksumFlag; } -static U32 g_removeSrcFile = 0; +static U32 g_removeSrcFile = 1; void FIO_setRemoveSrcFile(unsigned flag) { g_removeSrcFile = (flag>0); } static U32 g_memLimit = 0; void FIO_setMemLimit(unsigned memLimit) { g_memLimit = memLimit; } Modified: head/contrib/zstd/programs/zstdcli.c == --- head/contrib/zstd/programs/zstdcli.cSat Apr 15 20:06:24 2017 (r316979) +++ head/contrib/zstd/programs/zstdcli.cSat Apr 15 20:15:44 2017 (r316980) @@ -61,7 +61,7 @@ #define MB *(1 <<20) #define GB *(1U<<30) -#define DEFAULT_DISPLAY_LEVEL 2 +#define DEFAULT_DISPLAY_LEVEL 1 static const char*g_defaultDictName = "dictionary"; static const unsigned g_defaultMaxDictSize = 110 KB; ___ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"