Ivan wrote:
Hi,
I'd like to ask whether two small archive-oriented commands, pack and
unpack, would be considered within the scope of GNU coreutils.
The idea is to provide a simple interface for a common operation that
currently requires knowing the particular archive/compression tool and
its syntax:
pack FORMAT SOURCE...
unpack ARCHIVE
For example:
pack tar.zst src/
pack zip file1 file2 dir/
unpack archive.tar.zst
unpack archive.zip
pack requires the output representation to be selected explicitly,
while unpack detects the existing representation automatically.
The intention is not to replace tar, gzip, xz, 7z, etc., nor to
expose all of their specialized functionality. It is to provide a
small generic interface for the basic "pack these files" / "give me
back these files" operations.
I have a working GPLv3+ reference implementation in C (current tip
v1.6.1):
https://github.com/IRodriguez13/pack-unpack
The current implementation uses libarchive deliberately rather than
implementing archive formats itself. It currently supports tar and
compressed tar variants, zip, 7z, and several raw compression formats.
Some properties of the current implementation are:
- silent successful operation, with -v for member listing;
- relative archive member names;
- atomic archive creation using a temporary file and rename;
- defensive extraction against absolute paths, "..", and
symlink/hardlink escapes;
- explicit overwrite policies;
- conservative metadata restoration by default, with -p for
additional metadata;
- regression tests for malformed and hostile archives;
- CI testing on GNU/Linux (glibc and musl), FreeBSD, and macOS,
including ASan/UBSan testing.
I realize that the libarchive dependency may itself be an important
consideration for coreutils, so I would especially appreciate feedback
on whether such a dependency would be considered a fundamental obstacle,
independently of the command interface.
Before attempting to adapt the implementation to coreutils conventions
or preparing patches, I'd like to know whether commands with this scope
are something the project would consider in principle.
There is also some historical precedent for the names pack and unpack
in older Unix systems, although those commands had different,
format-specific compression semantics. I mention this only as naming
context; this proposal is not intended to reproduce their historical
implementation or exact behavior.
Feedback on the interface, scope, naming, or whether this functionality
belongs elsewhere would be very welcome.
This is useful functionality.
I've previously used atool for this.
https://repology.org/project/libarchive/ is widely available,
so we probably wouldn't vendor it, rather, not building the
utilities if not available (like we do for runcon and chcon),
or building a stub if explicitly requested without the dependency.
Saying that, I'm not convinced coreutils is the right home for this.
Perhaps utils-linux may be a better fit, as that has general
non linux specific tools despite the name.
Also worth noting is the bsdtar tool which is part of libarchive,
and has very similar functionality, and is generally available
where libarchive is. Perhaps an avenue worth investigating
is adding pack/unpack as another tool to libarchive?
thanks for the clear presentation.
Padraig.
p.s. This is another email that wasn't sent from the list to me.
A common factor with the last one is "RFC" in the subject. hmm...
