Re: Tar extract behaviour changed
On Wed, Oct 30, 2019 at 12:08:09PM +1100, matthew green wrote: > nullfs is terrible. i've always known it has the potential > to double-cache some file when accessed on both sides.. but > i recently had a problem where nullfs mounted system was > stopping ffs from freeing deleted files. i couldn't figure > out where all my data was hiding. eventually i unmounted > the nullfs (which took a couple of minutes) and suddenly i > had 80% of my disk back as unused. It can't double-cache, since the data handling is always pushed through. There are potentially bugs in the name cache invalidation, e.g. when it holds a reference to the upper layer, it doesn't kill the reference as usual when the vnode is unlinked in the lower layer. That said, none of those really matter for this case. Joerg
re: Tar extract behaviour changed
Joerg Sonnenberger writes: > On Thu, Oct 24, 2019 at 06:56:57AM +0700, Robert Elz wrote: > > Date:Wed, 23 Oct 2019 23:30:47 +0200 > > From:Joerg Sonnenberger > > Message-ID: <20191023213047.ga73...@bec.de> > > > > | (1) Abuse of symlinks to shuffle the tree somewhere else. IMO whoever > > | wants to do that should be using null mounts and deal with it > > | appropiately in sysinst or whatever on their own. > > > > With that attitude we may as well simply delete symlink support from > > NetBSD and use only null mounts everywhere. That's not workable at all. > > Seriously? We have a well supported mechanism for splitting the base > system across different file systems. It's called mount points. We > support no way to install a system with symlinks in random places, that > has to be done by hand. If a user has such special needs to have such a > special setup with all the associated sources of problems, they can also > manage to update sets by hand and include -P. nullfs is terrible. i've always known it has the potential to double-cache some file when accessed on both sides.. but i recently had a problem where nullfs mounted system was stopping ffs from freeing deleted files. i couldn't figure out where all my data was hiding. eventually i unmounted the nullfs (which took a couple of minutes) and suddenly i had 80% of my disk back as unused. this is not a reasonable position to take until it's way less stupid of a feature. .mrg.
Re: Tar extract behaviour changed
On Thu, Oct 24, 2019 at 10:17:17PM +0300, Valery Ushakov wrote:
> On Thu, Oct 24, 2019 at 05:26:41 +0200, Martin Husemann wrote:
>
> > Deal with this properly in sysinst would mean:
> >
> > 1) run a script like:
> > rm -f /tmp/list
> > for s in *.${suffix}
> > do
> >for dir in $( tar tvf "$s" | egrep '^d' | awk '{ print $9}' )
> >do
> > readlink "$dir" && echo "$dir" >> /tmp/list
> >done
> > done
>
> Isn't that info already available from mtree files? IIRC, set.base
> has all of the directories, including those that are populated by
> other sets. So you should be able to extract that and run mtree to
> check it.
I'm not really sure that would be a lot better. My prefered solution is to
just add -P to the extract args unconditionally (as this restores as close
to what we had as previous behaviour). If there would be an explicit
symlinks option I'd use that (unconditionally), but for our sets it does
not buy us much.
Martin
Re: Tar extract behaviour changed
On Thu, Oct 24, 2019 at 05:26:41 +0200, Martin Husemann wrote:
> Deal with this properly in sysinst would mean:
>
> 1) run a script like:
> rm -f /tmp/list
> for s in *.${suffix}
> do
>for dir in $( tar tvf "$s" | egrep '^d' | awk '{ print $9}' )
>do
> readlink "$dir" && echo "$dir" >> /tmp/list
>done
> done
Isn't that info already available from mtree files? IIRC, set.base
has all of the directories, including those that are populated by
other sets. So you should be able to extract that and run mtree to
check it.
-uwe
Re: Tar extract behaviour changed
On Thu, Oct 24, 2019 at 06:56:57AM +0700, Robert Elz wrote: > Date:Wed, 23 Oct 2019 23:30:47 +0200 > From:Joerg Sonnenberger > Message-ID: <20191023213047.ga73...@bec.de> > > | (1) Abuse of symlinks to shuffle the tree somewhere else. IMO whoever > | wants to do that should be using null mounts and deal with it > | appropiately in sysinst or whatever on their own. > > With that attitude we may as well simply delete symlink support from > NetBSD and use only null mounts everywhere. That's not workable at all. Seriously? We have a well supported mechanism for splitting the base system across different file systems. It's called mount points. We support no way to install a system with symlinks in random places, that has to be done by hand. If a user has such special needs to have such a special setup with all the associated sources of problems, they can also manage to update sets by hand and include -P. > | For that one [rc.d chroot movements], there are a couple of options: > | (a) Move the directories under /etc from base.tgz to etc.tgz. > > So updates to the files in them don't happen during an upgrade, or > don't necessarily happen. Please read again what I said and take a look at the content of base.tgz. Joerg
Re: Tar extract behaviour changed
On Thu, Oct 24, 2019 at 06:56:57AM +0700, Robert Elz wrote:
> Date:Wed, 23 Oct 2019 23:30:47 +0200
> From:Joerg Sonnenberger
> Message-ID: <20191023213047.ga73...@bec.de>
>
> | (1) Abuse of symlinks to shuffle the tree somewhere else. IMO whoever
> | wants to do that should be using null mounts and deal with it
> | appropiately in sysinst or whatever on their own.
>
> With that attitude we may as well simply delete symlink support from
> NetBSD and use only null mounts everywhere. That's not workable at all.
Deal with this properly in sysinst would mean:
1) run a script like:
rm -f /tmp/list
for s in *.${suffix}
do
for dir in $( tar tvf "$s" | egrep '^d' | awk '{ print $9}' )
do
readlink "$dir" && echo "$dir" >> /tmp/list
done
done
2) if /tmp/list is not empty, create a message showing its contents
and ask the user something like:
The following directories are symbolic links and will be replaced
with plain directories:
$(cat /tmp/list)
Do you want to
a) proceed and replace the symbolic links?
b) trust the $s archive and extract without symlink checks?
x) abort installation and fix the local setup (e.g. via null mounts)
Option (b) then would add -P to the tar extraction flags.
This will not change anything for nearly all users, but be a pretty strange
experience for the few where it hits.
> ps: if the end solution is to simply use -P in sysinst during extraction,
> then we really also need to be generating signed sets, and have sysinst
> verify the dignatures before extracting.
I'd rather go this very hackish way and always add -P.
If you install sets via sysinst and do not trust the archive, you are doing
something wrong. A strange symlink attack would be no worse than a
bogus usr/bin/su or even bin/sh.
Having sysinst verify the SHA512 checksums that we provide with the sets
would be easy at least on some install media (and for fast-enough
architectures could even be done by default). Verifying the signatures
on the check sums could only happen for real releases and would require
a bit of infrastructure (probably close to what we would need to set up
for signed binary pkgs anyway).
Martin
Re: Tar extract behaviour changed
Date:Wed, 23 Oct 2019 23:30:47 +0200 From:Joerg Sonnenberger Message-ID: <20191023213047.ga73...@bec.de> | (1) Abuse of symlinks to shuffle the tree somewhere else. IMO whoever | wants to do that should be using null mounts and deal with it | appropiately in sysinst or whatever on their own. With that attitude we may as well simply delete symlink support from NetBSD and use only null mounts everywhere. That's not workable at all. However much cleaner it might be (quite aside from null mounts not being operational until the system has proceeded far enough into the boot sequence for the mount commands to have been run, which means not at all in single user mode) having perhaps dozens of null mounts, perhaps even more than that (with mounts on mounts, etc) is not a practical solution. | For that one [rc.d chroot movements], there are a couple of options: | (a) Move the directories under /etc from base.tgz to etc.tgz. So updates to the files in them don't happen during an upgrade, or don't necessarily happen. | (b) Don't use symlinks, but null mounts. Possibly even ro-mounts. | Fix up existing setups in the rc.d scripts. This is a case where that could work, and perhaps is a better methpd in any case. kre ps: if the end solution is to simply use -P in sysinst during extraction, then we really also need to be generating signed sets, and have sysinst verify the dignatures before extracting.
Re: Tar extract behaviour changed
On Wed, Oct 23, 2019 at 09:41:30PM -, Christos Zoulas wrote: > In article <20191023213047.ga73...@bec.de>, > Joerg Sonnenberger wrote: > >On Wed, Oct 23, 2019 at 04:58:05PM -, Christos Zoulas wrote: > >> I am not advocating for either, perhaps we should just add -P to the > >> extraction and get over it :-) > > > >From what I can tell there are two completely separate issues: > > > >(1) Abuse of symlinks to shuffle the tree somewhere else. IMO whoever > >wants to do that should be using null mounts and deal with it > >appropiately in sysinst or whatever on their own. > > > >(2) Certain rc.d scripts populate a chroot with files from /etc and > >replace the original locations with symlinks. > > > >For that one, there are a couple of options: > >(a) Move the directories under /etc from base.tgz to etc.tgz. > >(b) Don't use symlinks, but null mounts. Possibly even ro-mounts. > >Fix up existing setups in the rc.d scripts. > > > >(a) should be the easiest way to avoid that set of surprises. > > I think that the reason that those files are in the base set and not > in etc is because they are expected to be managed by the system and > not customizable by the user. If we move them to etc, then postinstall > should handle them to preserve the user experience (of not having to > extract etc to upgrade). It is not about the *files*, but the *directories*. I don't think they are very useful without content, but yeah, postinstall might need to know about creating them. Joerg
Re: Tar extract behaviour changed
In article <20191023213047.ga73...@bec.de>, Joerg Sonnenberger wrote: >On Wed, Oct 23, 2019 at 04:58:05PM -, Christos Zoulas wrote: >> I am not advocating for either, perhaps we should just add -P to the >> extraction and get over it :-) > >From what I can tell there are two completely separate issues: > >(1) Abuse of symlinks to shuffle the tree somewhere else. IMO whoever >wants to do that should be using null mounts and deal with it >appropiately in sysinst or whatever on their own. > >(2) Certain rc.d scripts populate a chroot with files from /etc and >replace the original locations with symlinks. > >For that one, there are a couple of options: >(a) Move the directories under /etc from base.tgz to etc.tgz. >(b) Don't use symlinks, but null mounts. Possibly even ro-mounts. >Fix up existing setups in the rc.d scripts. > >(a) should be the easiest way to avoid that set of surprises. I think that the reason that those files are in the base set and not in etc is because they are expected to be managed by the system and not customizable by the user. If we move them to etc, then postinstall should handle them to preserve the user experience (of not having to extract etc to upgrade). christos
Re: Tar extract behaviour changed
On Wed, Oct 23, 2019 at 04:58:05PM -, Christos Zoulas wrote: > I am not advocating for either, perhaps we should just add -P to the > extraction and get over it :-) >From what I can tell there are two completely separate issues: (1) Abuse of symlinks to shuffle the tree somewhere else. IMO whoever wants to do that should be using null mounts and deal with it appropiately in sysinst or whatever on their own. (2) Certain rc.d scripts populate a chroot with files from /etc and replace the original locations with symlinks. For that one, there are a couple of options: (a) Move the directories under /etc from base.tgz to etc.tgz. (b) Don't use symlinks, but null mounts. Possibly even ro-mounts. Fix up existing setups in the rc.d scripts. (a) should be the easiest way to avoid that set of surprises. Joerg
Re: Tar extract behaviour changed
On Wed, 23 Oct 2019, Christos Zoulas wrote: I am not advocating for either, perhaps we should just add -P to the extraction and get over it :-) This one gets my vote! :) ++--+---+ | Paul Goyette | PGP Key fingerprint: | E-mail addresses: | | (Retired) | FA29 0E3B 35AF E8AE 6651 | p...@whooppee.com | | Software Developer | 0786 F758 55DE 53BA 7731 | pgoye...@netbsd.org | ++--+---+
Re: Tar extract behaviour changed
In article <20191022200319.ga83...@bec.de>, Joerg Sonnenberger wrote: >On Tue, Oct 22, 2019 at 08:00:35PM +0200, Christian Groessler wrote: >> "tar" had an option to delete files which it is about to extract before >> extraction. Wouldn't this solve the "symlink" issue at hand? What am I >> missing? > >See the SECURITY section in the man page. Both -U and -P are ways to >dealing with this, but with different end result. Here are two simple patches: 1. Track symlinks that tar created and for them keep the current behavior of overwriting them, but extract through pre-existing ones: https://www.netbsd.org/~christos/track-symlinks.diff Pros: default behavior Cons: adds complexity, have not thought through all the possible scenarios, can slow down things when there are lots of symlinks. 2. Add a flag to just allow symlinks: https://www.netbsd.org/~christos/track-symlinks.diff Pros: simple and reduces the attack surface, gets the behavior we had before Cons: not standard, malicious tars can still do damage. I am not advocating for either, perhaps we should just add -P to the extraction and get over it :-) christos
Re: Tar extract behaviour changed
On Tue, Oct 22, 2019 at 08:00:35PM +0200, Christian Groessler wrote: > "tar" had an option to delete files which it is about to extract before > extraction. Wouldn't this solve the "symlink" issue at hand? What am I > missing? See the SECURITY section in the man page. Both -U and -P are ways to dealing with this, but with different end result. Joerg
Re: Tar extract behaviour changed
On 2019-10-22 17:43, Robert Elz wrote: Date:Tue, 22 Oct 2019 14:33:27 - (UTC) From:chris...@astron.com (Christos Zoulas) Message-ID: | Well, one of the use cases is when we don't have enough disk space in the | same partition, so that will not work out. No, I meant symlinks in the archive, not pre-existing ones. While I suppose there are uses for archives containing symlinks aimed all over the place, I'd tend to assume they're only for locally created archives (and so could be extracted with an option to allow them) - archives from elsewhere cannot expect to know where other (non-archive) files are to be located on every system that might extract the archive, so symlinks in the archive that don't (at least potentially) refer to other files in the archive are not usually going to be of any use. So the test would be, before creating a symlink from the archive, whether the target starts with / or enough ../ sequences to escape the root of the extraction (or any /../ sequence inside the symlink - that should never be needed) if any of those is found, and not exprssly permitted then the symlmk should not be extracted. Of course, anything named explicitly on the command line is also OK, If I do tar xf archive /etc/passwd that should do exactly what I told it to do, as should tar xf archive some/symlink(whatever the target is) (and the equiv for the pax & cpio interfaces, but perhaps not when reading the list of files from stdin, haven't really considered that case). Sorry for appearing dense, I briefly followed this thread. "tar" had an option to delete files which it is about to extract before extraction. Wouldn't this solve the "symlink" issue at hand? What am I missing? regards, chris
Re: Tar extract behaviour changed
Date:Tue, 22 Oct 2019 14:33:27 - (UTC) From:chris...@astron.com (Christos Zoulas) Message-ID: | Well, one of the use cases is when we don't have enough disk space in the | same partition, so that will not work out. No, I meant symlinks in the archive, not pre-existing ones. While I suppose there are uses for archives containing symlinks aimed all over the place, I'd tend to assume they're only for locally created archives (and so could be extracted with an option to allow them) - archives from elsewhere cannot expect to know where other (non-archive) files are to be located on every system that might extract the archive, so symlinks in the archive that don't (at least potentially) refer to other files in the archive are not usually going to be of any use. So the test would be, before creating a symlink from the archive, whether the target starts with / or enough ../ sequences to escape the root of the extraction (or any /../ sequence inside the symlink - that should never be needed) if any of those is found, and not exprssly permitted then the symlmk should not be extracted. Of course, anything named explicitly on the command line is also OK, If I do tar xf archive /etc/passwd that should do exactly what I told it to do, as should tar xf archive some/symlink(whatever the target is) (and the equiv for the pax & cpio interfaces, but perhaps not when reading the list of files from stdin, haven't really considered that case). kre
Re: Tar extract behaviour changed
In article <19306.1571753...@jinx.noi.kre.to>, Robert Elz wrote: >Date:Tue, 22 Oct 2019 14:27:39 +0200 >From:Joerg Sonnenberger >Message-ID: <20191022122739.ga86...@bec.de> > > | Extraction of entries in streamable formats happens in isolation. The > | archiver has no knowledge about pre-existing symlinks or whether the > | archive itself just created the symlink. > >It should be able to deduce something from the ctime of the symlink >if it wanted - if that predates the start of the extraction, then the >symlink was there in advance, if after, then (most probably) the archive >contained the symlink. That's a good idea :-) >chris...@astron.com said: > | because then we would have to normalize and check all symlinks in the > | archive (and do what? allow only the symlink pointing to an empty directory > | case? > >only allow symlinks pointing inside the tree being extracted most likely. Well, one of the use cases is when we don't have enough disk space in the same partition, so that will not work out. >But in both cases, when the archive is untrusted, avoiding all of this >is best, when it is trusted (particularly when the user created it themselves) >things ought to be more flexible. Right. christos
Re: Tar extract behaviour changed
On Tue, Oct 22, 2019 at 07:26:05AM +0200, Martin Husemann wrote: > On Tue, Oct 22, 2019 at 06:37:44AM +0700, Robert Elz wrote: > > Date:Mon, 21 Oct 2019 21:20:25 +0200 > > From:Joerg Sonnenberger > > Message-ID: <20191021192025.ga33...@bec.de> > > > > | That said, I don't really see a point in > > | allowing one form of arbitrary file replacement and not another. > > > > If we're thinking of it purely as protection against potentially > > malicious archives obtained from some random internet site, then > > nor do I > > I am not sure. Clearly / and .. are protecting against malicious archives. > But in my view a directory entry in the (potential malicious) archive > overwriting an existing symlink is something where the explicit wish of the > user running the extraction is not honored. Extraction of entries in streamable formats happens in isolation. The archiver has no knowledge about pre-existing symlinks or whether the archive itself just created the symlink. Joerg
Re: Tar extract behaviour changed
In article <20191022052605.ga...@mail.duskware.de>,
Martin Husemann wrote:
>On Tue, Oct 22, 2019 at 06:37:44AM +0700, Robert Elz wrote:
>> Date:Mon, 21 Oct 2019 21:20:25 +0200
>> From:Joerg Sonnenberger
>> Message-ID: <20191021192025.ga33...@bec.de>
>>
>> | That said, I don't really see a point in
>> | allowing one form of arbitrary file replacement and not another.
>>
>> If we're thinking of it purely as protection against potentially
>> malicious archives obtained from some random internet site, then
>> nor do I
>
>I am not sure. Clearly / and .. are protecting against malicious archives.
>But in my view a directory entry in the (potential malicious) archive
>overwriting an existing symlink is something where the explicit wish of the
>user running the extraction is not honored.
>
>The attack vector here would be someone modifying my file system
>placing malicious symlinks somewhere and later me running the
>extraction of the archive - which is very different from not trusting
>the archive in the first place.
>
Actually a malicious archive can first create the malicious symlink that
points outside the intended extraction tree and then overwrite the
destination. It is much harder to protect against, because then we would
have to normalize and check all symlinks in the archive (and do what?
allow only the symlink pointing to an empty directory case? -- the whole
thing sounds too complicated)
>The other open question is: given that we only have -P, we need to either:
>
> - make sysinst list all directories in the archive and check them for
> existing symlinks, then ask the user wether the existing symlink should
> be kept (and then add -P to the tar command line) or
> - simply use -P always on set extractions (where we already know that no
> .. or / should exist and we kind of trust the archives anyway)
>
>The current state silently breaks existing valid setups ("valid" of course
>in my view, as I personally ran into one that I created myself).
It is simple enough to add extra flags, but as joerg@ implied, it is more
security theater. It does protect against accidents though (tars created
with / files by).
christos
Re: Tar extract behaviour changed
--
J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig
> On 22. Oct 2019, at 07:26, Martin Husemann wrote:
>
> The current state silently breaks existing valid setups ("valid" of course
> in my view, as I personally ran into one that I created myself).
It breaks chrooted services, I got non-working "unbound" and "nsd".
Suppose this will hurt a bunch of installations when they
go from -8 to -9.
--
J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig
signature.asc
Description: Message signed with OpenPGP
Re: Tar extract behaviour changed
On Tue, Oct 22, 2019 at 06:37:44AM +0700, Robert Elz wrote:
> Date:Mon, 21 Oct 2019 21:20:25 +0200
> From:Joerg Sonnenberger
> Message-ID: <20191021192025.ga33...@bec.de>
>
> | That said, I don't really see a point in
> | allowing one form of arbitrary file replacement and not another.
>
> If we're thinking of it purely as protection against potentially
> malicious archives obtained from some random internet site, then
> nor do I
I am not sure. Clearly / and .. are protecting against malicious archives.
But in my view a directory entry in the (potential malicious) archive
overwriting an existing symlink is something where the explicit wish of the
user running the extraction is not honored.
The attack vector here would be someone modifying my file system
placing malicious symlinks somewhere and later me running the
extraction of the archive - which is very different from not trusting
the archive in the first place.
The other open question is: given that we only have -P, we need to either:
- make sysinst list all directories in the archive and check them for
existing symlinks, then ask the user wether the existing symlink should
be kept (and then add -P to the tar command line) or
- simply use -P always on set extractions (where we already know that no
.. or / should exist and we kind of trust the archives anyway)
The current state silently breaks existing valid setups ("valid" of course
in my view, as I personally ran into one that I created myself).
Martin
Re: Tar extract behaviour changed
Date:Mon, 21 Oct 2019 21:20:25 +0200 From:Joerg Sonnenberger Message-ID: <20191021192025.ga33...@bec.de> | That said, I don't really see a point in | allowing one form of arbitrary file replacement and not another. If we're thinking of it purely as protection against potentially malicious archives obtained from some random internet site, then nor do I - but that's a case where you don't really want to allow any of this, with the possible exception of symlinks: If I have deliberately set up a bunch of symlinks to direct parts of the extracted file tree to different mounted filesystems (if I don't have space to extract it all into one, and then move parts later, or simply don't want to waste the time doing that if it is a very big data set) then following my carefully positioned symlinks is not an issue -- as long as we're still not allowing .. or / the extraction must remain within my otherwise empty target tree, so the only existing symlinks it can see are the ones I deliberately placed there). But if instead we're considering an archive I created a few years back when I was short on space, and had a large collection of highly compressible files I wasn't likely to need any time soon, but needed the space in a hurry, so wasn't all that careful when creating the archive, then things are different. Now, when I need the files to exist again, I can find the space, but not in the same places as before, so I want it to follow symlinks, I still want to extract the files I archives using ../filename (when I wasn't in the correct parent directory for everything when I made the archive) but I don't want to put the few files archived using full pathnames back into those names, so I want the leading '/' removed, so I can still extract those few files, after which I will move them to where they now belong. I don't see a problem with a default -P to override all of the checks (even if it is different than what we had before) but I can certainly see a use for specific options for each of them, because there can always be situations where one is wanted and another is not. On the other hand, all that said, I personally have no such weird archives that I'm aware of, so I'm not likely to spend the time any time soon to make this happen (not that it should really be very difficult, it is mostly just option parsing and then testing the correct (different) option state in each of the cases rather than just a single "insecure" option). kre
Re: Tar extract behaviour changed
On Mon, Oct 21, 2019 at 05:34:44PM -, Christos Zoulas wrote: > In article <20191021163005.ga4...@bec.de>, > Joerg Sonnenberger wrote: > >On Mon, Oct 21, 2019 at 06:29:18AM -0700, Hisashi T Fujinaka wrote: > >> On Mon, 21 Oct 2019, Martin Husemann wrote: > >> > >> > On Mon, Oct 21, 2019 at 11:54:44AM +0200, J. Hannken-Illjes wrote: > >> > > Somewhere between Netbsd-8 and NetBSD-9 "tar" changed its behaviour > >> > > when it has to extract a directory and the path exists as a symlink. > >> > > >> > I still believe it should be fixed, but J?rg disagrees. You need to use > >> > -P > >> > now. See PR 54467. > >> > >> Yeah it's a real pain in my you-know-what. Is it Joerg vs everyone else? > > > >It is NetBSD pax vs every pretty much any maintained tar implementation. > > Indeed, and it is a security issue revert to the original tar behavior. > The new behavior is clearly better from a security PoV. > What I don't like about -P though is that it is an "all or nothing" deal: > > N FunctionPaX as Tar Libarchive Tar > -- > 1 keeping leading '/' -P -P > 2 extracting files containing ".." --insecure -P > 3 obeying existing symlinks default -P > > I would prefer to have a separate option that just does [3], but if upstream > does not think it is useful it is better to live with -P. Feel free to write a patch :) That said, I don't really see a point in allowing one form of arbitrary file replacement and not another. Joerg
Re: Tar extract behaviour changed
In article <20191021163005.ga4...@bec.de>, Joerg Sonnenberger wrote: >On Mon, Oct 21, 2019 at 06:29:18AM -0700, Hisashi T Fujinaka wrote: >> On Mon, 21 Oct 2019, Martin Husemann wrote: >> >> > On Mon, Oct 21, 2019 at 11:54:44AM +0200, J. Hannken-Illjes wrote: >> > > Somewhere between Netbsd-8 and NetBSD-9 "tar" changed its behaviour >> > > when it has to extract a directory and the path exists as a symlink. >> > >> > I still believe it should be fixed, but J?rg disagrees. You need to use -P >> > now. See PR 54467. >> >> Yeah it's a real pain in my you-know-what. Is it Joerg vs everyone else? > >It is NetBSD pax vs every pretty much any maintained tar implementation. Indeed, and it is a security issue revert to the original tar behavior. The new behavior is clearly better from a security PoV. What I don't like about -P though is that it is an "all or nothing" deal: N Function PaX as Tar Libarchive Tar -- 1 keeping leading '/' -P -P 2 extracting files containing ".." --insecure -P 3 obeying existing symlinks default -P I would prefer to have a separate option that just does [3], but if upstream does not think it is useful it is better to live with -P. christos
Re: Tar extract behaviour changed
On 21.10.2019 18:30, Joerg Sonnenberger wrote: > On Mon, Oct 21, 2019 at 06:29:18AM -0700, Hisashi T Fujinaka wrote: >> On Mon, 21 Oct 2019, Martin Husemann wrote: >> >>> On Mon, Oct 21, 2019 at 11:54:44AM +0200, J. Hannken-Illjes wrote: Somewhere between Netbsd-8 and NetBSD-9 "tar" changed its behaviour when it has to extract a directory and the path exists as a symlink. >>> >>> I still believe it should be fixed, but J?rg disagrees. You need to use -P >>> now. See PR 54467. >> >> Yeah it's a real pain in my you-know-what. Is it Joerg vs everyone else? > > It is NetBSD pax vs every pretty much any maintained tar implementation. > > Joerg > I have no personal preferences here, but syncing the behavior is a reasonable move. Especially aligning pax to libarchive/gtar. I have got another case where NetBSD pax is incompatible with gtar/libarchive, -t is printed in a different format. PAX tar output does not use C-escapes and outputs the paths literally. This difference used to break one test in lld for NetBSD. In my opinion it should be aligned. signature.asc Description: OpenPGP digital signature
Re: Tar extract behaviour changed
On Mon, Oct 21, 2019 at 06:29:18AM -0700, Hisashi T Fujinaka wrote: > On Mon, 21 Oct 2019, Martin Husemann wrote: > > > On Mon, Oct 21, 2019 at 11:54:44AM +0200, J. Hannken-Illjes wrote: > > > Somewhere between Netbsd-8 and NetBSD-9 "tar" changed its behaviour > > > when it has to extract a directory and the path exists as a symlink. > > > > I still believe it should be fixed, but J?rg disagrees. You need to use -P > > now. See PR 54467. > > Yeah it's a real pain in my you-know-what. Is it Joerg vs everyone else? It is NetBSD pax vs every pretty much any maintained tar implementation. Joerg
Re: Tar extract behaviour changed
On Mon, 21 Oct 2019, Martin Husemann wrote: On Mon, Oct 21, 2019 at 11:54:44AM +0200, J. Hannken-Illjes wrote: Somewhere between Netbsd-8 and NetBSD-9 "tar" changed its behaviour when it has to extract a directory and the path exists as a symlink. I still believe it should be fixed, but J?rg disagrees. You need to use -P now. See PR 54467. Yeah it's a real pain in my you-know-what. Is it Joerg vs everyone else? -- Hisashi T Fujinaka - ht...@twofifty.com BSEE + BSChem + BAEnglish + MSCS + $2.50 = coffee
Re: Tar extract behaviour changed
On Mon, Oct 21, 2019 at 11:54:44AM +0200, J. Hannken-Illjes wrote: > Somewhere between Netbsd-8 and NetBSD-9 "tar" changed its behaviour > when it has to extract a directory and the path exists as a symlink. I still believe it should be fixed, but Jörg disagrees. You need to use -P now. See PR 54467. Martin
Tar extract behaviour changed
Somewhere between Netbsd-8 and NetBSD-9 "tar" changed its behaviour when it has to extract a directory and the path exists as a symlink. The attached script on -8 gives: NetBSD 8.0_STABLE == Initial: total 8 drwxr-xr-x 2 hannken staff 512 Oct 21 11:47 realtarget drwxr-xr-x 2 hannken staff 512 Oct 21 11:47 target == Change to symlink: total 4 drwxr-xr-x 2 hannken staff 512 Oct 21 11:47 realtarget lrwxr-xr-x 1 hannken staff 10 Oct 21 11:47 target -> realtarget == After extract: total 4 drwxr-xr-x 2 hannken staff 512 Oct 21 11:47 realtarget lrwxr-xr-x 1 hannken staff 10 Oct 21 11:47 target -> realtarget On -9 it gives: NetBSD 9.0_BETA == Initial: total 4 drwxr-xr-x 2 root wheel 512 Oct 21 11:48 realtarget drwxr-xr-x 2 root wheel 512 Oct 21 11:48 target == Change to symlink: total 2 drwxr-xr-x 2 root wheel 512 Oct 21 11:48 realtarget lrwxr-xr-x 1 root wheel 10 Oct 21 11:48 target -> realtarget == After extract: total 4 drwxr-xr-x 2 root wheel 512 Oct 21 11:48 realtarget drwxr-xr-x 2 root wheel 512 Oct 21 11:48 target Here "target" was changed from symlink to directory. On NetBSD-9 extracting the "base" set overrides the symlink "/etc/unbound" with a directory and therefore unbound fails to start. Is this a bug in "tar" or is there a switch to get the old behaviour back? -- J. Hannken-Illjes - hann...@eis.cs.tu-bs.de - TU Braunschweig (Germany) #! /bin/sh uname -sr mkdir junk mkdir junk/target junk/realtarget printf "\n== Initial:\n" ; ls -l junk tar -c -f junk.tar junk rmdir junk/target ln -s realtarget junk/target printf "\n== Change to symlink:\n" ; ls -l junk tar -x -f junk.tar printf "\n== After extract:\n" ; ls -l junk rm -rf junk junk.tar
