Re: lldb ...
Sent from my iPhone...
> On Sep 10, 2016, at 16:51, Rainer Müller wrote:
>
> On 2016-09-10 17:52, Jeremy Huddleston Sequoia wrote:
>>> On OS X 10.10 Yosemite, signing only the ggdb binary was certainly
>>> enough. I cannot reproduce this on macOS 10.12 Sierra, so the
>>> requirements might have changed.
>>
>> 10.10 predates SIP and related hardening around ptrace(). That
>> version is so far in my rearview that I forget the details there,
>> sorry. I'll have to dig into it, but it certainly seems wrong to me
>> that a process could become privileged if it linked against unsigned
>> libraries.
>
> I would assume if we find a solution that passes the current
> restrictions on Sierra that will also work for older releases with less
> strict checking.
>
> I got gdb to work now on Sierra now. In fact I did not even have to sign
> any of the libraries it links to.
>
>
> $ otool -L /opt/local/bin/ggdb |awk 'NR>1 {print $1}' \
>|grep '^/opt/local' | xargs -I{} codesign -d -v {}
> /opt/local/lib/libintl.8.dylib: code object is not signed at all
> /opt/local/lib/libncurses.6.dylib: code object is not signed at all
> /opt/local/lib/libz.1.dylib: code object is not signed at all
> /opt/local/lib/libiconv.2.dylib: code object is not signed at all
> /opt/local/lib/libexpat.1.dylib: code object is not signed at all
>
> $ /opt/local/bin/ggdb -q /opt/local/bin/curl
> Reading symbols from /opt/local/bin/curl...(no debugging symbols
> found)...done.
> (gdb) r
> Starting program: /opt/local/bin/curl
> warning: unhandled dyld version (15)
> curl: try 'curl --help' or 'curl --manual' for more information
> [Inferior 1 (process 6964) exited with code 02]
> (gdb) q
Hmm. That isn't what I'd expect. Gonna need to check why that is. It looks
like CS_RESTRICT isn't implying CS_HARD like I thought it should.
>
> The main problem I encountered was that the setgid for the procmod group
> seems to interfere with the validation now. Once I removed that by
> changing the permissions to a regular 0755, I can use the code-signed
> ggdb just fine to debug other programs.
>
> By the way, as I did lots of trial and error, is there a way to get
> debug output (from taskgated?) to see why task_for_pid() was denied?
Is it not being logged? You should see it in the system log (Console.app, log
collect, etc).
>
> Rainer
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev
Re: lldb ...
On 2016-09-10 17:52, Jeremy Huddleston Sequoia wrote:
>> On OS X 10.10 Yosemite, signing only the ggdb binary was certainly
>> enough. I cannot reproduce this on macOS 10.12 Sierra, so the
>> requirements might have changed.
>
> 10.10 predates SIP and related hardening around ptrace(). That
> version is so far in my rearview that I forget the details there,
> sorry. I'll have to dig into it, but it certainly seems wrong to me
> that a process could become privileged if it linked against unsigned
> libraries.
I would assume if we find a solution that passes the current
restrictions on Sierra that will also work for older releases with less
strict checking.
I got gdb to work now on Sierra now. In fact I did not even have to sign
any of the libraries it links to.
$ otool -L /opt/local/bin/ggdb |awk 'NR>1 {print $1}' \
|grep '^/opt/local' | xargs -I{} codesign -d -v {}
/opt/local/lib/libintl.8.dylib: code object is not signed at all
/opt/local/lib/libncurses.6.dylib: code object is not signed at all
/opt/local/lib/libz.1.dylib: code object is not signed at all
/opt/local/lib/libiconv.2.dylib: code object is not signed at all
/opt/local/lib/libexpat.1.dylib: code object is not signed at all
$ /opt/local/bin/ggdb -q /opt/local/bin/curl
Reading symbols from /opt/local/bin/curl...(no debugging symbols
found)...done.
(gdb) r
Starting program: /opt/local/bin/curl
warning: unhandled dyld version (15)
curl: try 'curl --help' or 'curl --manual' for more information
[Inferior 1 (process 6964) exited with code 02]
(gdb) q
The main problem I encountered was that the setgid for the procmod group
seems to interfere with the validation now. Once I removed that by
changing the permissions to a regular 0755, I can use the code-signed
ggdb just fine to debug other programs.
By the way, as I did lots of trial and error, is there a way to get
debug output (from taskgated?) to see why task_for_pid() was denied?
Rainer
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev
Re: Fakeroot destrooting [Was: Re: lldb ...]
> On Sep 10, 2016, at 09:38, Clemens Lang wrote: > > Hi, > > On Sat, Sep 10, 2016 at 09:14:16AM -0700, Jeremy Huddleston Sequoia wrote: >> No, the DYLD_INSERT_LIBRARIES approach is the right one here. >> Interested users would need to disable SIP. > > "Interested users" would be everybody who uses MacPorts. I'd vote > against telling all our users to disable SIP. It's a useful > security/safety feature and it even helps us because users can no longer > mess up their /usr/bin. No. "Interested users" would be everybody who wants to use this fakeroot implementation instead of sudo. Users would need to make the tradeoff decision for themselves as to whether they want SIP+sudo or !SIP+fakeroot. > I don't see why the kernel, dyld, or whoever strips the flags could not > just behave like running a copy of the binary at hand when it sees a > DYLD variable, i.e. do the workaround we're doing manually at the > moment. Because that'e exactly what the system binary bit is intended to prevent. The system binary bit on the executables is in place to ensure that binary is not interposed by third party code that can potentially degrade the integrity of the running system. >> It would be nice if a mechanism were in place to determine trust of >> certain libraries in DYLD_INSERT_LIBRARIES. > > So you're suggesting DYLD_INSERT_LIBRARIES on SIP-protected binaries > should only work if the inserted library is signed? How would that > improve anything? Are you suggesting every open source project out there > that uses library preloading now pays for a certificate and regularly > builds and releases binaries for macOS? Frankly, I don't see that > happening. IMO, there should be a way to establish a trust model such that certain binaries could be inserted (maybe with a particular entitlement on the dylib). That isn't possible right now. Please file radars to vote for this getting addressed in a future release. OSS projects are free to do what they want for the benefit of their customers. They can sign their code with an ADC-provided cert, a cert from another CA, or not at all and just instruct users on how to sign the binary themselves. >> Please file radars and point me to them, so I can make sure they get >> routed to the right place (likely as dupes, but dupes are very useful >> "votes" for bugs). > > Those tickets have been filed when SIP was introduced and > DYLD_INSERT_LIBRARIES stopped working. > Evidently, it wasn't important > enough to get fixed, so you'll forgive me if I have better things to do > with my time. All the radars that I am aware of are mainly focused around "DYLD_INSERT_LIBRARIES doesn't get passed through system binaries" not "DYLD_INSERT_LIBRARIES doesn't get honored by system binaries". The latter is what this is about. smime.p7s Description: S/MIME cryptographic signature ___ macports-dev mailing list macports-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-dev
Re: Fakeroot destrooting [Was: Re: lldb ...]
On Saturday September 10 2016 18:38:36 Clemens Lang wrote: > I don't see why the kernel, dyld, or whoever strips the flags could not > just behave like running a copy of the binary at hand when it sees a > DYLD variable, i.e. do the workaround we're doing manually at the > moment. Then what would be the point of stripping those variables? :) > > Please file radars and point me to them, so I can make sure they get > > routed to the right place (likely as dupes, but dupes are very useful > > "votes" for bugs). > > Those tickets have been filed when SIP was introduced and > DYLD_INSERT_LIBRARIES stopped working. Evidently, it wasn't important > enough to get fixed, so you'll forgive me if I have better things to do > with my time. Erm, surely it wouldn't take a lot of time to point Jeremy to those radars if you have a list of them somewhere? R. ___ macports-dev mailing list macports-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-dev
Re: lldb ...
On Saturday September 10 2016 08:45:38 Jeremy Huddleston Sequoia wrote: > Yes, for most cases we would probably not need root access at activation > time, but for those cases you just listed, we would. We could do a pass over > the bom to determine if root access is needed or not. The UI for handling > such a case is non-trivial. Do we prompt once for the entire run of the > process or escalate permissions for each port we encounter and then drop them > again? If 8 ports are being installed and 4 require root, that's not a good > user experience to prompt for password 4 times. I think it'd be up to the port to indicate if it requires root privileges during the destroot. This is (implicitly) what happens with ports that create a new user or group; there's a phase that requires root (configure, I think). How's that handled at the UI level? I'm guessing that the phase will simply fail for the port(s) that don't have the appropriate privileges. > > I understand that the certificate catching is coupled to the file's inode > > (whatever that is under HFS+), and the new copy indeed had a new inode. And > > yet another inode after signing it. > > If you can reproduce this, please let me know and file a radar with details > and let me know the number, so I can followup internally on it. I haven't > seen such cases. The only issue I'm aware of is with overwriting an existing > file (eg: using cp instead of mv). To make this reproducible I'd probably be looking at the following sequence? 1) copy a binary from some original and ad-hoc sign it 2) delete it 3) copy the original once more and sign it with another certificate Correct me if I'm wrong, but I think that's what a `port -n upgrade --force` boils down to if signing is done in the post-activate, no? Is there any reason to expect that the certificate used in 3) must be a new one not yet used to sign other executables? > iOS was never branded as UNIX. It is UNIX-like, but it was never UNIX. AFAIK it is or used to be based on OS X, evidently without the userland but with a comparable kernel. Was that just a shortcut way of thinking of things? > I've got an internal 1TB SSD and it is more than enough for most of my needs. > I mainly use USB for thumbsticks and my camera. In such cases, the > throughput is bottlenecked on the attached device, not the bus. > > That being said, my server machine is a 2012 MacMini, and I've got a > DroboMini and a LaCie RAID attached to it via thunderbolt, and the throughput > and latency are more than adequate for my demanding needs. Thunderbolt external disks and 1TB SSDs are simply out of my budget. > For folks like you and me, it might be a minor concern being limited to 1TB, > but we can certainly use an external SSD or an SDXC to increase storage as > needed in the future. I see it as more than a minor concern. Not because I expect I'll ever need more than 1TB internal space, but I would certainly not feel comfortable when I know I cannot replace something that can fail that easily and unpredictable as an SSD (or HDD for that matter). I don't recall but it's quite likely too that I replaced the HDD in my current MBP with a Hitachi as soon as I received the machine. My only experience to date with SSDs was as an external. Connected over FW800 (the fastest bus I had at the time) showed 0 gain over a good HDD ... and the whole thing died catastrophically after not even 4 months. FWIW, that Clevo notebook I mentioned has an internal 2.5" disk (came with an SSHD) but even at its price point it has a fashionable slot to add an SSD. That's the kind of approach I like: adding expansion options instead of removing them. R. ___ macports-dev mailing list macports-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-dev
Re: Fakeroot destrooting [Was: Re: lldb ...]
Hi, On Sat, Sep 10, 2016 at 09:14:16AM -0700, Jeremy Huddleston Sequoia wrote: > No, the DYLD_INSERT_LIBRARIES approach is the right one here. > Interested users would need to disable SIP. "Interested users" would be everybody who uses MacPorts. I'd vote against telling all our users to disable SIP. It's a useful security/safety feature and it even helps us because users can no longer mess up their /usr/bin. I don't see why the kernel, dyld, or whoever strips the flags could not just behave like running a copy of the binary at hand when it sees a DYLD variable, i.e. do the workaround we're doing manually at the moment. > It would be nice if a mechanism were in place to determine trust of > certain libraries in DYLD_INSERT_LIBRARIES. So you're suggesting DYLD_INSERT_LIBRARIES on SIP-protected binaries should only work if the inserted library is signed? How would that improve anything? Are you suggesting every open source project out there that uses library preloading now pays for a certificate and regularly builds and releases binaries for macOS? Frankly, I don't see that happening. > Please file radars and point me to them, so I can make sure they get > routed to the right place (likely as dupes, but dupes are very useful > "votes" for bugs). Those tickets have been filed when SIP was introduced and DYLD_INSERT_LIBRARIES stopped working. Evidently, it wasn't important enough to get fixed, so you'll forgive me if I have better things to do with my time. -- Clemens ___ macports-dev mailing list macports-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-dev
Re: Fakeroot destrooting [Was: Re: lldb ...]
> On Sep 10, 2016, at 05:22, Clemens Lang wrote:
>
> Hi,
>
> On Fri, Sep 09, 2016 at 01:59:50PM -0700, Jeremy Huddleston Sequoia wrote:
>>> As an aside, I'd be in favour of setting up MacPorts such that
>>> ${prefix} is owned by a ${macports_operator} who's got admin rights
>>> (= myself) and reserve use of actual root privilege to those few
>>> ports that require setting up SETUID/GETUID executables or that need
>>> to create users or groups.
>>
>> YES! We should not be needing to do such things as root. That is
>> 100% true, and I am in full support of moving away from that and only
>> using root for activate. We should be able to use fakeroot
>> (https://wiki.debian.org/FakeRoot) for destdir.
>
> Except that fakeroot uses library preloading, a technique that's more or
> less dead on modern OS X thanks to Apple's changes related to SIP:
> DYLD_INSERT_LIBRARIES is stripped for all binaries with the SIP bit set.
> Combine that with every binary in /usr/bin and /bin having the bit, and
> you'll end up the variable being removed on the first invocation of a
> shell (which is basically everywhere in the build systems of our ports).
>
> It can still be done with utter hacks (copying the binary into a file
> without the SIP bit and executing it from there, which we do for trace
> mode), but I have neither seen any other library preloading utility that
> used to work on OS X implement these changes nor convinced any of their
> developers to do so.
>
> Other approaches that would allow simulating permissions, such as Linux'
> user namespaces don't exist on OS X at all. I think it's pretty obvious
> that implementing new code that relies on library preloading is riding a
> dead horse on macOS.
No, the DYLD_INSERT_LIBRARIES approach is the right one here. Interested users
would need to disable SIP. It would be nice if a mechanism were in place to
determine trust of certain libraries in DYLD_INSERT_LIBRARIES. Please file
radars and point me to them, so I can make sure they get routed to the right
place (likely as dupes, but dupes are very useful "votes" for bugs).
Thanks,
Jeremy
smime.p7s
Description: S/MIME cryptographic signature
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev
Re: lldb ...
> On Sep 10, 2016, at 05:09, Rainer Müller wrote: > > On 2016-09-09 22:59, Jeremy Huddleston Sequoia wrote: >> >>> On Sep 9, 2016, at 04:38, René J.V. Bertin wrote: >>> >>> On Friday September 09 2016 12:10:05 Rainer Müller wrote: >>> >>> > different than your case either. Either way, the debugger and all > its dependencies need to be signed by a valid certificate. That does not seem to be the case. In my testing on OS X 10.10 Yosemite, it is enough to sign /opt/local/bin/ggdb with a trusted certificate to get it working. >> >> It requires the ggdb executable and all libraries it links against to be >> signed. The port is written such that it only links against Apple-provided >> executables, so that solves that dependency. > > No? > > $ otool -L /opt/local/bin/ggdb > /opt/local/bin/ggdb: > /opt/local/lib/libintl.8.dylib (compatibility version 10.0.0, current > version 10.5.0) > /opt/local/lib/libncurses.6.dylib (compatibility version 6.0.0, current > version 6.0.0) > /opt/local/lib/libz.1.dylib (compatibility version 1.0.0, current > version 1.2.8) > /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current > version 1213.0.0) > /opt/local/lib/libiconv.2.dylib (compatibility version 8.0.0, current > version 8.1.0) > /opt/local/lib/libexpat.1.dylib (compatibility version 8.0.0, current > version 8.2.0) > > > On OS X 10.10 Yosemite, signing only the ggdb binary was certainly > enough. I cannot reproduce this on macOS 10.12 Sierra, so > the requirements might have changed. 10.10 predates SIP and related hardening around ptrace(). That version is so far in my rearview that I forget the details there, sorry. I'll have to dig into it, but it certainly seems wrong to me that a process could become privileged if it linked against unsigned libraries. > Also on Sierra it looks like I can no longer give codesign a > certificate, which is not known and trusted to the system. > > Both of these facts would destroy my idea of signing with a self-signed > certificate, but requiring the user to add trust on the certificate. > > Rainer > ___ > macports-dev mailing list > macports-dev@lists.macosforge.org > https://lists.macosforge.org/mailman/listinfo/macports-dev smime.p7s Description: S/MIME cryptographic signature ___ macports-dev mailing list macports-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-dev
Re: lldb ...
> On Sep 10, 2016, at 02:15, René J.V. Bertin wrote:
>
> On Friday September 09 2016 13:59:50 Jeremy Huddleston Sequoia wrote:
>
>>> As an aside, I'd be in favour of setting up MacPorts such that ${prefix} is
>>> owned by a ${macports_operator} who's got admin rights (= myself) and
>>> reserve use of actual root privilege to those few ports that require
>>> setting up SETUID/GETUID executables or that need to create users or groups.
>>
>> YES! We should not be needing to do such things as root. That is 100%
>> true, and I am in full support of moving away from that and only using root
>> for activate. We should be able to use fakeroot
>> (https://wiki.debian.org/FakeRoot) for destdir.
>
> Why would we even require root for activation, except for the few exceptions
> that install items outside of ${prefix} or that install SETUID/GUID to
> another user/group?
Exactly, so we'll need root access at activation time for such things.
Yes, for most cases we would probably not need root access at activation time,
but for those cases you just listed, we would. We could do a pass over the bom
to determine if root access is needed or not. The UI for handling such a case
is non-trivial. Do we prompt once for the entire run of the process or
escalate permissions for each port we encounter and then drop them again? If 8
ports are being installed and 4 require root, that's not a good user experience
to prompt for password 4 times.
> It's already an implicit requirement that the MacPorts operator (the user who
> installs it and ports) be an admin user, and once ${prefix} is created
> there's no need for it and anything below it to require root access.
>
> I've run MacPorts for years like that, and only moved away from it very
> recently because I made an error (forgot a ${destroot}) that caused pollution
> of my ${prefix}. Of course the protection I gained with that move is very
> relative, and depends on the destroot step NOT being run as root.
>
> Would fakeroot work on OS X, including on versions that predate SIP/rootless?
There's absolutely nothing preventing something like that from working
technically. All it's doing is interposing FS syscalls like stat(), chown(),
chmod(), and storing that information as a shadow. The limitation that I see
is with executables that are restricted from using DYLD_INSERT_LIBRARIES for
the interposition (ie: the same problem we have with darwintrace). We would
need to use our own versions of cp, mv, chmod, chown, stat, install, etc
command line utilities instead of the system binaries.
> Funny btw, I trust Debian to have written a safe fakeroot implementation, but
> if you read the wiki you get the impression it's a dangerous little hacking
> tool, which could be misused easily e.g. to make any executable setuid root...
Such executables still need to be installed through real root access in order
to become "really" setuid root.
>> It's quite a bit more complicated than that. First off, these settings are
>> on by default but can be configured through SIP flags, boot args, etc.
>> There >are also many types of restrictions that have different effects.
>
> Hah, TMI :)
>
>> Because of the CS_HARD restriction, all libraries that are linked against
>> require a valid code signature.
>
> Out of curiosity, if an IDE were to use a proper lldb debugger implementation
> that uses liblldb rather than an existing external driver (lldb-mi, python,
> ...), will all of the IDE have to be signed or is that still a requirement
> only for the debugserver utility?
The process doing the debugging (the one that is calling ptrace(2)) is the one
that has these restrictions, ie: debugserver.
>> This is because you likely already launched the executable, so the old
>> signature for that particular inode was already cached. If you copied
>> debugserver somewhere else and then copied it back, it would have addressed
>> the problem for you.
>
> Presumable, but that's the point, it didn't. I tried that manually, but
> shouldn't reinstalling via MacPorts have taken care of that too? Because that
> does
> - delete the previous copy
> - install a new, unsigned copy
> - sign the new copy
>
> I understand that the certificate catching is coupled to the file's inode
> (whatever that is under HFS+), and the new copy indeed had a new inode. And
> yet another inode after signing it.
If you can reproduce this, please let me know and file a radar with details and
let me know the number, so I can followup internally on it. I haven't seen
such cases. The only issue I'm aware of is with overwriting an existing file
(eg: using cp instead of mv).
>>> I'm concerned about every step that takes OS X away from a regular Unix
>>> (underneath a nice and truly integrated desktop) and towards a locked-in OS
>>> like iOS.'
>>
>> Well macOS is still UNIX. We continue to verify that through continual
>> conformance testing. I don't expect that to ever
Re: Fakeroot destrooting [Was: Re: lldb ...]
On Saturday September 10 2016 15:56:54 Clemens Lang wrote: > cat and rm deal with files, consequently trace mode needs its code to be > injected into them (as would fakeroot). Not copying them would mean > their unmodified copies run without the "sandbox" enabled, since > DYLD_INSERT_LIBRARIES wouldn't have any effect on them. Right, I didn't think this out far enough. > It's kept, but how would you do that? You cannot inject code into that > executable, and a port's build system specifies what shell scripts (for > example) get run. Not that it's going to be of much help, but shells and interpreters usually can read files that set environment variables. R. ___ macports-dev mailing list macports-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-dev
Re: Fakeroot destrooting [Was: Re: lldb ...]
Hi, On Sat, Sep 10, 2016 at 03:45:25PM +0200, René J.V. Bertin wrote: > But I see quite a few items on there that are unlikely to call other > applications. If this is only about preserving DYLD_LIBRARY_PRELOAD, > why do you need to treat utilities like cat or rm? cat and rm deal with files, consequently trace mode needs its code to be injected into them (as would fakeroot). Not copying them would mean their unmodified copies run without the "sandbox" enabled, since DYLD_INSERT_LIBRARIES wouldn't have any effect on them. > And what happens when you (re)set one of the tainted env. variables in > a shell or interpreter with the SIP bit set? Is it unset or filtered > out when you call another executable, even if that exec doesn't have > the SIP bit set? It's kept, but how would you do that? You cannot inject code into that executable, and a port's build system specifies what shell scripts (for example) get run. -- Clemens ___ macports-dev mailing list macports-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-dev
Re: Fakeroot destrooting [Was: Re: lldb ...]
On Saturday September 10 2016 15:15:54 Clemens Lang wrote: Hi, >There are no other solutions that I'm aware of on OS X. We discussed Sorry, I meant for Apple to develop something like fakeroot from the ground up. >FYI, I'm including a list of the files that are currently affected on my >system by the last 200-or-so runs of MacPorts at the end of this mail. Right ... >additional effort to support macOS for little gain. It's no secret that >macOS isn't the platform for Unix devs that it used to be. Well, what we call "bad tongues" in French could say that MacOS never was a platform for Unix development ;) >Here's the list: I'll know where to point the next person who wonders why disabling SIP will be the 1st thing I'll do after I've finally upgraded from 10.9 But I see quite a few items on there that are unlikely to call other applications. If this is only about preserving DYLD_LIBRARY_PRELOAD, why do you need to treat utilities like cat or rm? And what happens when you (re)set one of the tainted env. variables in a shell or interpreter with the SIP bit set? Is it unset or filtered out when you call another executable, even if that exec doesn't have the SIP bit set? R. ___ macports-dev mailing list macports-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-dev
Re: Fakeroot destrooting [Was: Re: lldb ...]
Hi, On Sat, Sep 10, 2016 at 02:42:36PM +0200, René J.V. Bertin wrote: > >Except that fakeroot uses library preloading, a technique that's more > >or less dead on modern OS X thanks to Apple's changes related to SIP: > >DYLD_INSERT_LIBRARIES is stripped for all binaries with the SIP bit > >set. > > Fakeroot uses library preloading on Linux, but that doesn't mean other > solutions aren't possible. Evidently that would require an official > Apple fakeroot implementation. There are no other solutions that I'm aware of on OS X. We discussed using the sandboxing mechanism, but it doesn't support our use case. Short of writing a kernel module or writing your own loader, there are no other options. > > It can still be done with utter hacks (copying the binary into a > > file without the SIP bit and executing it from there, which we do > > for trace mode), > > Do you mean something along the lines of `cat /bin/sh > /tmp/sh`? If > so, why not simply use port:bash for trace mode? Doing this only for the Shell isn't enough. Next affected tool would be make(1), then install(1), cp(1), sandbox-exec(1), gzip(1), etc. pp. Just FYI, I'm including a list of the files that are currently affected on my system by the last 200-or-so runs of MacPorts at the end of this mail. > BTW, how do you get calls like system() to use something other than > /bin/sh ? Hook execve(2) and posix_spawn(2) to transparently run a different binary. That requires parsing the shebang line, too. > > but I have neither seen any other library preloading utility that > > used to work on OS X implement these changes nor convinced any of > > their developers to do so. > > They simpy advise to disable SIP, then? They simply stopped caring. For most of them, it has always been additional effort to support macOS for little gain. It's no secret that macOS isn't the platform for Unix devs that it used to be. Here's the list: . ├── System │ └── Library │ └── Frameworks │ ├── Python.framework │ │ └── Versions │ │ ├── 2.6 │ │ │ └── Resources │ │ │ └── Python.app │ │ │ └── Contents │ │ │ └── MacOS │ │ │ └── Python │ │ └── 2.7 │ │ └── Resources │ │ └── Python.app │ │ └── Contents │ │ └── MacOS │ │ └── Python │ └── Ruby.framework │ └── Versions │ └── Current │ └── usr │ └── bin │ └── ruby ├── bin │ ├── bash │ ├── cat │ ├── chmod │ ├── cp │ ├── csh │ ├── date │ ├── dd │ ├── echo │ ├── ed │ ├── expr │ ├── hostname │ ├── ksh │ ├── launchctl │ ├── ln │ ├── ls │ ├── mkdir │ ├── mv │ ├── ps │ ├── pwd │ ├── rm │ ├── rmdir │ ├── sh │ ├── sleep │ └── test ├── sbin │ └── md5 └── usr ├── bin │ ├── Rez │ ├── SetFile │ ├── ar │ ├── arch │ ├── awk │ ├── basename │ ├── bc │ ├── bison │ ├── bzip2 │ ├── c++ │ ├── c++filt │ ├── cc │ ├── clang │ ├── clang++ │ ├── cmp │ ├── codesign │ ├── codesign_allocate │ ├── comm │ ├── cpio │ ├── cpp │ ├── ctags │ ├── curl │ ├── cut │ ├── diff │ ├── dirname │ ├── dsymutil │ ├── egrep │ ├── env │ ├── etags │ ├── expand │ ├── false │ ├── fgrep │ ├── file │ ├── find │ ├── flex │ ├── g++ │ ├── gcc │ ├── getconf │ ├── git │ ├── gm4 │ ├── gnumake │ ├── gperf │ ├── grep │ ├── groff │ ├── grotty │ ├── gzip │ ├── head │ ├── hiutil │ ├── hostinfo │ ├── ibtool │ ├── id │ ├── indent │ ├── install │ ├── install_name_tool │ ├── jar │ ├── java │ ├── javac │ ├── javadoc │ ├── join │ ├── ld │ ├── less │ ├── libtool │ ├── lipo │ ├── llvm-gcc │ ├── locale │ ├── logname │ ├── m4 │ ├── machine │ ├── make │ ├── makeinfo │ ├── man │ ├── mig │ ├── mkfifo │ ├── mktemp │ ├── nm │ ├── nmedit │ ├── od │ ├── otool │ ├── patch │ ├── perl │ ├── perl5.18 │ ├── pr │ ├── python │ ├── python2.6 │ ├── python2.7 │ ├── ranlib │ ├── readlink │ ├── rpcgen │ ├── rsync │ ├── ruby │ ├── sandbox-exec │ ├── sed │ ├── size │ ├── sort │ ├── split │ ├── stat │ ├── strings │ ├── strip │ ├── svn │ ├── svnversion │ ├── sw_vers │ ├── tail │ ├── tar │ ├── tbl │ ├── tclsh
Re: Fakeroot destrooting [Was: Re: lldb ...]
On Saturday September 10 2016 14:22:16 Clemens Lang wrote: Hi, >Except that fakeroot uses library preloading, a technique that's more or >less dead on modern OS X thanks to Apple's changes related to SIP: >DYLD_INSERT_LIBRARIES is stripped for all binaries with the SIP bit set. Fakeroot uses library preloading on Linux, but that doesn't mean other solutions aren't possible. Evidently that would require an official Apple fakeroot implementation. >It can still be done with utter hacks (copying the binary into a file >without the SIP bit and executing it from there, which we do for trace >mode), Do you mean something along the lines of `cat /bin/sh > /tmp/sh`? If so, why not simply use port:bash for trace mode? BTW, how do you get calls like system() to use something other than /bin/sh ? >but I have neither seen any other library preloading utility that >used to work on OS X implement these changes nor convinced any of their >developers to do so. They simpy advise to disable SIP, then? R. ___ macports-dev mailing list macports-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-dev
Fakeroot destrooting [Was: Re: lldb ...]
Hi,
On Fri, Sep 09, 2016 at 01:59:50PM -0700, Jeremy Huddleston Sequoia wrote:
> > As an aside, I'd be in favour of setting up MacPorts such that
> > ${prefix} is owned by a ${macports_operator} who's got admin rights
> > (= myself) and reserve use of actual root privilege to those few
> > ports that require setting up SETUID/GETUID executables or that need
> > to create users or groups.
>
> YES! We should not be needing to do such things as root. That is
> 100% true, and I am in full support of moving away from that and only
> using root for activate. We should be able to use fakeroot
> (https://wiki.debian.org/FakeRoot) for destdir.
Except that fakeroot uses library preloading, a technique that's more or
less dead on modern OS X thanks to Apple's changes related to SIP:
DYLD_INSERT_LIBRARIES is stripped for all binaries with the SIP bit set.
Combine that with every binary in /usr/bin and /bin having the bit, and
you'll end up the variable being removed on the first invocation of a
shell (which is basically everywhere in the build systems of our ports).
It can still be done with utter hacks (copying the binary into a file
without the SIP bit and executing it from there, which we do for trace
mode), but I have neither seen any other library preloading utility that
used to work on OS X implement these changes nor convinced any of their
developers to do so.
Other approaches that would allow simulating permissions, such as Linux'
user namespaces don't exist on OS X at all. I think it's pretty obvious
that implementing new code that relies on library preloading is riding a
dead horse on macOS.
--
Clemens
___
macports-dev mailing list
macports-dev@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-dev
Re: lldb ...
On 2016-09-09 22:59, Jeremy Huddleston Sequoia wrote: > >> On Sep 9, 2016, at 04:38, René J.V. Bertin wrote: >> >> On Friday September 09 2016 12:10:05 Rainer Müller wrote: >> >> different than your case either. Either way, the debugger and all its dependencies need to be signed by a valid certificate. >>> >>> That does not seem to be the case. In my testing on OS X 10.10 Yosemite, >>> it is enough to sign /opt/local/bin/ggdb with a trusted certificate to >>> get it working. > > It requires the ggdb executable and all libraries it links against to be > signed. The port is written such that it only links against Apple-provided > executables, so that solves that dependency. No? $ otool -L /opt/local/bin/ggdb /opt/local/bin/ggdb: /opt/local/lib/libintl.8.dylib (compatibility version 10.0.0, current version 10.5.0) /opt/local/lib/libncurses.6.dylib (compatibility version 6.0.0, current version 6.0.0) /opt/local/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.8) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0) /opt/local/lib/libiconv.2.dylib (compatibility version 8.0.0, current version 8.1.0) /opt/local/lib/libexpat.1.dylib (compatibility version 8.0.0, current version 8.2.0) On OS X 10.10 Yosemite, signing only the ggdb binary was certainly enough. I cannot reproduce this on macOS 10.12 Sierra, so the requirements might have changed. Also on Sierra it looks like I can no longer give codesign a certificate, which is not known and trusted to the system. Both of these facts would destroy my idea of signing with a self-signed certificate, but requiring the user to add trust on the certificate. Rainer ___ macports-dev mailing list macports-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-dev
Re: lldb ...
On 2016-09-10 00:23, Jeremy Sequoia wrote: >>> YES! We should not be needing to do such things as root. That is >>> 100% true, and I am in full support of moving away from that and only >>> using root for activate. We should be able to use fakeroot >>> (https://wiki.debian.org/FakeRoot) for destdir. +1 > It isn't that complicated. It should be fairly easy for someone to > write the moral equivalent of it under a virus-free license. I'd also > be happy to sponsor/advise anyone interested in doing it for GSoC or > self-ed. I added this to the list of ideas, in case we want to do GSoC again next year. https://trac.macports.org/wiki/SummerOfCode#fakeroot Rainer ___ macports-dev mailing list macports-dev@lists.macosforge.org https://lists.macosforge.org/mailman/listinfo/macports-dev
Re: lldb ...
On Friday September 09 2016 13:59:50 Jeremy Huddleston Sequoia wrote:
>> As an aside, I'd be in favour of setting up MacPorts such that ${prefix} is
>> owned by a ${macports_operator} who's got admin rights (= myself) and
>> reserve use of actual root privilege to those few ports that require setting
>> up SETUID/GETUID executables or that need to create users or groups.
>
>YES! We should not be needing to do such things as root. That is 100% true,
>and I am in full support of moving away from that and only using root for
>activate. We should be able to use fakeroot
>(https://wiki.debian.org/FakeRoot) for destdir.
Why would we even require root for activation, except for the few exceptions
that install items outside of ${prefix} or that install SETUID/GUID to another
user/group? It's already an implicit requirement that the MacPorts operator
(the user who installs it and ports) be an admin user, and once ${prefix} is
created there's no need for it and anything below it to require root access.
I've run MacPorts for years like that, and only moved away from it very
recently because I made an error (forgot a ${destroot}) that caused pollution
of my ${prefix}. Of course the protection I gained with that move is very
relative, and depends on the destroot step NOT being run as root.
Would fakeroot work on OS X, including on versions that predate SIP/rootless?
Funny btw, I trust Debian to have written a safe fakeroot implementation, but
if you read the wiki you get the impression it's a dangerous little hacking
tool, which could be misused easily e.g. to make any executable setuid root...
>It's quite a bit more complicated than that. First off, these settings are on
>by default but can be configured through SIP flags, boot args, etc. There
>>are also many types of restrictions that have different effects.
Hah, TMI :)
>Because of the CS_HARD restriction, all libraries that are linked against
>require a valid code signature.
Out of curiosity, if an IDE were to use a proper lldb debugger implementation
that uses liblldb rather than an existing external driver (lldb-mi, python,
...), will all of the IDE have to be signed or is that still a requirement only
for the debugserver utility?
>This is because you likely already launched the executable, so the old
>signature for that particular inode was already cached. If you copied
>debugserver somewhere else and then copied it back, it would have addressed
>the problem for you.
Presumable, but that's the point, it didn't. I tried that manually, but
shouldn't reinstalling via MacPorts have taken care of that too? Because that
does
- delete the previous copy
- install a new, unsigned copy
- sign the new copy
I understand that the certificate catching is coupled to the file's inode
(whatever that is under HFS+), and the new copy indeed had a new inode. And yet
another inode after signing it.
>> I'm concerned about every step that takes OS X away from a regular Unix
>> (underneath a nice and truly integrated desktop) and towards a locked-in OS
>> like iOS.'
>
>Well macOS is still UNIX. We continue to verify that through continual
>conformance testing. I don't expect that to ever change.
iOS used to be a Unix too. I don't know whether one can still think of it as
that, though. But anyway that wasn't my real point. For me, "regular Unix"
carries connotations and associations of an open developers' OS of choice that
date back to the late 80s.
Something "breaks" in that perception of things when we get to the point where
you have to ask (or pay) for a certificate to install and run even your own
code even if it stays away from system areas. That's exactly why I never got
into iOS development, too.
>FWIW, I really love my 2015 rMBP. I was a holdout staying on the pre-retina
>ones so I could continue to have a DVD drive and a 2.5" drive bay, but I
>finally gave that up and am really glad that I did because the newer SSDs are
>blazing fast.
Going completely OT here :)
I guess that if I had the money I might be less sceptic, but the fact is that I
simply cannot afford to dump the amount required to replace my current "mobile
workstation" mid 2011 13" MBP with something comparable from Apple (or anyone
else if I want to stick to an i7). It's got a 1Tb Hitachi HDD which is plenty
fast for me (and cost me all of 80€), evolved from 4 to 8 to 12Gb RAM as prices
dropped and lacks an expensive LCD screen that won't outlive the computer
itself. I've got a 1080p 21" external screen that is largely sufficient. When I
go mobile I either can make do with the low internal resolution, or I have
another external screen.
That large disk allowed me to make 4 partitions and keep large working
directories like Qt5's source+build+destroot around without running out of
space. I wouldn't feel comfortable at all with a non-replaceable SSD of a
comparable size knowing its cost and that rapidly evolving tech is almost by
definitio
