Re: [PATCH] Fix PR driver/78206 by silently ignoring EPERM as well as ENOENT

[Richard Biener]

On Sun, Nov 6, 2016 at 2:36 PM, Jack Howarth  wrote:
> The use of an Apple sandbox with denied file access permissions into 
> /usr/local
> exposed that cc1 fails on errors of...
>
> cc1: error: /usr/local/include: Operation not permitted
>
> The commonly suggested solution of using --with-local-prefix= set to something
> other than /usr/local is undeirable on darwin because that creates a compiler
> which retains library searches in /usr/local/lib despite no longer searching
> for headers in /usr/local/include (which makes it suspicable to header/library
> mismatches during builds).
>
> The following trivial fix solves the issue by silently ignoring errors from
> denied permissions as well as non-existent dirs from the stat (cur->name, )
> call in remove_dup() of gcc/incpath.c. Okay for gcc trunk and backports to
> gcc-5-branch and gcc-6-branch?

I think the patch is reasonable, thus it is ok (also for backporting).

Thanks,
Richard.

>Jack Howarth

Re: [PATCH] Fix PR driver/78206 by silently ignoring EPERM as well as ENOENT

[Iain Sandoe]

> On 6 Nov 2016, at 06:57, Jack Howarth  wrote:
> 
> On Sun, Nov 6, 2016 at 9:45 AM, Iain Sandoe  wrote:
>> 
>>> On 6 Nov 2016, at 06:04, Jack Howarth  wrote:
>>> 
>>> On Sun, Nov 6, 2016 at 8:36 AM, Jack Howarth  
>>> wrote:
 The use of an Apple sandbox with denied file access permissions into 
 /usr/local
 exposed that cc1 fails on errors of...
 
 cc1: error: /usr/local/include: Operation not permitted
 
 The commonly suggested solution of using --with-local-prefix= set to 
 something
 other than /usr/local is undeirable on darwin because that creates a 
 compiler
 which retains library searches in /usr/local/lib despite no longer 
 searching
 for headers in /usr/local/include (which makes it suspicable to 
 header/library
 mismatches during builds).
 
 The following trivial fix solves the issue by silently ignoring errors from
 denied permissions as well as non-existent dirs from the stat (cur->name, 
 )
 call in remove_dup() of gcc/incpath.c. Okay for gcc trunk and backports to
 gcc-5-branch and gcc-6-branch?
  Jack Howarth
>>> 
>>>   Perhaps it would be useful if I expounded a bit on the
>>> complexities that this
>>> PR introduces on darwin. Both MacPorts and now fink leverage the Apple 
>>> sandbox
>>> to avoid contaminating their builds with development files installed
>>> in /usr/local.
>>> However the FSF gcc compiler packages built still should allow end-users to
>>> build against /usr/local as normal outside of the packaging systems. On 
>>> darwin,
>>> it has been suggested that the sandbox build issues of FSF gcc be addressed 
>>> by
>>> using ---sysroot instead. Unfortunately that approach in not viable because 
>>> the
>>> Xcode developers no longer bundle the SDK of the prior OS in Xcode.app once
>>> a new OS is released. Thus using the SDK installed at / is the only
>>> option (since
>>> building against the next OS SDK on the prior OS is unsupported on darwin),
>>> and this nullifies the ability to use --sysroot to work around this issue.
>>> 
>>> I believe the proposed patch is a trivial and straightforward solution
>>> which allows
>>> darwin developers to package the FSF gcc compilers within an Apple sandbox
>>> while retaining the ability of the built compilers to behave as expect
>>> with regard
>>> to /usr/local outside of the Apple sandbox.
>> 
>> a. The patch seems reasonable on one level (if you don’t have permission to 
>> read the directory, then there’s no point in adding it to the search path).
>> 
>> b. However, ISTM that your configuration process should be pointing the 
>> compiler to a usable sysroot [which contains only directories to which the 
>> toolchain has suitable permissions].
>> 
>> c. It is clear that the situation is complex (for Darwin), we want users to 
>> be able to use either the xcode-provided sdk as the sysroot or an 
>> installation of the ‘command line tools’ (which effectively provides a 
>> sysroot in ‘/‘).
>> 
>> d.  One day we might be able to build enough of the sysroot to support the 
>> compiler without needing this (but we’re not there yet).
>> 
>> We cannot [AFAIU, INAL etc.] take the option of packaging the sysroot 
>> ourselves until (d) is achieved.
>> 
>> ===
>> 
>> So… I’m not sure we have a good way of achieving this completely 
>> automatically, in a user-transparent manner (yet) …  however, one way of 
>> achieving this in a not-to-painful manner, is to have a toolchain layout like
>> 
>> bin/
>> lib/
>> SDKs/
>>   native => symlink to either the XCode sysroot - or ‘/‘.
>> 
>> and to configure gcc  “—with-sysroot=/SDKs/native”
> 
> One can no longer safely assume that any place other than / has the
> appropriate development SDK for open source software on darwin. As
> Jeremy Huddleston Sequoia recently described the situation to me...
> 
> "Yeah, most OSS projects assume that the buildtime headers and
> libraries match the minimum deployment target.  They decide that if
> configure determines that openat(2) is available at build time, it can
> be unconditionally used by the program.
> 
> The Apple model is that the SDK contains availability markup that
> allows developers to specify older OS versions as a deployment target.
> Newer symbols get weakly bound, and developers need to check for their
> existence before usage.
> 
> Simply picking the 10.12 SDK when the developer/user has asked for the
> system-matched SDK is going to result in problems, as we've repeatedly
> seen when users try to build against the +1 SDK with software that
> doesn't know how to handle that properly."
> 
> You really can't safely depend on the Apple Xcode.app SDK any more.
> You will be okay one day and if you update your Xcode.app, will be
> broken (i.e. building against the wrong SDK) the next day.
> 
>> 
>> The end user would then need to amend/make 

Re: [PATCH] Fix PR driver/78206 by silently ignoring EPERM as well as ENOENT

[Jack Howarth]

On Sun, Nov 6, 2016 at 9:45 AM, Iain Sandoe  wrote:
>
>> On 6 Nov 2016, at 06:04, Jack Howarth  wrote:
>>
>> On Sun, Nov 6, 2016 at 8:36 AM, Jack Howarth  
>> wrote:
>>> The use of an Apple sandbox with denied file access permissions into 
>>> /usr/local
>>> exposed that cc1 fails on errors of...
>>>
>>> cc1: error: /usr/local/include: Operation not permitted
>>>
>>> The commonly suggested solution of using --with-local-prefix= set to 
>>> something
>>> other than /usr/local is undeirable on darwin because that creates a 
>>> compiler
>>> which retains library searches in /usr/local/lib despite no longer searching
>>> for headers in /usr/local/include (which makes it suspicable to 
>>> header/library
>>> mismatches during builds).
>>>
>>> The following trivial fix solves the issue by silently ignoring errors from
>>> denied permissions as well as non-existent dirs from the stat (cur->name, 
>>> )
>>> call in remove_dup() of gcc/incpath.c. Okay for gcc trunk and backports to
>>> gcc-5-branch and gcc-6-branch?
>>>   Jack Howarth
>>
>>Perhaps it would be useful if I expounded a bit on the
>> complexities that this
>> PR introduces on darwin. Both MacPorts and now fink leverage the Apple 
>> sandbox
>> to avoid contaminating their builds with development files installed
>> in /usr/local.
>> However the FSF gcc compiler packages built still should allow end-users to
>> build against /usr/local as normal outside of the packaging systems. On 
>> darwin,
>> it has been suggested that the sandbox build issues of FSF gcc be addressed 
>> by
>> using ---sysroot instead. Unfortunately that approach in not viable because 
>> the
>> Xcode developers no longer bundle the SDK of the prior OS in Xcode.app once
>> a new OS is released. Thus using the SDK installed at / is the only
>> option (since
>> building against the next OS SDK on the prior OS is unsupported on darwin),
>> and this nullifies the ability to use --sysroot to work around this issue.
>>
>> I believe the proposed patch is a trivial and straightforward solution
>> which allows
>> darwin developers to package the FSF gcc compilers within an Apple sandbox
>> while retaining the ability of the built compilers to behave as expect
>> with regard
>> to /usr/local outside of the Apple sandbox.
>
> a. The patch seems reasonable on one level (if you don’t have permission to 
> read the directory, then there’s no point in adding it to the search path).
>
> b. However, ISTM that your configuration process should be pointing the 
> compiler to a usable sysroot [which contains only directories to which the 
> toolchain has suitable permissions].
>
> c. It is clear that the situation is complex (for Darwin), we want users to 
> be able to use either the xcode-provided sdk as the sysroot or an 
> installation of the ‘command line tools’ (which effectively provides a 
> sysroot in ‘/‘).
>
> d.  One day we might be able to build enough of the sysroot to support the 
> compiler without needing this (but we’re not there yet).
>
> We cannot [AFAIU, INAL etc.] take the option of packaging the sysroot 
> ourselves until (d) is achieved.
>
> ===
>
> So… I’m not sure we have a good way of achieving this completely 
> automatically, in a user-transparent manner (yet) …  however, one way of 
> achieving this in a not-to-painful manner, is to have a toolchain layout like
>
>  bin/
>  lib/
>  SDKs/
>native => symlink to either the XCode sysroot - or ‘/‘.
>
> and to configure gcc  “—with-sysroot=/SDKs/native”

One can no longer safely assume that any place other than / has the
appropriate development SDK for open source software on darwin. As
Jeremy Huddleston Sequoia recently described the situation to me...

"Yeah, most OSS projects assume that the buildtime headers and
libraries match the minimum deployment target.  They decide that if
configure determines that openat(2) is available at build time, it can
be unconditionally used by the program.

The Apple model is that the SDK contains availability markup that
allows developers to specify older OS versions as a deployment target.
Newer symbols get weakly bound, and developers need to check for their
existence before usage.

Simply picking the 10.12 SDK when the developer/user has asked for the
system-matched SDK is going to result in problems, as we've repeatedly
seen when users try to build against the +1 SDK with software that
doesn't know how to handle that properly."

You really can't safely depend on the Apple Xcode.app SDK any more.
You will be okay one day and if you update your Xcode.app, will be
broken (i.e. building against the wrong SDK) the next day.

>
> The end user would then need to amend/make the symlink to point to “/" or the 
> xcode SDK at installation time.
>
> Alternately, you could default it to “/“ if you believe that the end users 
> are mostly going to be using the “command line tools” installtion - or you 
> 

Re: [PATCH] Fix PR driver/78206 by silently ignoring EPERM as well as ENOENT

[Iain Sandoe]

> On 6 Nov 2016, at 06:04, Jack Howarth  wrote:
> 
> On Sun, Nov 6, 2016 at 8:36 AM, Jack Howarth  wrote:
>> The use of an Apple sandbox with denied file access permissions into 
>> /usr/local
>> exposed that cc1 fails on errors of...
>> 
>> cc1: error: /usr/local/include: Operation not permitted
>> 
>> The commonly suggested solution of using --with-local-prefix= set to 
>> something
>> other than /usr/local is undeirable on darwin because that creates a compiler
>> which retains library searches in /usr/local/lib despite no longer searching
>> for headers in /usr/local/include (which makes it suspicable to 
>> header/library
>> mismatches during builds).
>> 
>> The following trivial fix solves the issue by silently ignoring errors from
>> denied permissions as well as non-existent dirs from the stat (cur->name, 
>> )
>> call in remove_dup() of gcc/incpath.c. Okay for gcc trunk and backports to
>> gcc-5-branch and gcc-6-branch?
>>   Jack Howarth
> 
>Perhaps it would be useful if I expounded a bit on the
> complexities that this
> PR introduces on darwin. Both MacPorts and now fink leverage the Apple sandbox
> to avoid contaminating their builds with development files installed
> in /usr/local.
> However the FSF gcc compiler packages built still should allow end-users to
> build against /usr/local as normal outside of the packaging systems. On 
> darwin,
> it has been suggested that the sandbox build issues of FSF gcc be addressed by
> using ---sysroot instead. Unfortunately that approach in not viable because 
> the
> Xcode developers no longer bundle the SDK of the prior OS in Xcode.app once
> a new OS is released. Thus using the SDK installed at / is the only
> option (since
> building against the next OS SDK on the prior OS is unsupported on darwin),
> and this nullifies the ability to use --sysroot to work around this issue.
> 
> I believe the proposed patch is a trivial and straightforward solution
> which allows
> darwin developers to package the FSF gcc compilers within an Apple sandbox
> while retaining the ability of the built compilers to behave as expect
> with regard
> to /usr/local outside of the Apple sandbox.

a. The patch seems reasonable on one level (if you don’t have permission to 
read the directory, then there’s no point in adding it to the search path).

b. However, ISTM that your configuration process should be pointing the 
compiler to a usable sysroot [which contains only directories to which the 
toolchain has suitable permissions].

c. It is clear that the situation is complex (for Darwin), we want users to be 
able to use either the xcode-provided sdk as the sysroot or an installation of 
the ‘command line tools’ (which effectively provides a sysroot in ‘/‘).

d.  One day we might be able to build enough of the sysroot to support the 
compiler without needing this (but we’re not there yet).

We cannot [AFAIU, INAL etc.] take the option of packaging the sysroot ourselves 
until (d) is achieved.

===

So… I’m not sure we have a good way of achieving this completely automatically, 
in a user-transparent manner (yet) …  however, one way of achieving this in a 
not-to-painful manner, is to have a toolchain layout like

 bin/
 lib/
 SDKs/
   native => symlink to either the XCode sysroot - or ‘/‘.

and to configure gcc  “—with-sysroot=/SDKs/native”

The end user would then need to amend/make the symlink to point to “/" or the 
xcode SDK at installation time.

Alternately, you could default it to “/“ if you believe that the end users are 
mostly going to be using the “command line tools” installtion - or you could 
make your installation script discover Xcode’s SDK and point to that if you 
believe that most users will have Xcode installed,

If you simply want “/“ to be the runtime-default sysroot, then 
—with-sysroot=/ --with-build-sysroot=/path/to/build/time/sysroot
should work, and if it doesn’t we should fix the configury machinery so that it 
does.

Iain

Re: [PATCH] Fix PR driver/78206 by silently ignoring EPERM as well as ENOENT

[Jack Howarth]

On Sun, Nov 6, 2016 at 8:36 AM, Jack Howarth  wrote:
> The use of an Apple sandbox with denied file access permissions into 
> /usr/local
> exposed that cc1 fails on errors of...
>
> cc1: error: /usr/local/include: Operation not permitted
>
> The commonly suggested solution of using --with-local-prefix= set to something
> other than /usr/local is undeirable on darwin because that creates a compiler
> which retains library searches in /usr/local/lib despite no longer searching
> for headers in /usr/local/include (which makes it suspicable to header/library
> mismatches during builds).
>
> The following trivial fix solves the issue by silently ignoring errors from
> denied permissions as well as non-existent dirs from the stat (cur->name, )
> call in remove_dup() of gcc/incpath.c. Okay for gcc trunk and backports to
> gcc-5-branch and gcc-6-branch?
>Jack Howarth

Perhaps it would be useful if I expounded a bit on the
complexities that this
PR introduces on darwin. Both MacPorts and now fink leverage the Apple sandbox
to avoid contaminating their builds with development files installed
in /usr/local.
However the FSF gcc compiler packages built still should allow end-users to
build against /usr/local as normal outside of the packaging systems. On darwin,
it has been suggested that the sandbox build issues of FSF gcc be addressed by
using ---sysroot instead. Unfortunately that approach in not viable because the
Xcode developers no longer bundle the SDK of the prior OS in Xcode.app once
a new OS is released. Thus using the SDK installed at / is the only
option (since
building against the next OS SDK on the prior OS is unsupported on darwin),
and this nullifies the ability to use --sysroot to work around this issue.

I believe the proposed patch is a trivial and straightforward solution
which allows
darwin developers to package the FSF gcc compilers within an Apple sandbox
while retaining the ability of the built compilers to behave as expect
with regard
to /usr/local outside of the Apple sandbox.
 Jack

[PATCH] Fix PR driver/78206 by silently ignoring EPERM as well as ENOENT

[Jack Howarth]

The use of an Apple sandbox with denied file access permissions into /usr/local
exposed that cc1 fails on errors of...

cc1: error: /usr/local/include: Operation not permitted

The commonly suggested solution of using --with-local-prefix= set to something
other than /usr/local is undeirable on darwin because that creates a compiler
which retains library searches in /usr/local/lib despite no longer searching
for headers in /usr/local/include (which makes it suspicable to header/library
mismatches during builds).

The following trivial fix solves the issue by silently ignoring errors from
denied permissions as well as non-existent dirs from the stat (cur->name, )
call in remove_dup() of gcc/incpath.c. Okay for gcc trunk and backports to
gcc-5-branch and gcc-6-branch?
   Jack Howarth


PR78206.patch
Description: Binary data