Re: Make clang accept and use relative filenames for tools

[Joerg Sonnenberger]

On Mon, May 29, 2017 at 06:54:52PM +0100, Stuart Henderson wrote:
> On 2017/05/29 20:26, Vadim Zhukov wrote:
> > The clang and gcc behave differently regarding executing tools.
> > While gcc simply runs what he said to, clang tries to be clever
> > and always find absolute path for a tool, refusing start otherwise.
> > 
> > The actual problem is starting a linker: ports infrastructure
> > expects tools are called by name, not by path, and thus could be
> > overriden via stuff in ${WRKDIR}/bin. This functionality is used,
> > e.g., to implement USE_WXNEEDED port option.
> > 
> > But clang calls "/usr/bin/ld", not "ld", and thus ${WRKDIR}/bin/ld
> > misses a chance to do its magic, and binaries are built without
> > OPENBSD_WXNEEDED, and some ports blow up (when compiled using clang).
> 
> One thing we _could_ do is pass -fuse-ld=${WRKDIR}/bin/ld when linking ..

Just -B ${WRKDIR}/bin should be enough and cover other tools like as as
needed.

Joerg

Re: Make clang accept and use relative filenames for tools

[Vadim Zhukov]

2017-05-29 21:14 GMT+03:00 Mark Kettenis :
>> From: Vadim Zhukov 
>> Date: Mon, 29 May 2017 20:29:20 +0300
>>
>> 2017-05-29 20:26 GMT+03:00 Vadim Zhukov :
>> > The clang and gcc behave differently regarding executing tools.
>> > While gcc simply runs what he said to, clang tries to be clever
>> > and always find absolute path for a tool, refusing start otherwise.
>> >
>> > The actual problem is starting a linker: ports infrastructure
>> > expects tools are called by name, not by path, and thus could be
>> > overriden via stuff in ${WRKDIR}/bin. This functionality is used,
>> > e.g., to implement USE_WXNEEDED port option.
>> >
>> > But clang calls "/usr/bin/ld", not "ld", and thus ${WRKDIR}/bin/ld
>> > misses a chance to do its magic, and binaries are built without
>> > OPENBSD_WXNEEDED, and some ports blow up (when compiled using clang).
>> >
>> > The patch below is enough to make clang accept and use non-absolute
>> > tool file names and make lang/mono and lang/libv8 happy (more likely
>> > to follow, there is much work to be done).
>> >
>> > So I'm asking compiler gurus and base maintainers, if a patch like
>> > that would be acceptable? And if yes, how do you want it look like
>> > to make supporting it easier?
>> >
>> > Thank you in advance.
>>
>> P.S.: The Program.inc file is not picked up as dependency, so you'll
>> need to touch gnu/llvm/lib/Support/Program.cpp manually before
>> building clang.
>
> I'm not in favour of this approach.  I'm pretty sure it'll break the
> cross-compilation toolchain.  And it makes us deviate from how clang
> behaves on other platforms quite a bit.

OK, understood, thank you.

--
  WBR,
  Vadim Zhukov

Re: Make clang accept and use relative filenames for tools

[Vadim Zhukov]

2017-05-29 20:54 GMT+03:00 Stuart Henderson :
> On 2017/05/29 20:26, Vadim Zhukov wrote:
>> The clang and gcc behave differently regarding executing tools.
>> While gcc simply runs what he said to, clang tries to be clever
>> and always find absolute path for a tool, refusing start otherwise.
>>
>> The actual problem is starting a linker: ports infrastructure
>> expects tools are called by name, not by path, and thus could be
>> overriden via stuff in ${WRKDIR}/bin. This functionality is used,
>> e.g., to implement USE_WXNEEDED port option.
>>
>> But clang calls "/usr/bin/ld", not "ld", and thus ${WRKDIR}/bin/ld
>> misses a chance to do its magic, and binaries are built without
>> OPENBSD_WXNEEDED, and some ports blow up (when compiled using clang).
>
> One thing we _could_ do is pass -fuse-ld=${WRKDIR}/bin/ld when linking ..

This requires: 1) linker not being called directly; 2) LDFLAGS being
passed. I'm not sure if all 50 (or so) USE_WXNEEDED ports behave like
this. The reason we have USE_WXNEEDED implemented the current way was
to catch such situations, IIUC

 Yes, we could tweak ${WRKDIR}/bin/{cc,cxx} to force always passing
-fuse-ld=${WRKDIR}/bin/ld. That'll require combining gcc4.port.mk and
clang.port.mk, though; otherwise they'll become unmaintaineable,
because they already have to respect USE_CCACHE. Maybe we'll have a
${PORTSDIR}/mk/compiler.mk that's called from modules.port.mk and
implements all the logic we want from different compilers... Until
that the dirty hacks in mono, libv8 and whatever else will stay.

--
  WBR,
  Vadim Zhukov

Re: Make clang accept and use relative filenames for tools

[Mark Kettenis]

> From: Vadim Zhukov 
> Date: Mon, 29 May 2017 20:29:20 +0300
> 
> 2017-05-29 20:26 GMT+03:00 Vadim Zhukov :
> > The clang and gcc behave differently regarding executing tools.
> > While gcc simply runs what he said to, clang tries to be clever
> > and always find absolute path for a tool, refusing start otherwise.
> >
> > The actual problem is starting a linker: ports infrastructure
> > expects tools are called by name, not by path, and thus could be
> > overriden via stuff in ${WRKDIR}/bin. This functionality is used,
> > e.g., to implement USE_WXNEEDED port option.
> >
> > But clang calls "/usr/bin/ld", not "ld", and thus ${WRKDIR}/bin/ld
> > misses a chance to do its magic, and binaries are built without
> > OPENBSD_WXNEEDED, and some ports blow up (when compiled using clang).
> >
> > The patch below is enough to make clang accept and use non-absolute
> > tool file names and make lang/mono and lang/libv8 happy (more likely
> > to follow, there is much work to be done).
> >
> > So I'm asking compiler gurus and base maintainers, if a patch like
> > that would be acceptable? And if yes, how do you want it look like
> > to make supporting it easier?
> >
> > Thank you in advance.
> 
> P.S.: The Program.inc file is not picked up as dependency, so you'll
> need to touch gnu/llvm/lib/Support/Program.cpp manually before
> building clang.

I'm not in favour of this approach.  I'm pretty sure it'll break the
cross-compilation toolchain.  And it makes us deviate from how clang
behaves on other platforms quite a bit.

Re: Make clang accept and use relative filenames for tools

[Stuart Henderson]

On 2017/05/29 20:26, Vadim Zhukov wrote:
> The clang and gcc behave differently regarding executing tools.
> While gcc simply runs what he said to, clang tries to be clever
> and always find absolute path for a tool, refusing start otherwise.
> 
> The actual problem is starting a linker: ports infrastructure
> expects tools are called by name, not by path, and thus could be
> overriden via stuff in ${WRKDIR}/bin. This functionality is used,
> e.g., to implement USE_WXNEEDED port option.
> 
> But clang calls "/usr/bin/ld", not "ld", and thus ${WRKDIR}/bin/ld
> misses a chance to do its magic, and binaries are built without
> OPENBSD_WXNEEDED, and some ports blow up (when compiled using clang).

One thing we _could_ do is pass -fuse-ld=${WRKDIR}/bin/ld when linking ..

Re: Make clang accept and use relative filenames for tools

[Vadim Zhukov]

2017-05-29 20:26 GMT+03:00 Vadim Zhukov :
> The clang and gcc behave differently regarding executing tools.
> While gcc simply runs what he said to, clang tries to be clever
> and always find absolute path for a tool, refusing start otherwise.
>
> The actual problem is starting a linker: ports infrastructure
> expects tools are called by name, not by path, and thus could be
> overriden via stuff in ${WRKDIR}/bin. This functionality is used,
> e.g., to implement USE_WXNEEDED port option.
>
> But clang calls "/usr/bin/ld", not "ld", and thus ${WRKDIR}/bin/ld
> misses a chance to do its magic, and binaries are built without
> OPENBSD_WXNEEDED, and some ports blow up (when compiled using clang).
>
> The patch below is enough to make clang accept and use non-absolute
> tool file names and make lang/mono and lang/libv8 happy (more likely
> to follow, there is much work to be done).
>
> So I'm asking compiler gurus and base maintainers, if a patch like
> that would be acceptable? And if yes, how do you want it look like
> to make supporting it easier?
>
> Thank you in advance.

P.S.: The Program.inc file is not picked up as dependency, so you'll
need to touch gnu/llvm/lib/Support/Program.cpp manually before
building clang.

--
  WBR,
  Vadim Zhukov

Make clang accept and use relative filenames for tools

[Vadim Zhukov]

The clang and gcc behave differently regarding executing tools.
While gcc simply runs what he said to, clang tries to be clever
and always find absolute path for a tool, refusing start otherwise.

The actual problem is starting a linker: ports infrastructure
expects tools are called by name, not by path, and thus could be
overriden via stuff in ${WRKDIR}/bin. This functionality is used,
e.g., to implement USE_WXNEEDED port option.

But clang calls "/usr/bin/ld", not "ld", and thus ${WRKDIR}/bin/ld
misses a chance to do its magic, and binaries are built without
OPENBSD_WXNEEDED, and some ports blow up (when compiled using clang).

The patch below is enough to make clang accept and use non-absolute
tool file names and make lang/mono and lang/libv8 happy (more likely
to follow, there is much work to be done).

So I'm asking compiler gurus and base maintainers, if a patch like
that would be acceptable? And if yes, how do you want it look like
to make supporting it easier?

Thank you in advance.

--
WBR,
  Vadim Zhukov


Index: tools/clang/lib/Driver/ToolChain.cpp
===
RCS file: /cvs/src/gnu/llvm/tools/clang/lib/Driver/ToolChain.cpp,v
retrieving revision 1.1.1.3
diff -u -p -r1.1.1.3 ToolChain.cpp
--- tools/clang/lib/Driver/ToolChain.cpp24 Jan 2017 08:33:12 -  
1.1.1.3
+++ tools/clang/lib/Driver/ToolChain.cpp29 May 2017 17:17:19 -
@@ -357,17 +357,17 @@ std::string ToolChain::GetLinkerPath() c
   if (llvm::sys::path::is_absolute(UseLinker)) {
 // If we're passed what looks like an absolute path, don't attempt to
 // second-guess that.
-if (llvm::sys::fs::exists(UseLinker))
+//FOOif (llvm::sys::fs::exists(UseLinker))
   return UseLinker;
   } else if (UseLinker.empty() || UseLinker == "ld") {
 // If we're passed -fuse-ld= with no argument, or with the argument ld,
 // then use whatever the default system linker is.
-return GetProgramPath(getDefaultLinker());
+return /*FOO GetProgramPath(*/ getDefaultLinker() /*)*/;
   } else {
 llvm::SmallString<8> LinkerName("ld.");
 LinkerName.append(UseLinker);
 
-std::string LinkerPath(GetProgramPath(LinkerName.c_str()));
+std::string LinkerPath(/*FOO GetProgramPath(*/ LinkerName.c_str() /*)*/ );
 if (llvm::sys::fs::exists(LinkerPath))
   return LinkerPath;
   }
@@ -375,7 +375,7 @@ std::string ToolChain::GetLinkerPath() c
   if (A)
 getDriver().Diag(diag::err_drv_invalid_linker_name) << 
A->getAsString(Args);
 
-  return GetProgramPath(getDefaultLinker());
+  return /*FOO GetProgramPath(*/ getDefaultLinker() /*)*/;
 }
 
 types::ID ToolChain::LookupTypeForExtension(StringRef Ext) const {
Index: lib/Support/Unix/Program.inc
===
RCS file: /cvs/src/gnu/llvm/lib/Support/Unix/Program.inc,v
retrieving revision 1.1.1.1
diff -u -p -r1.1.1.1 Program.inc
--- lib/Support/Unix/Program.inc3 Sep 2016 22:47:02 -   1.1.1.1
+++ lib/Support/Unix/Program.inc29 May 2017 17:17:19 -
@@ -181,12 +181,14 @@ static void SetMemoryLimits (unsigned si
 static bool Execute(ProcessInfo , StringRef Program, const char **args,
 const char **envp, const StringRef **redirects,
 unsigned memoryLimit, std::string *ErrMsg) {
+/* let the OS decides if the executable exists
   if (!llvm::sys::fs::exists(Program)) {
 if (ErrMsg)
   *ErrMsg = std::string("Executable \"") + Program.str() +
 std::string("\" doesn't exist!");
 return false;
   }
+*/
 
   // If this OS has posix_spawn and there is no memory limit being implied, use
   // posix_spawn.  It is more efficient than fork/exec.
@@ -240,7 +242,7 @@ static bool Execute(ProcessInfo , Str
 // Explicitly initialized to prevent what appears to be a valgrind false
 // positive.
 pid_t PID = 0;
-int Err = posix_spawn(, Program.str().c_str(), FileActions,
+int Err = posix_spawnp(, Program.str().c_str(), FileActions,
   /*attrp*/nullptr, const_cast(args),
   const_cast(envp));