On Nov 7, 2018, at 11:00, Ken Cunningham wrote:
> On Nov 6, 2018, at 1:54 PM, Ryan Schmidt wrote:
>
>> On Nov 6, 2018, at 14:48, Ken Cunningham wrote:
>>
>>> I noticed this recently "fixing" some ports that don't use a configure step.
>>>
>>> During the run of portconfigure.tcl, various things (sdkroot) might be
>>> tested, and the appropriate values appended to the ENV variables.
>>>
>>> But these things don't seem to come out in the configure.variables, like I
>>> would have expected them to.
>>>
>>> Is this desired behaviour?
>>>
>>> Or should portconfigure flesh out the configure.variables at the same time
>>> as it's appending to the ENV variables (like I would have expected)?
>>>
>>> (Please tell me I'm missing some well-known step that everyone else knows
>>> about but I don't :> )
>>
>> Can you give a specific example?
>>
>
>
> OK. Let’s say to you want to build lz4 against an SDK. I defined the SDK to
> 10.13. To make the Portfile show the configure environment, I had to give it
> a sham configure phase by doing this tiny change:
>
> #use_configure no
> configure.cmd /usr/bin/true
>
>
>
> When you go to build the port, sudo port -d build lz4, you see the configure
> environment variables are set up right, wth the SDK specified in several
> places, as it should be:
>
> DEBUG: Preferred compilers: clang macports-clang-5.0 macports-clang-4.0
> DEBUG: Using compiler 'Xcode Clang'
> DEBUG: Executing org.macports.configure (lz4)
> DEBUG: Environment:
> CC='/usr/bin/clang'
> CC_PRINT_OPTIONS='YES'
> CC_PRINT_OPTIONS_FILE='/opt/universal/var/macports/build/_opt_universal_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_archivers_lz4/lz4/work/.CC_PRINT_OPTIONS'
> CFLAGS='-pipe -Os
> -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk
> -arch x86_64'
> CPATH='/opt/universal/include'
> CPPFLAGS='-I/opt/universal/include
> -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk'
> CXX='/usr/bin/clang++'
> CXXFLAGS='-pipe -Os -stdlib=libc++
> -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk
> -arch x86_64'
> F77FLAGS='-m64'
> F90FLAGS='-pipe -Os -m64'
> FCFLAGS='-pipe -Os -m64'
> FFLAGS='-pipe -Os'
> INSTALL='/usr/bin/install -c'
> LDFLAGS='-L/opt/universal/lib -Wl,-headerpad_max_install_names
> -Wl,-syslibroot,/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk
> -arch x86_64'
> LIBRARY_PATH='/opt/universal/lib'
> MACOSX_DEPLOYMENT_TARGET='10.13'
> OBJC='/usr/bin/clang'
> OBJCFLAGS='-pipe -Os
> -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk
> -arch x86_64'
> OBJCXX='/usr/bin/clang++'
> OBJCXXFLAGS='-pipe -Os -stdlib=libc++
> -isysroot/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk
> -arch x86_64'
> Executing: cd
> "/opt/universal/var/macports/build/_opt_universal_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_archivers_lz4/lz4/work/lz4-1.8.3"
> && /usr/bin/true --prefix=/opt/universal
> DEBUG: system: cd
> "/opt/universal/var/macports/build/_opt_universal_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_archivers_lz4/lz4/work/lz4-1.8.3"
> && /usr/bin/true --prefix=/opt/universal
> DEBUG: Privilege de-escalation not attempted as not running as root.
> DEBUG: build phase started at Wed Nov 7 08:51:33 PST 2018
>
>
>
> However, when the port is actually being built, the build args, which should
> be setup correctly by this:
>
> build.args-append CFLAGS="${configure.cflags}
> [get_canonical_archflags cc]" \
> CXXFLAGS="${configure.cxxflags}
> [get_canonical_archflags cxx]"
>
>
> don’t actually show up on the build line with all the ${configure.cxxflags}
> as above. Instead, you just get the “default” values from the top of
> portconfigure.tcl:
>
> Executing: cd
> "/opt/universal/var/macports/build/_opt_universal_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_archivers_lz4/lz4/work/lz4-1.8.3"
> && /usr/bin/make -j16 -w all CC=/usr/bin/clang CXX=/usr/bin/clang++
> PREFIX=/opt/universal CFLAGS="-Os -arch x86_64" CXXFLAGS="-Os -stdlib=libc++
> -arch x86_64"
> DEBUG: system: cd
> "/opt/universal/var/macports/build/_opt_universal_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_archivers_lz4/lz4/work/lz4-1.8.3"
> && /usr/bin/make -j16 -w all CC=/usr/bin/clang CXX=/usr/bin/clang++
> PREFIX=/opt/universal CFLAGS="-Os -arch x86_64" CXXFLAGS="-Os -stdlib=libc++
> -arch x86_64"
>
>
>
> The lz4 portfile has manually added the -stdlib=llibc++ in the Portfile, but
> it should not need to do that as it is already being done by
> portconfigure.tcl — but the modifications made by portconfigure.tcl to the
> CXXFLAGS are not making it into ${configure.cxxflags}, at least at the point
> where the build arguments are being set up.
>
>
>
>
> I hope this is a sufficiently clean example.
Yes. And the reason why I asked for an example is that the answer would be
different for different options!
For the -stdlib=libc++ flag, portconfigure.tcl does in fact add that into
configure.cxxflags for you when needed (i.e. when the compiler is clang++) and
goes to some lengths (using traces) to ensure that it cannot be removed, except
via the approved mechanism of changing configure.cxx_stdlib. There's no need
for the lz4 port to add this manually so I've removed that code. If a port
needs the flag in configure.ldflags, then the port needs to add it there
manually, but that's not the case for lz4.
For SDK flags, you're right, they're not added to the configure.*flags
variables; instead, they're added directly to the environment by
portconfigure::configure_main when they're needed.
The reason for this probably has to do with the fact that originally the
decision about whether or not to use SDK flags was based on whether or not we
were building universal. To build universal on Tiger PowerPC, you need to use
the 10.4u SDK, otherwise you don't.
The way that the default universal variant gets added is as follows: After the
entire portfile has been processed, MacPorts base defines an empty default
universal variant if ${universal_variant} is "yes". The default value of
universal_variant is ${use_configure}, and the default value of use_configure
is "yes".
The addition of -arch flags is handled the same way, for the same reason -- we
need to add different -arch flags depending on whether we're universal or not,
and we won't know for sure if the port will have a universal variant until
after the entire portfile has been processed.
This way of defining the default universal variant certainly causes some
problems which have been pointed out before. For example, if you want to add
universal support to a port that uses "use_configure no", you have to manually
add the -arch flags using [get_canonical_archflags], but you have to make sure
the universal variant exists before then. You might think you could just
"universal_variant yes" prior to calling it, but that doesn't work because the
default universal variant won't get created until after the portfile is
processed. Instead you have to manually create an empty universal variant
("variant universal {}"). Alternately, you could restrict yourself to using
[get_canonical_archflags] in a phase (for example in a pre-build block), since
the phases run after the portfile is processed, but that makes the Portfile
code uglier and seems to make it impossible to copy values between phases, such
as assign destroot.args the value of build.args, which I've done in many
portfiles and is very convenient to be able to do.
How could we improve this? What if MacPorts defined the empty default universal
variant before processing the portfile (on systems where we can build
universal)? We would then need a way of deleting the variant in the event that
"universal_variant no" (or "use_configure no") is encountered in the portfile.
I don't think MacPorts base currently has a way to delete a variant. But maybe
that wouldn't be too hard to add.
If we could do that, then we could modify the way the -arch flags and SDK flags
are added. We could handle them similar to how configure.cxx_stdlib is handled.
That should do what you want.
But there are a *lot* of ports out there already that handle this manually.
There's a good chance that by fixing this in base, we'll end up breaking ports
or portgroups. Which could then be fixed. But we should be very careful.
I see for example that the cmake portgroup specifies the SDK root using
-DCMAKE_OSX_SYSROOT and the architectures in -DCMAKE_OSX_ARCHITECTURES. It
doesn't want them mixed in with the CXXFLAGS. We should consider that when
making changes. For example, we may want to have separate variables like
configure.archflags, configure.sdk_ldflags, configure.sdk_cflags, which we put
in configure.cflags, configure.ldflags, etc. by default but which the cmake
portgroup could clear out.
