Le 16.08.17 à 16:27, Eric Wing a écrit :
I've been using a derivative of the iOS toolchain for many years that
you probably can find easily with a Google search. It has a lot of
shortcomings, but it generally works. And most of the shortcomings I
think are only solvable by properly fixing/modifying the CMake core.
Hi,
thanks for your answer. I also found several examples online, some of
them are good, but I cannot just copy-paste them :) I need to understand
what is going on, and sometimes things are done not in a good way.
On 8/15/17, Raffi Enficiaud <raffi.enfici...@mines-paris.org> wrote:
Le 10.08.17 à 17:04, Brad King a écrit :
On 08/08/2017 08:08 AM, Raffi Enficiaud wrote:
I have looked a bit to the Android toolchains, and I have to say I found
those quite complicated as a first reading :)
I personally think the Android toolchain is way more complicated than
the iOS toolchain. Among the reasons are that every NDK release broke
something different as they kept changing the compiler and conventions
(the gcc to clang move was the most recent biggie, but old-timers
might remember the standalone toolchain difficulties.). Then you have
to pick different API levels because the each NDK release ships a
separate API subtarget for all prior versions of Android. Then add all
the multiple architectures (mips, arm, x86, 64-bit) and the
subvariants (armv5, armv7, armv7+NEON, etc), 4 different C++ standard
libraries you have to choose from, and other nuisances like Android on
Windows...makes the whole thing a mess.
Right now, I am completely discarding whatever has been done for Android :)
Ideally CMake would gain iOS platform modules such that one could
set CMAKE_SYSTEM_NAME to `iOS`.
where this path is hard coded, and points to the fat static libraries
prefix path of boost. If I remove this path, FindBoost does not find the
boost libraries anymore (of course I am passing BOOST_ROOT). In
addition, I have this:
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
These last three lines tell the find commands to only look at
paths re-rooted under CMAKE_FIND_ROOT_PATH and CMAKE_SYSROOT.
If boost is not under one of those then it won't be found.
That sounds right. The general problem is you don't want to
accidentally pick up OSX stuff on your system.
It appears that this is too restrictive. For instance, I compiled boost
and made fat libraries in an installation folder, and I am not being
able to pick them up. By removing them, FindBoost works fine.
But the problem is that it is unclear to me what should be allowed and
what not, and where are the problems:
* specifically for the FindBoost problem mentioned above, is it a
limitation of FindBoost that is not honouring cross compilation well? I
believe that if I provide a BOOST_ROOT, then it should be used no matter
what default configuration is provided by the toolchain file
* another example: I need eg. the python interpreter to run some tests.
If I remove the lines above, the host interpreter is found. In the case
of my project, it is ok, because I use that for running some tests. In
some other projects I have, I think this is not ok because I may use the
interpreter for finding other libraries on the OS (cython, numpy, etc).
The question is:
what is the best practice for letting the developer do his job right?
Preventing accessing some paths when searching for a binary or a
shared/library is limiting. Especially, it is not easy to know if a
library that is found is part of the cross-compilation toolchain that
can run on the host (say "codesign", "clang", "python" etc) or part of
the bundle we want to create (eg boost_something.dylib, that is a fat
binary containing also the architecture of the host - because of the iOS
simulator).
As of today, find_library or find_program does not make any distinction,
but in case of cross-compilation, I would like to have ideally a
"CROSS_COMPILATION_TOOLCHAIN" property that I may use in the CMake scripts.
For instance, I need to find a library that will be integrated in the
target platform binary:
--------
find_library(MyLIB
NAMES my_lib_arch_arm
TARGET_PLATFORM) # or TARGET_PLATFORM implicit
add_executable(MyFinalBundle ... TARGET_PLATFORM)
target_link_library(MyFinalBundle MyLIB) # check for consistency: all
dependencies should be TARGET_PLATFORM as well
--------
In this case, the find_library will look into the sysroot that is
provided by the toolchain file, plus some given by the user and that are
specific to the module (BOOST_ROOT for instance)
OTOH:
--------
find_library(MyToolchainLib
NAMES my_toolchainlib
HOST_PLATFORM)
find_program(my_other_toolchain_program
HOST_PLATFORM)
add_executable(my_intermediade_tool HOST_PLATFORM)
target_link_library(my_intermediade_tool MyToolchainLib)
add_custom_command(
OUTPUT output output2
COMMAND my_intermediade_tool input output
COMMAND my_other_toolchain_program input2 output2)
--------
Those binaries are compiled on the host with the host architecture, or
looked for inside the platform default toolchain (eg codesign), and are
then being part of the overall toolchain that is being used.
Tools like qtautomoc for instance can be part of this, and we know often
for sure if a program is part of the build process or the target.
I've generally used a combination of two things. I modified my
toolchain to have special new root paths which I can define and I put
all my iOS stuff there, and/or I use the -C initial cache feature of
CMake to prespecify the locations. (And sometimes my initial caches
use my new root path variables so the initial cache is flexible and
not hard coded.)
That is interesting, do you have an example of the -C thinggy?
set(CMAKE_MACOSX_BUNDLE YES)
Is it possible to build any binary of any form on iOS without this?
You're right, I do not think this is possible.
As far as I know, it always must be a bundle.
As I understand it, this is a problem of try_compile: as signing of application
is required for generating a binary for iOS, this one fails very early when
CMake discovers the capabilities of the compiler.
I have not encountered this. I don't think it is true, but I rarely
use try_compile.
The examples I have found on the Internet are forcing the compiler
checks. Example:
---------
include (CMakeForceCompiler)
set(CMAKE_C_COMPILER clang Clang)
set(CMAKE_CXX_COMPILER clang++ Clang)
set(CMAKE_AR ar CACHE FILEPATH "" FORCE)
# Skip the platform compiler checks for cross compiling
set (CMAKE_CXX_COMPILER_WORKS TRUE)
set (CMAKE_C_COMPILER_WORKS TRUE)
---------
I would like to avoid this, this is why I want to delegate the compiler
checks to CMake, as it does it well. The problem I had is that CMake
emits several try_compile, and one of those apparently is trying to
build a bundle, which causes an error because of the final required signing.
Building iOS static libraries are not signed. And I don't think
simulator targets require signing either.
Right now I am not being able to install on the simulator precisely
because of the signing ... Maybe I am doing it wrong
Signing is required for running on device.
That said, there are a ton of other problems related to signing which
are completely broken with CMake.
Right now, the biggest pain point for me is the new Automatic Code
signing. Prior to Xcode 8, you could set a CMake Xcode attribute to
explicitly set the key to use. But this now conflicts with the new
Automatic Code signing, and this his cannot be controlled by a CMake
attribute and it breaks everything. And even if that were fixed, Xcode
needs a second property, the Team. While this can be set by an
attribute, the problem is just about every user has a different team
and it is not easy to know. Xcode has a way of finding out what teams
are available so you can pick it in a drop-down list (same with the
keys). But for a CMake user, this is nearly impossible to know
apriori. I think the only way to fix this is to modify CMake to 1) try
to defer to Xcode's built-in behavior & 2) if the user sets this, some
how don't let CMake overwrite/clobber just that setting when the
project regenerates on an update.
Semi-related is the Mac OS X entitlements system which is connected to
code signing and shipping on the Mac App Store. It has a lot of these
same problems and I think it needs to be fixed in the same fashion.
The entitlements are supposed to show up in a dedicated tab in the
project settings, but CMake projects tend to be blank in this area.
And related to that is the iOS device orientation and launch screen
setting which is the same story.
Signing is also an issue for me:
- right now I disabled anything automatic done by Xcode. It just does it
wrong all the time
- I have the private key and the associated certificate in my keychain
- I select the right signing certificate from Xcode detailed options
- I indicate the team ID manually
- I indicate the provisioning file manually
and then signing is part of the build that happens automatically by
XCode as a last post-build step. For preparing the bundle, I created a
post-build script to copy all the relevant dylibs in the final bundle
and correct for the @rpath, and it seems to work fine. The signing
happens after, on all dylibs and frameworks that are contained in the
bundle.
I need to set CODE_SIGNING_REQUIRED attribute to YES though manually as
well, as it is inherited by the
CMAKE_XCODE_ATTRIBUTE_CODE_SIGNING_REQUIRED of the toolchain.
XCode needs to access the private key that will be used for signing
anyway. I will not be happy with distributing a private key on my build
machines, so I am fine with having the signing thinggy manual.
Currently the main issue I am seeing is the multiarch/multisysroot
target of XCode that is kind of lost when using CMake. By
multiarch/multisysroot, I mean that Xcode is able to switch from
iPhoneSimulatorXY to iPhoneXY without changing the project, and within
the same view.
Yes, that is a huge problem. I've found this to be very fragile.
However for the moment, it kind of works.
FIND_LIBRARY(APPLE_UIKIT_LIBRARY "UIKit")
I think this was something I did to my toolchain, but I honestly can't
remember the specifics. It might have had something to do with
SYSROOT. I think the default one out there was setting something too
aggressively. By removing it, it allowed Xcode to do it's normal
things when switching between simulator and device targets (i.e.
switch the SDK paths), and it still worked.
I am not sure to understand what problem the
FIND_LIBRARY(APPLE_UIKIT_LIBRARY "UIKit") addresses. Are you saying
that, by just adding that, Xcode would be able to eg. find libz in
different sysroots?
I am not sure to understand.
However, I think the real solution is again to fix CMake so libraries
& frameworks use te exact same built-in mechanism Xcode uses. That way
Xcode takes control of everything in this situation because it looks
no different than a hand-crafted Xcode project in this case.
I also believe that we should take this path:
- different sysroots, as we handle different configuration
(debug/release). I believe however that this is not semantically the
same as a build configuration, and OTOH increasing exponentially the
number of variables is not a good idea in the end.
- something that indicates that we are targeting the host or target
platform (see above) for better cross-compilation support
Additionally, bundling (copying resources) and code signing are
another huge problem. Xcode formally recognizes that libraries and
frameworks must be copied into the bundle during the build process.
There is an explicit settings tab for this now. And it also knows that
it must codesign everything after this step. And all this is an atomic
operation as part of the regular build. When you hit the
build/run/profile buttons in Xcode, all this stuff happens in one step
and also includes the launch in that step. CMake has been resistant to
this workflow for years, and it is a huge problem because this how the
entire process is designed to operate and not supporting this breaks
stuff badly. So again, if we can modify CMake to fully utilize the
built-in Xcode features for libraries & frameworks, I think this
problem gets fixed.
Though there is one more nasty problem with the simulator vs. device
switching. For 3rd party (dynamic) frameworks which were just finally
allowed a few years ago, Apple has failed to give developers a good
solution for dealing with pre-built libraries. Unlike a static
library, Apple doesn't strip the dynamic libraries you don't use in
your bundled framework. So if you built a fat framework with both
simulator and device binareis, you get rejected by the App Store. But
if you don't build a fat framework, there is no Apple supported way to
switch between simulator and device targets on the fly.
I haven't been in a state to push to app-store yet, and I am pretty sure
a lot of new problem will arise there. From what I understood from
"-fembed-bitcode" option of clang, this should let Apple to do the work
of preparing the binary to a specific device.
So in summary, my feeling is that to fix this, these are the top items
that must be addressed:
1) A way to turn off Automatic Code signing or a way to fully embrace
it so it just works
(may need a way to not clobber manual user settings for key selection
on regenerate)
2) Fix manual code signing (similar to above)
3) Make CMake fully embrace native Xocde mechanisms for libraries &
frameworks (which includes the atomic build/bundle/codesign/run)
4a) Make it easier to build fat binaries and then strip out the unused
ones on the final app build
or
4b) We need CMake to recognize a system where when the user switches
between simulator and device targets, it can also switch 3rd-party SDK
directories which contain separate targets.
5) Fix proper integration for entitlements and device settings
The more I explore this, the more things are piling up :) and I believe
the roadmap for better handling is quite huge.
What I would be tempted to suggest is, if we can come up with a
toolchain file that is general enough and properly document all the
current shortcomings or the best practices, then it will gather
attention and bring more people and resources into the discussion.
This would be my first step
* having different build trees for different sysroots is a workaround
that should normally work
* maybe promoting apple frameworks instead of "find_library" commands is
the way to go for developing for iOS
* fixing the small bugs there and there from the proposed toolchain will
take time (eg. the try_compile thing that requires signing: I believe
this should be a try_compile that understand that it is not meant for
the final project but for an intermediate check, and signing in that
case should not be required)
* being able to distinguish program/libraries that are part of the build
and the others that are part of the toolchain is currently not possible,
without creating a dedicated toolchain file (maybe already in the
documentation)
etc etc.
* Extra: I haven't dealt with AppleTV and Apple Watch yet. I don't
know what extra things may be needed besides the LLVM IR "BitCode"
option.
-Eric
P.S. I'm reaching a point where the codesiging is becoming a big
enough problem for me that I'm interested in fixing it. And I might be
able to dedicate some time to it. I saw some other people are already
ahead of me on this and have looked at it in more detail. I'm willing
to help out or collaborate.
Please try what I suggested above. If that works for you, maybe we do
not need to deal with that but rather have the right options in Xcode
and have a consistent way for preparing the bundles.
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers
