On 2014-08-02, at 05:04 AM, Adam Strzelecki <[email protected]> wrote:
>> Please don't exaggerate the performance implications of copying a few
>> frameworks on the first build (…)
>
> This can be done, but discussion is going off-topic.
>
> Let us try first to move Qt frameworks to use @rpath & remove some
> unnecessary operations while keeping current workflow with minimal changes to
> the code. Therefore I hereby propose to (1) keep rpath to Qt SDK in built app
> executable, (2) not copy anything on build, (3) make macdeploy modify ONLY
> executable removing absolute rpath (instead fixing frameworks), and (4) just
> copy needed frameworks and (5) optionally sign whole bundle.
>
> This will not require any wrappers, environment variables or changes in Qt
> Creator to run your app. Only Qmake and macdeploy need to be changed.
>
> Frankly I don't see anything wrong with keeping full path to SDK in
> executable when it isn't yet completely bundled. And the initial intention
> was to NOT touch Qt SDK modules once they are built.
>
> Once this is done & working we can follow this discussion and think what can
> be done next.
>
> --
> Adam
This isn't off topic at all. @rpath, copying frameworks, DYLD_ environment
variables are all inseparably linked and very much part of the same discussion
and overall issue. Your proposal to simply add @rpath and do nothing else has
no benefits. What problem does it solve, other than deleting a bit of code from
macdeployqt that currently works and will continue to work without maintenance?
None. There is no point in doing it unless we go all the way and make the Qt
SDK completely relocatable.
I understand you want to start by completing one objective at a time, but keep
in mind each of these objectives is part of a greater overall goal. Alone, they
are pointless.
>> Actually you CAN add Info.plist to bundle-less apps using the linker
>> arguments: `-sectcreate __TEXT __info_plist Info.plist` which will embed it
>> in the binary. All OS X APIs handle this transparently. Though,
>> LSEnvironment is not a good solution for the problem we are trying to solve
>> here.
>
> I was expecting that ;) But wasn't your intention to not modify binary once
> it has been built and not hardcode any paths into it?
Yes, which is why I said LSEnvironment is not a good solution.
>> Why would you do any of these things?
>
> Because you dislike adding absolute rpath to Qt SDK in built binary during
> dev process.
All of those things involve hardcoding absolute paths *somewhere*. There should
be no absolute paths *anywhere* except environment variables that are
automatically set per-session.
>> "Inject"? DYLD_LIBRARY_PATH is an environment variable. You simply set it in
>> the process's environment before spawning.
>
> It is not so simple, first of all as you shown it works only from console,
> but if you launch it via GUI you need this variable to be set in launchd. As
> ~/.launchd.conf doesn't work anymore since Lion only permanent solution is to
> use /etc/launchd-user.conf which require admin to create/modify.
>
> Secondly DYLD_LIBRARY_PATH has strong security implications as the path is
> searched BEFORE default locations. So it does in fact let you inject/replace
> libraries. This is the reason dyld disables that for any process running
> under root account.
>
> man dyld
>
> DYLD_LIBRARY_PATH
>
> The dynamic linker searches these directories BEFORE it searches the
> default locations for libraries.
>
> What might be considered there is DYLD_FALLBACK_FRAMEWORK_PATH.
This is another excuse - there are no real security implications in practice.
If you're even thinking about this, your machine has already been compromised
and you no longer own it. Remember that this is a development time concern,
too, and has nothing to do with a production environment.
The security conscious Apple Inc. doesn't seem to think it's a big problem
either, because Xcode sets it during development (see below).
I'm curious in what realistic scenario you think this is a real problem.
>
>> In a terminal session, simply:
>>
>> $ export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/path/to/Qt/Frameworks
>> $ open /apps/MyApp.app
>>
>> In Creator, just click run and this will be handled automatically. In qbs
>> just type `qbs run -p foo` and this will be handled automatically.
>
> This won't work for apps launched directly via Finder/Dock.
Like I said, that doesn't matter. It didn't work in Qt 4 and the fact that it
works in Qt 5 is a mere side effect. No one will notice.
Most people build Qt apps on OS X using some IDE, most likely Qt Creator. Are
you seriously telling me you click build, go find the build directory, and
launch the app in Finder? That's ridiculous. You click the run button. If
you're willing to go through all the former effort you can easily export
DYLD_LIBRARY_PATH at the beginning of your terminal session and type `open
MyApp.app`. Note that the open command uses launch services which is the same
as double clicking the bundle in Finder.
Furthermore, if you simply copy frameworks into the bundle at build time like
every other native OS X app on the planet, this becomes a non-issue and you
don't even need DYLD_LIBRARY_PATH. With my QMAKE_EMBEDDED_FRAMEWORKS
suggestion, too, there would be a choice between the two, and be closer to
native tools behaviour at the same time.
>> ANY solution that involves placing absolute paths in ANY file inside the
>> MyApp.app/ directory tree is a wrong solution.
>
> You are wrong there. All system libs are referenced with their absolute paths.
Those are system libraries that are part of the operating system, present on
every OS X installation in the world and not designed to be relocatable. Qt is
not a system library and should be relocatable. Anything residing in /System
has no bearing on this discussion.
> Moreover the Qt's lib/ absolute path will be there only during development,
> in deployed binary absolute path will be removed via install_name_tool
> -delete_rpath.
This still breaks the rule of absolute paths appearing somewhere. We do not
want absolute paths anywhere during development except in per-session
environment variables.
>> Note that Qt Creator already does the logical equivalent of what I described
>> on Windows; the Qt libraries path is added to the PATH when the process is
>> launched. Launching directly in Explorer doesn't work, and launching
>> directly from Finder doesn't need to work either.
>
> Currently it DOES WORK when launching via Finder, because it does hardcode
> absolute path, so I don't see any reason it should stop working. Also it does
> work like that on Linux too.
Linux is not OS X (also did you forget about $ORIGIN?). And if we're going to
compare to other operating systems, currently it DOESN'T work on Windows. Never
seen a complaint about it. And that is not a problem because in 2014 people use
IDEs and click the run button which configures search paths for you. This is
why we have tools.
>> This is closer to how the native tools do things anyways.
>
> Which tools? AFAIK Xcode doesn't copy external frameworks into app bundle by
> default.
Yes it does. It has much specific functionality and behaviours for doing this
and this technique is encouraged by Apple and used by pretty much every
Xcode-using project on the planet. And the copying happens at development time,
for both debug and release. No one postpones this process to a release-time
packaging step like macdeployqt.
Examples of how Xcode facilitates this:
(1) Embedded binaries chooser on the Info tab of a target configuration
(2) Copy phase with an explicit selection for Frameworks on the Build Phases
tab of a target configuration
(3) Default prompt to embed a framework in an application target when creating
a new framework
(4) DYLIB_INSTALL_NAME_BASE set to @rpath by default when creating a new
framework
(5) OS X: LD_RUNPATH_SEARCH_PATHS set to @executable_path/../Frameworks for
apps and [@executable_path/../Frameworks, @loader_path/Frameworks] for
frameworks when creating a new target
(6) iOS: LD_RUNPATH_SEARCH_PATHS set to @executable_path/Frameworks for apps
and [@executable_path/Frameworks, @loader_path/Frameworks] for frameworks when
creating a new target
(7)
https://developer.apple.com/library/mac/documentation/macosx/conceptual/BPFrameworks/Tasks/CreatingFrameworks.html
(some parts are outdated as DYLD_LIBRARY_PATH is now set automatically, but
the rest of the documentation pretty much instructs users to do exactly what I
am suggesting)
This is not a novel idea I just came up with. This has been the standard
workflow on OS X for over a decade.
>> Symlinks make no sense. Just copy the full frameworks, there is no
>> disadvantage to this. This is the standard workflow for development on Apple
>> platforms anyways and has been since NeXTSTEP.
>
> See above.
>
>> I'm sure someone will bring up "performance!!1" but that is a poor argument;
>
> This is going off-topic. It was about to bring @rpath to Qt frameworks not
> rework entire workflow.
The whole point of @rpath is to enable this improvement in workflow. By itself
there is no point in adding it other than *slightly* simplifying some
macdeployqt code.
>>> I don't think you should add these by default since Qt doesn't need them,
>>> so unlikely your app need them. If you use some 2rd party library you are
>>> free to extend QMAKE_RPATH list yourself.
>>
>> It might need them. Depends if we change macdeployqt to place a dylibs
>> (non-frameworks) build of Qt libraries inside Libraries instead of
>> Frameworks. Not a major part of the discussion anyways, just an idea.
>
> I don't see point of handling some custom scenarios that anyway are not what
> is expected (standard) app bundle.
Fair enough, let's focus on the task at hand.
>> Let's use standard solutions instead of strange contraptions. Standard
>> solutions involve @rpath along with DYLD_LIBRARY_PATH, copying frameworks at
>> build time, or both (preferably both).
>
> I am sorry, but you are wrong. DYLD_LIBRARY_PATH is nowhere used in Xcode and
> DYLD_ variables exist for dyld debug purposes.
Apologies for parroting, but... you are in fact wrong. I've JUST tested this
and Xcode does in fact set many environment variables when running an
application. I've added NSLog(@"%@", [[NSProcessInfo processInfo]
environment]); in my application delegate. Here's a subset of the output:
2014-08-02 17:12:48.775 Silverlock[24258:303] {
"DYLD_FRAMEWORK_PATH" =
"/Users/jakepetroules/Library/Developer/Xcode/DerivedData/Silverlock-dozbffgewlkgfucjgrjfbjopeyhq/Build/Products/DebugAppStore";
"DYLD_LIBRARY_PATH" =
"/Users/jakepetroules/Library/Developer/Xcode/DerivedData/Silverlock-dozbffgewlkgfucjgrjfbjopeyhq/Build/Products/DebugAppStore";
PATH =
"/Applications/Xcode.app/Contents/Developer/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin";
PWD =
"/Users/jakepetroules/Library/Developer/Xcode/DerivedData/Silverlock-dozbffgewlkgfucjgrjfbjopeyhq/Build/Products/DebugAppStore";
"__XCODE_BUILT_PRODUCTS_DIR_PATHS" =
"/Users/jakepetroules/Library/Developer/Xcode/DerivedData/Silverlock-dozbffgewlkgfucjgrjfbjopeyhq/Build/Products/DebugAppStore";
"__XPC_DYLD_FRAMEWORK_PATH" =
"/Users/jakepetroules/Library/Developer/Xcode/DerivedData/Silverlock-dozbffgewlkgfucjgrjfbjopeyhq/Build/Products/DebugAppStore";
"__XPC_DYLD_LIBRARY_PATH" =
"/Users/jakepetroules/Library/Developer/Xcode/DerivedData/Silverlock-dozbffgewlkgfucjgrjfbjopeyhq/Build/Products/DebugAppStore";
}
I work with Xcode on a daily basis building native Cocoa apps. I can assure you
I'm well aware of the standards and conventions in use by Apple and their
development tools. Bringing Qt in line with standard expectations for OS X and
iOS development is important to maintain and grow its competitive edge. Tools
like macdeployqt are an extremely foreign concept on OS X and iOS and are
confusing to newcomers and annoying for veterans. No other SDK that I know if
has anything of the sort, nor uses absolute sonames.
No need to mention MacPorts, Homebrew, etc., either, which probably use
absolute paths for most builds. Package management systems in general have a
whole set of conventions. For example, Fedora prohibits rpath usage in the
package management system. This is correct and justified. Installed packages in
a PMS are never relocated. Application bundles and SDKs on OS X *are*.
> Try to create simple framework and see what happens, checkout install name of
> built framework.
>
> $ otool -L
> /Users/ono/Library/Developer/Xcode/DerivedData/TestFramework-aduwyulqilsereeqgoizoxubgbdy/Build/Products/Debug/TestFramework.framework/TestFramework
>
> /Users/ono/Library/Developer/Xcode/DerivedData/TestFramework-aduwyulqilsereeqgoizoxubgbdy/Build/Products/Debug/TestFramework.framework/TestFramework:
> /Library/Frameworks/TestFramework.framework/Versions/A/TestFramework
> (compatibility version 1.0.0, current version 1.0.0)
> /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa
> (compatibility version 1.0.0, current version 20.0.0)
> /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
> (compatibility version 300.0.0, current version 1056.13.0)
> /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version
> 228.0.0)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 1197.1.1)
>
>
>
> Now create a Cocoa app and framework from the framework project as see how
> the framework is referred and check if you can find the framework inside
> bundle.
>
> $ otool -L
> /Users/ono/Library/Developer/Xcode/DerivedData/SampleApp-fwmalbohyjlyqgdhjvnicwjqqmfr/Build/Products/Debug/SampleApp.app/Contents/MacOS/SampleApp
>
> /Users/ono/Library/Developer/Xcode/DerivedData/SampleApp-fwmalbohyjlyqgdhjvnicwjqqmfr/Build/Products/Debug/SampleApp.app/Contents/MacOS/SampleApp:
> /Library/Frameworks/TestFramework.framework/Versions/A/TestFramework
> (compatibility version 1.0.0, current version 1.0.0)
> /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa
> (compatibility version 1.0.0, current version 20.0.0)
> /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
> (compatibility version 300.0.0, current version 1056.13.0)
> /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version
> 228.0.0)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 1197.1.1)
> /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
> (compatibility version 45.0.0, current version 1265.19.0)
Point being? The system default DYLIB_INSTALL_NAME_BASE for frameworks in the
Xcode build system happens to be /Library/Frameworks. For new projects it
should be set to @rpath (and the IDE does this when creating a new target).
Nothing to see here.
> --
> Adam
So, let's certainly start with adding @rpath to Qt libraries. But let's not
stop there, either.
--
Jake Petroules - jake.petroules at petroules.com
Chief Technology Officer - Petroules Corporation_______________________________________________
Development mailing list
[email protected]
http://lists.qt-project.org/mailman/listinfo/development
