Re: [osg-users] Huge binaries on Android.
It is a pretty nifty mechanism - it saves the huge linking times, because the OpenCV is linked dynamically, you don't need to redistribute the OpenCV binary with your app, it gets downloaded and updated automatically and you don't need to try to figure out the path where to load the libraries from (and which ones to load). -- Read this topic online here: http://forum.openscenegraph.org/viewtopic.php?p=51724#51724 ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Huge binaries on Android.
Hello Philip, On 11/22/2012 11:56 PM, Philip Lamb wrote: Yes, since we (ARToolworks) use OSG in our ARToolKit SDKs (across Windows, OS X, Linux, iOS and hopefully soon, Android :-) ), we have to anticipate that the users of our SDK will potentially run on any architecture supported by Android. Intel is making a push with Atom-based Android tablets, so x86 is likely to be more common going into 2013. There are a bunch of low-cost Android devices in China using MIPS. OK, makes sense. However, I would probably release several APKs in such case, to keep the download sizes down. On another note made in the thread about attempting to replicate OpenCV's OpenCV Manager strategy for Android: to those developing the Android port of OSG, and on behalf of the users, don't do this. It makes for bad user experience. The user just wants to install and run your app, and then has to deal with what is this asking me to install NOW? along with the delay, logging in and fiddling with the Android market. We use OpenCV and we have gone to some lengths to statically link OpenCV to as to avoid having to put our users through this. It's basically trying to solve a problem which doesn't exist -- users really don't care that much about a couple of extra MB in size of your app. It also potentially breaks your app if the user installs new binaries including changes which are not completely backwards compatible. Well, I beg to differ here. The extra download is a very common thing with many games in the Google Market - you download/buy a game and then it asks for an extra 100MB+ download of the game data. So that is hardly an unprecedented feature, moreover it is required by the Market policies that limit the maximum app size, I believe. Users certainly *do care* about the few extra megs of download - especially those on metered data plans with no wifi in range. Just read the comments under some applications whenever size of the app changes. So any way to cut that down is good - especially on the multi-architecture APKs that contain binaries for all supported architectures (in the worst case 4x larger download ...) Also the storage size on many devices is quite limited and the trend is to do away with SD Card slots on newer devices, letting the users rely only on internal storage. So if you link your binary statically and the user downloads two, three of your applications, they get all the libraries duplicated. That is what the Manager tried to address, because Android prevents one app from using shared libraries installed by another one directly. Regarding the potential breakage when OpenCV is updated - that is not supposed to happen, because your app declares which version of OpenCV it needs. The manager then finds and loads an ABI compatible one. At least that is how it is intended to work - I haven't had a breakage so far, despite several updates, but that is no proof, of course. Finally, linking of OpenCV outside the manager is deprecated and unsupported now, so if something in your app breaks, you are pretty much on your own. Regards, Jan ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Huge binaries on Android.
I think Matthias is totally right. Android NDK toolchain doesn't use strip when generating static libraries (it's not safe for all the cases). Usually when you are compiling the final dylib that is loaded through Java the toolchan uses stripping to reduze the size. but i think that it is only used IF you are using the ndk-scripts. So maybe if you have set your own compile configuration you didn't include that step. More info about stripping here: http://www.technovelty.org/linux/stripping-shared-libraries.html But the simple version is that stripping a dylib is safe. 2012/11/22 Matthias Sattler osgfo...@tevs.eu Did you try to strip your generated binary? The command is just: Code: strip programfile in the commandline. If you're building dynamic you might strip the generated libs too. I'm not sure if you can still dynamically link against stripped dynamic libs though. So keep the original lib around. But it is worth a try. -- Read this topic online here: http://forum.openscenegraph.org/viewtopic.php?p=51200#51200 ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Huge binaries on Android.
Thanks everyone for the help. I have identified two things. The first was a blunder on my part; I was looking at my linked dylib in obj/local, which includes all the debug symbols. The ndk-build 'Install' step actually strips the shared lib before placing it in lib/, and this was a much more reasonable size. The second thing I did was add: LOCAL_CPPFLAGS += -fvisibility-hidden to my Android.mk and that shrinks things a bit further too. End result is total of 40MB total for all 4 architectures. Regards, Philip Lamb On 22/11/2012, at 9:12 PM, Jorge Izquierdo Ciges jori...@gmail.com wrote: I think Matthias is totally right. Android NDK toolchain doesn't use strip when generating static libraries (it's not safe for all the cases). Usually when you are compiling the final dylib that is loaded through Java the toolchan uses stripping to reduze the size. but i think that it is only used IF you are using the ndk-scripts. So maybe if you have set your own compile configuration you didn't include that step. More info about stripping here: http://www.technovelty.org/linux/stripping-shared-libraries.html But the simple version is that stripping a dylib is safe. ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Huge binaries on Android.
Hello, On Thu, Nov 22, 2012 at 9:12 AM, Jorge Izquierdo Ciges jori...@gmail.comwrote: I think Matthias is totally right. Android NDK toolchain doesn't use strip when generating static libraries (it's not safe for all the cases). Usually when you are compiling the final dylib that is loaded through Java the toolchan uses stripping to reduze the size. but i think that it is only used IF you are using the ndk-scripts. So maybe if you have set your own compile configuration you didn't include that step. That's not true, actually. The binaries loaded to the device are *always* stripped, no matter what you do. Even when you are compiling for debugging. The reason for that is that you do not need the symbols when debugging - the debugger is running on the host (where an unstripped binary is available), not on the device. I have discovered this when trying to debug native code - I was trying to get the non-stripped binary on the device and had to hack the ndk-build to achieve that, before realizing that it isn't actually useful because of the remote debugging being done. You can check that the binaries are always stripped in the ndk-build script and also by comparing the size of the generated libraries under libs and obj/local. The obj folder has the non-stripped ones for debugging, the ones from libs are deployed in the APK. The striping done by ndk-build should be safe, it strips only not needed symbols. The huge size of the libraries is not really a big deal apart from the terrible linking times - the linker removes all non-needed parts (unused functions) from the resulting binary, so I have about 1MB .so while I am linking in 200MB+ of OSG libraries. Regards, Jan ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Huge binaries on Android.
Hello, On Thu, Nov 22, 2012 at 10:48 AM, Philip Lamb p...@eden.net.nz wrote: The second thing I did was add: LOCAL_CPPFLAGS += -fvisibility-hidden Careful about this - visibility could break things if the shared libraries are not properly exporting symbols. In Linux it is quite common to build shared libs out of static ones by simple relinking, without explicitly exporting, relying on the implicit export of every symbol. This is exactly the opposite behaviour than in Windows where the default is to export nothing from the DLLs. If you enable visibility, the non-exported symbols will not be accessible anymore (and the linker likely removes them) and the application will likely crash when it loads such shared library. I think OSG is safe from this point of view due to its extensive macros dealing with imports/exports, but if you are using a library that doesn't have a DLL version on Windows (something Linux or Android-specific), you could get burned easily. to my Android.mk and that shrinks things a bit further too. End result is total of 40MB total for all 4 architectures. Do you actually need to compile for MIPS and x86? x86 Android devices are very very rare and I have never heard about a MIPS one (maybe some TVs?). The armeabi and armeabi-v7a are the most common ones, in fact, if you want to cut down even more (and don't mind a bit of performance loss), armeabi binaries will run on everything. Regards, Jan ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Huge binaries on Android.
Jan - if you have set your own compile configuration you didn't include that step. Which means if you bypass the ndk-build scripts and use directly the gcc you, yourself. :P 2012/11/22 Jan Ciger jan.ci...@gmail.com e are *always* stripped, no matter what you do. Even when you are compiling for debugging. The reason for that is that you do not need the symbols when debugging - the debugger is running on the host (where an unstripped binary is available), not on the device. I have discovered this when trying to debug native code - I was trying to get the non-stripped binary on the device and had to hack the ndk-build to achieve that, before realizing that it isn't actually useful because of the remote debugging being done. ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Huge binaries on Android.
On Thu, Nov 22, 2012 at 12:07 PM, Jorge Izquierdo Ciges jori...@gmail.comwrote: Jan - if you have set your own compile configuration you didn't include that step. Which means if you bypass the ndk-build scripts and use directly the gcc you, yourself. :P Well, he talked about Android.mk, that makefile is used by ndk-build. He never mentioned doing the compilation from scratch. Jan ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Huge binaries on Android.
On 22/11/2012, at 11:44 PM, Jan Ciger jan.ci...@gmail.com wrote: Do you actually need to compile for MIPS and x86? x86 Android devices are very very rare and I have never heard about a MIPS one (maybe some TVs?). The armeabi and armeabi-v7a are the most common ones, in fact, if you want to cut down even more (and don't mind a bit of performance loss), armeabi binaries will run on everything. Yes, since we (ARToolworks) use OSG in our ARToolKit SDKs (across Windows, OS X, Linux, iOS and hopefully soon, Android :-) ), we have to anticipate that the users of our SDK will potentially run on any architecture supported by Android. Intel is making a push with Atom-based Android tablets, so x86 is likely to be more common going into 2013. There are a bunch of low-cost Android devices in China using MIPS. On another note made in the thread about attempting to replicate OpenCV's OpenCV Manager strategy for Android: to those developing the Android port of OSG, and on behalf of the users, don't do this. It makes for bad user experience. The user just wants to install and run your app, and then has to deal with what is this asking me to install NOW? along with the delay, logging in and fiddling with the Android market. We use OpenCV and we have gone to some lengths to statically link OpenCV to as to avoid having to put our users through this. It's basically trying to solve a problem which doesn't exist -- users really don't care that much about a couple of extra MB in size of your app. It also potentially breaks your app if the user installs new binaries including changes which are not completely backwards compatible. Regards, Phil. ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Huge binaries on Android.
Truth is that it is more or less that way. Still your dylib seems a little bit huge to me, my usual examples and code with OSG stay in a 12-15-17 mark depending on plugins and 3rd party... 170 i think is too much... The static with 1Gb that's unavoidable and the reason why my pc is suffering a lot handling my compilations. Still check your building process of your dylib, maybe you are including several more libraries or something like that. 2012/11/21 Philip Lamb phil.openscenegr...@eden.net.nz de, the libraries are weighing in at over 1GB per architecture. Once linked into a dylib, the finished product is still over 170MB per architecture.By way of comparison, the same type of build on iOS weighs in at around 40MB total for 2 architectures. ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Huge binaries on Android.
Hello, Well, one thing to do would be to change from static linking to linking OSG dynamically. The OpenCV project does this using their OpenCV Manager app. The main Activity in Android calls an asynchronous init that connects with the manager. That manager downloads (if required) the proper version of OpenCV for your device from the market and loads the shared library(ies) for you. Then a callback in the app is called and you don't need to bother with loading the .so anymore. It is a pretty nifty mechanism - it saves the huge linking times, because the OpenCV is linked dynamically, you don't need to redistribute the OpenCV binary with your app, it gets downloaded and updated automatically and you don't need to try to figure out the path where to load the libraries from (and which ones to load). I think this type of setup could be extended for OSG too. The description of the system is here: http://docs.opencv.org/android/service/doc/index.html?highlight=manager Jan On Wed, Nov 21, 2012 at 9:35 AM, Jorge Izquierdo Ciges jori...@gmail.comwrote: Truth is that it is more or less that way. Still your dylib seems a little bit huge to me, my usual examples and code with OSG stay in a 12-15-17 mark depending on plugins and 3rd party... 170 i think is too much... The static with 1Gb that's unavoidable and the reason why my pc is suffering a lot handling my compilations. Still check your building process of your dylib, maybe you are including several more libraries or something like that. 2012/11/21 Philip Lamb phil.openscenegr...@eden.net.nz de, the libraries are weighing in at over 1GB per architecture. Once linked into a dylib, the finished product is still over 170MB per architecture.By way of comparison, the same type of build on iOS weighs in at around 40MB total for 2 architectures. ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
Re: [osg-users] Huge binaries on Android.
Did you try to strip your generated binary? The command is just: Code: strip programfile in the commandline. If you're building dynamic you might strip the generated libs too. I'm not sure if you can still dynamically link against stripped dynamic libs though. So keep the original lib around. But it is worth a try. -- Read this topic online here: http://forum.openscenegraph.org/viewtopic.php?p=51200#51200 ___ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
