Re: Considering /opt and MyDocs in your packages
Hi, It would be helpful if you could provide instructions on how to install apps directly into /opt without changing too much of the packaging stuff (remember, some package maintainers have to keep the Diablo version and maybe even a Linux Desktop version up to date as well, so anything that avoids big differences in packaging/code helps here). This depends on how you have setup your build-system. With GNU autotools, running configure.sh with --prefix=/opt/packagename would be a way, but you have to take care that the application icon, .desktop file and D-Bus service activation description still get into the rootfs. If you're using a simple Makefile instead if autotools then you have full control already. Relocating Python apps to /opt should be pretty simple. I do so by installing into /opt/packagename and putting a symlink to the executable module into /usr/bin/ (although this is not really required if the path in the D-Bus service activation description already points to /opt). If I need to find out about the installation path of my application, I read the __file__ variable which in every Python module points to the file path of that module file, so I know where to find my graphics and other data files, no matter where my application got installed. For C applications you may experience other problems such as not finding .so files. A way around this could be replacing the executable in /usr/bin with a shell script setting up the LD_LIBRARY_PATH variable for your app correctly. XDG_DATA_DIR can be specified by environment vatiables, too, IIRC. I think a subsection on this wiki page would be ideal: http://wiki.maemo.org/Opt_Problem I fully agree. Developers should be made aware that maemo-optify is not always the best way to move stuff to /opt. If I find some time this evening I'm going to make a start on the wiki page. There are some more caveats I stumpled upon, e.g. putting application graphics other than the app icon into /usr/share/icons results in a huge icon cache file. These issues should all be covered on the wiki page IMHO. Adding maemo-optify as build-depends and to debian/rules is just simpler and more straightforward than doing more intrusive changes. Yes, nothing against it. But developers should be made aware of the problems that may arise and that alternatives do exist. For porting stuff with hardcoded paths optify might still be the only feasible way, especially for developers without deep Linux experience. Martin ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Hi, ext Graham Cobb wrote: On Thursday 10 September 2009 12:16:59 Marius Vollmer wrote: By the way, I have been experimenting with maemo-optify. I think it is currently generating too many links for quite small files. All files, even symlinks, take some space. On UBIFS single file overhead is about 1/4 KB (inodes + filename). I think 20K would be a better default for the size and, if feasible, I would like to see the size settable as an option on the command line, to allow the developer to tune it for their particular package. As UBIFS compresses the file contents[1] with LZO, it's the lzop compressed file sizes which should be used for this kind of decision. Does maemo-optify compare the lzop'ed or non-compressed ones file sizes? - Eero [1] /opt is on ext3 which isn't compressed. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Hi, in my opinion, maemo-optify shouldn't be hailed as the good-for-everyone savior to the rootfs space issue. I see it as a workaround for software that cannot be easily relocated to /opt. Optification symlinks take space on the rootfs too, so if you have a lot of them, you still waste several megabytes. Optification breaks package upgrading. File opening operations take longer as the system has to resolve a symlink first. Please don't get me wrong. optify is still useful and good to have, but everybody should bear in mind that it's only a workaround, and that relocating the software to /opt would be a much safer and more elegant solution. IMHO optify is OK for porting software from upstream, but if you write specifically for the N900, you should rather consider making your software easily relocatable eventually. Cheers, Martin ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Wed, Oct 21, 2009 at 21:43, Martin Grimme martin.gri...@gmail.com wrote: in my opinion, maemo-optify shouldn't be hailed as the good-for-everyone savior to the rootfs space issue. I see it as a workaround for software that cannot be easily relocated to /opt. Agreed. Software which is written specifically (or, primarily) for Maemo can be designed to either be relocatable, or installed into /opt/package. This separates maemo-optify stuff (/opt/maemo/path) from the more specific /opt/package. See, for example: * Bounce * Classic Gaming Wallpapers * Hermes * mClock (uses /opt/vendor/package, which is also allowed under FHS) Please don't get me wrong. optify is still useful and good to have, but everybody should bear in mind that it's only a workaround, and that relocating the software to /opt would be a much safer and more elegant solution. IMHO optify is OK for porting software from upstream, but if you write specifically for the N900, you should rather consider making your software easily relocatable eventually. Hear hear! Cheers, Andrew -- Andrew Flegg -- mailto:and...@bleb.org | http://www.bleb.org/ ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
2009/10/21 Martin Grimme martin.gri...@gmail.com: Please don't get me wrong. optify is still useful and good to have, but everybody should bear in mind that it's only a workaround, and that relocating the software to /opt would be a much safer and more elegant solution. It would be helpful if you could provide instructions on how to install apps directly into /opt without changing too much of the packaging stuff (remember, some package maintainers have to keep the Diablo version and maybe even a Linux Desktop version up to date as well, so anything that avoids big differences in packaging/code helps here). I think a subsection on this wiki page would be ideal: http://wiki.maemo.org/Opt_Problem IMHO optify is OK for porting software from upstream, but if you write specifically for the N900, you should rather consider making your software easily relocatable eventually. Adding maemo-optify as build-depends and to debian/rules is just simpler and more straightforward than doing more intrusive changes. I still hope that we get rid of maemo-optify at some point in the future, as it's a dirty dirty hack. In the mean time, Marius is doing a great job of fixing remaining bugs in the optify tool, even if it means adding even more dirty hacks (symlink/directory fixing in postinst ;). Thomas ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Hi, Am Mittwoch 16 September 2009 schrieb Marius Vollmer: The top ten entries are: enigma 19 16276 I just /opt-imized enigma Till ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Opt Bof? (was: Considering /opt and MyDocs in your packages)
Vollmer Marius (Nokia-D/Helsinki) marius.voll...@nokia.com writes: The **really** interesting question is going to be whether someone can come up with a solution which Nokia can (and will agree to) apply in a Fremantle update! Indeed! Hmm, what about a Opt BoF at the Maemo Summit? We could get together to talk about what we should do near term, and maybe also about the long term. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Opt Bof? (was: Considering /opt and MyDocs in your packages)
On Mon, Sep 21, 2009 at 16:54, Marius Vollmer marius.voll...@nokia.com wrote: Hmm, what about a Opt BoF at the Maemo Summit? We could get together to talk about what we should do near term, and maybe also about the long term. Sounds good. I've added an optify facility to MUD, so that if you specify optify=1 in the XML file, it'll stick maemo-optify into debian/rules in the appropriate place. Interestingly, my first with it seemed to move the stuff into /opt correctly, but didn't put any symlinks in. Perhaps because it was trying to do it on a directory which already existed from a previous version? Cheers, Andrew -- Andrew Flegg -- mailto:and...@bleb.org | http://www.bleb.org/ Maemo Community Council chair ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Graham Cobb g+...@cobb.uk.net writes: So, given all the controversy and discussion, why don't we start with the simple hack. Yep. How shall we start? I noticed that the vor package is already optified. Nice! I just ran maemo-optify over all extras-devel packages. The result is here: deb http://zagadka.vm.bytemark.co.uk/optify/opts-2k/ ./ Please test your package if you are curious. (But please be gentle with that little server.) Some data is here: http://zagadka.vm.bytemark.co.uk/optify/opts-2k/DATA.filtered It has the format: packagen-links kilobytes-saved The top ten entries are: mysql-server-5.0 753 60770 libicu40 8 16469 enigma 19 16276 mysql-client-5.032 15756 libqt4-webkit2 15508 pokerth 7 12548 glom54 10569 glom-sqlite 62 10565 libqt4-gui 7 10039 cmake79173 The 700+ symlinks in mysql-server are certainly excessive and need manual intervention, mysql-client, glom and glom-sqlite could need some tuning, but the rest looks reasonable, I'd say. The **really** interesting question is going to be whether someone can come up with a solution which Nokia can (and will agree to) apply in a Fremantle update! Indeed! ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /Opt And MyDocs In Your Packages
Hello, Sorry for the broken threading. I just subscribed and the quote is from copy- paste from nabble. ext Andrew Flegg andrew at bleb.org writes: Although a unionfs solution would be a bit more further dev on Nokia's part, it will reduce the developer complexity and gives us a real world solution now. I'm sure the community would help as well, with patching/building/testing kernel modules (once again, Nokia should realise there are clever technical people in the community who could help design an optimal (= quick good) solution if engaged at the right time). Yes, agreed. Let's give this unionfs thing a shot. I was previously arguing against it, but only as a long-term solution. As a Fremantle-only hack, it might be better than the /opt hack. After reading the whole thread I could offer you the following suggestions/thoughts: 1) /opt is not a good solution (tm). You can install large programs there (e.g. openoffice) but you should not install programs that other programs depends to there (e.g. libraries or command-line utils). This means that (according to what I understand from FHS) a program should not depend on another program in /opt. 2) You should consider making /usr/local a separate filesystem and suggest that programs use that instead of /opt. A system can work with most of its programs in /usr/local and you don't need of links, etc. I had a slackware- based system which had almost everything in /usr/local for 8+ years and there weren't any problems at all. 3) If finally you use /opt/maemo, it may be better to suggest some hashing. For example /opt/maemo/foo could be in /opt/maemo/f/foo. 4) unionfs should be a very good and clean solution. I strongly suggest that you don't use UnionFS and you use AUFS. I had a lot of problems and oopses with the former. 5) If you use aufs, it will be a bit tricky to upgrade packages that are installed in / in a consistent way. For example, apt-get upgrade could end up installing things in the other partition. To solve this you should either do a chroot somewhere else (not-good) or use some voodoo magic :-) 6) If you somehow use aufs, it is possible to speed things up a bit. For example, you could: * Create a union of the small (old) / and a X GB partition which is then mounted as / * Mount the small (old) / as /oldroot as read-only. * Prepend /oldroot/bin:/oldroot/sbin:/oldroot/usr/bin:/oldroot/usr/sbin in $PATH * Perhaps use LD_LIBRARY_PATH with /oldroot/lib:/oldroot/usr/lib first. This would make many of the things load directly from /oldroot while be able to fallback to the unionfs (which should not be that slow). Of course /usr/share won't be used directly, but I believe you can live with that. 7) I can report a success story from 5 computer labs with 120 PCs and more than 300 different students using aufs every day for two years now without a problem and AFAIK without a single kernel panic or oops. It is a little different configuration (union of a physical FS and a tmpfs) but it is a good test for AUFS. 8) You could reconsider (or explain) the need and usage of the internal memory (should I call it JFFS2 NAND?) completely. What is it's purpose? Clearly, it is not fit for the / filesystem if N900 is going to be used in a standard unix-way with non-nokia programs and upgrades. 9) If you need this memory to just be able to provide re-flash updates/upgrades, then I cannot find a reasonable way to make this work without problems. Upgrading a library in /usr/lib (for example) could break programs. apt and dpkg are far better in handling this. However, this could always act as a fall-back solution (fail-safe), in case something breaks with the other memory. 10) If you want to use this memory for speed-up, then I believe there are a number of ways to do it. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, 2009-09-10 at 17:20 +0200, Vollmer Marius (Nokia-D/Helsinki) wrote: ext Andrew Flegg and...@bleb.org writes: Although a unionfs solution would be a bit more further dev on Nokia's part, it will reduce the developer complexity and gives us a real world solution now. I'm sure the community would help as well, with patching/building/testing kernel modules (once again, Nokia should realise there are clever technical people in the community who could help design an optimal (= quick good) solution if engaged at the right time). Yes, agreed. Let's give this unionfs thing a shot. I was previously arguing against it, but only as a long-term solution. As a Fremantle-only hack, it might be better than the /opt hack. Does UnionFS support block rotation (like ubifs) for the internal flash? -Kimmo ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Kimmo Hämäläinen wrote: Does UnionFS support block rotation (like ubifs) for the internal flash? UnionFS works on top of other file system(s) in directory level. In this case, UBIFS would be still there for the underlying root filesystem. It seems that the overhead of using UnionFS is about 10% [1], which should be noted when making decisions. This performance lost will affect all files, not should optified files as in the original plan. BR, Henrik [1] http://www.linuxjournal.com/article/7714 -- Henrik Hedberg - http://www.henrikhedberg.net/ ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Fri, 2009-09-11 at 09:15 +0200, ext Henrik Hedberg wrote: Kimmo Hämäläinen wrote: Does UnionFS support block rotation (like ubifs) for the internal flash? UnionFS works on top of other file system(s) in directory level. In this case, UBIFS would be still there for the underlying root filesystem. It seems that the overhead of using UnionFS is about 10% [1], which should be noted when making decisions. This performance lost will affect all files, not should optified files as in the original plan. Maybe it could be used selectively for installed applications only. I think the 10-12% speed loss is not going to be accepted for all file system access (doesn't make sense either if it can be avoided). -Kimmo BR, Henrik [1] http://www.linuxjournal.com/article/7714 ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
David Weinehall wrote: IMHO we have three options: - Real VFAT (with all the drawbacks it brings) - VVFAT - A separate program (PC Suite, most likely) to do the transfers (probably leaving Linux and MacOS users out in the cold) There is also MTP [1]. It is less generic than one would want but still it tries to solve this problem. Too bad there is no generic USB standardized filesystem gadget. 1. http://en.wikipedia.org/wiki/Media_Transfer_Protocol ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Andrew Flegg wrote: ...because /opt is a hack because no-one at Nokia had the foresight to imagine that users might want to install multiple applications, and large new frameworks like Qt. It wasn't really hard to see this coming. We are booting from MMC since the OS2006 days and one of reasons users do it is that the space in internal NAND is too small. 3 years later someone noticed this problem and we have this ugly /opt hack done in last minutes. Oh well ;-) ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Andrew Flegg and...@bleb.org writes: On Thu, Sep 10, 2009 at 16:12, Marius Vollmer marius.voll...@nokia.com wrote: ext Andrew Flegg and...@bleb.org writes: Instead of using a fixed prefix of /opt/maemo/path, use /opt/package/trimmed path. [big snip] I'm not going to get into a point-by-point rebuttal of these. Hmm, I am really in a detail-oriented let's get this done mode. As I said, the question of whether or not and if so how to use /opt for distribution packages is bigger than this, and I don't think we should try to answer it here. People haven't really used /opt in their packages until now, and nothing really has changed to reconsider this on a massive scale. The name /opt that appears in this discussions is a big giant red herring. Just pretent it really was /space all the time, and I hope you can see how different the discussion would have been. But installing stuff in /opt on Maemo by third-parties isn't really going to happen. Maybe not, but that's no reason to repurpose the whole of /opt. You could also say that Maemo will never really be multi-user, but that's no reason to get rid of /home/user. We own the space, pretty much everything is going to be installed from packages, and we already make all manner of assumptions in a Linux system that there's some unique UNIX name for a package. Why *not* make the one-line change to maemo-optify to make its results slightly cleaner? Because it's not cleaner in my view, it would be an even bigger abuse of /opt than hiding things in /opt/maemo. If you really care, we can make this an option to maemo-optify, and you can use it in your packages. I will still recommend the default of just moving everything to /opt/maemo without any further distortion. - Computing the trimmed path from path is an extra complication, and we must make sure that no collisions happen. It's doable of course, but in the light of the arguments above, why bother? ...because /opt is a hack because no-one at Nokia had the foresight to imagine that users might want to install multiple applications, and large new frameworks like Qt. The /opt is hack statement needs to be qualified, of course. Moving stuff into /opt/maemo is a hack, of course. But at least in my opinion, Moving stuff into /opt/package/ is a bigger hack, and a bigger violation of the letter and spirit of /opt. *shrug* ...because /opt is a hack which should be *embarassing*. It is! ...because maemo-optify creating a forest of symlinks is messy, unelegant and possibly prone to failure (see my earlier question about Python modules and sub-directories of optified packages). If I understand your proposal here right, we would still have the same forest of symlinks, just more messy since they have a more complicated pattern. Mainly, though, because last minute fixes shouldn't throw good design out of the window. I don't think using /opt/package in distribution packages is good design. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Andrew Flegg and...@bleb.org writes: On Thu, Sep 10, 2009 at 16:20, Marius Vollmer marius.voll...@nokia.com wrote: ext Andrew Flegg and...@bleb.org writes: Although a unionfs solution would be a bit more further dev on Nokia's part, it will reduce the developer complexity and gives us a real world solution now. [...] Yes, agreed. Let's give this unionfs thing a shot. I was previously arguing against it, but only as a long-term solution. As a Fremantle-only hack, it might be better than the /opt hack. Cool. I should also point out that when I say a unionfs solution, I mean a union FS solution; rather than advocating unionfs over aufs (for example). Thanks to Stskeeps/Carsten for picking me up on that. Noted, thanks! ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Graham Cobb g+...@cobb.uk.net writes: On Thursday 10 September 2009 12:16:59 Marius Vollmer wrote: Also, you can make it so that maemo-optify only runs in debian/rules when it is present: which maemo-optify maemo-optify Small correction: that doesn't work (because it returns an error status when maemo-optify is not present). Right, sorry for hastily posting untested code. (I should know better not to do that.) Is the idea that the autobuilder will automatically install maemo-optify if it exists for that version or do we have to add it to our Build-Depends? Good question. I haven't really thought this through, obviously... :-/ Right now I would say that the buildbot should automatically install it, but maybe that is a bit brittle. What about uploading a version of maemo-optify to all buildbots, and only the one in Fremantle does any actual optification? Or we could do something more general ala the type-handling package in Debian. We could have a package that Provides virtual packages like fremantle, not+fremantle, diablo, not+diablo as appropriate for the distribution it is in, and then packages can have Build-Depends like Build-Depends: maemo-optify | not+fremantle Might be overkill, dunno. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Marius Vollmer (marius.voll...@nokia.com) wrote: The /opt is hack statement needs to be qualified, of course. Moving stuff into /opt/maemo is a hack, of course. But at least in my opinion, Moving stuff into /opt/package/ is a bigger hack, and a bigger violation of the letter and spirit of /opt. *shrug* Uh, I am not sure about this. Reading the Filesystem Hierarchy Standard (http://www.pathname.com/fhs/) it seems that both uses (/opt/maemo/* and /opt/package) are covered by it. In fact configuring applications with --prefix=/opt and installing to /opt is not. (for /opt/maemo one apparently should register maemo with http://www.lanana.org/ which is apparently not really a problem: nokia and nokiasiemensnetworks are already registered) I don't think using /opt/package in distribution packages is good design. True. Bye, Simon -- si...@budig.de http://simon.budig.de/ simon.bu...@kernelconcepts.de http://www.kernelconcepts.de/ ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Simon Budig si...@budig.de writes: Marius Vollmer (marius.voll...@nokia.com) wrote: The /opt is hack statement needs to be qualified, of course. Moving stuff into /opt/maemo is a hack, of course. But at least in my opinion, Moving stuff into /opt/package/ is a bigger hack, and a bigger violation of the letter and spirit of /opt. *shrug* Uh, I am not sure about this. Reading the Filesystem Hierarchy Standard (http://www.pathname.com/fhs/) it seems that both uses (/opt/maemo/* and /opt/package) are covered by it. Yes, but package is not in the same namespace as the distribution packages. As you point out, one should register the package names with Lanana. (I didn't know about this, thanks for the information.) I feel confident that we can get away with maemo without registering it. Using /opt/nokia feels wrong, since it gives the impressions that everything under /opt/nokia is actually provided by Nokia, which isn't true, and strictly speaking we would have to use /opt/nokia/maemo anyway since /opt/nokia is for all of Nokia, not just for our little hack here. Anyway, let's not discuss this to death. Patches to maemo-optify are welcome as long as they don't change the current default behavior of putting things in /opt/maemo. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Fri, Sep 11, 2009 at 08:15, Henrik Hedberg henrik.hedb...@innologies.fi wrote: It seems that the overhead of using UnionFS is about 10% [1], which should be noted when making decisions. This performance lost will affect all files, not should optified files as in the original plan. The article in question *is* five years old, and I believe the preferred union FS solution wouldn't be UnionFS, it'd possibly be aufs or one of the others explored by Valerie. Cheers, Andrew -- Andrew Flegg -- mailto:and...@bleb.org | http://www.bleb.org/ Maemo Community Council chair ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Fri, Sep 11, 2009 at 11:11 AM, Marius Vollmer marius.voll...@nokia.com wrote: ext Graham Cobb g+...@cobb.uk.net writes: On Thursday 10 September 2009 12:16:59 Marius Vollmer wrote: Also, you can make it so that maemo-optify only runs in debian/rules when it is present: which maemo-optify maemo-optify Small correction: that doesn't work (because it returns an error status when maemo-optify is not present). About the maemo-optify usage. The opkg package manager support a offline root mode that allows you to install packages using a different base. The idea is that one might want to install some content on a removable media. it doesn't require you to change the packages. perhaps this is a less intrusive option? Greetings ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Kees Jongenburger kees.jongenbur...@gmail.com writes: The opkg package manager support a offline root mode that allows you to install packages using a different base. Like dpkg's --root option? The idea is that one might want to install some content on a removable media. it doesn't require you to change the packages. perhaps this is a less intrusive option? Nah, come on now. None of the packages will work if you install them somewhere else than in /. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Hi, I don't think /opt/package is a bad idea, and I wouldn't call it a hack. On other Unices, like Solaris, 3rd party software usually goes to /opt. In fact, by default, /usr is write-protected in virtual containers (mounted read-only from the host system). The good thing IMHO about /opt is that all stuff resides in one place, not being scattered all across the filesystem. When looking at Solaris, you can find directories like lib and bin inside /opt/package/. This way it's also possible for programs to bring in libraries that would otherwise break or mess up the root system. LD_LIBRARY_PATH, etc. can be setup by the start scripts of the programs accordingly. I'm all for keeping the root system clean of 3rd party stuff. Regards, Martin 2009/9/11, Kees Jongenburger kees.jongenbur...@gmail.com: On Fri, Sep 11, 2009 at 11:11 AM, Marius Vollmer marius.voll...@nokia.com wrote: ext Graham Cobb g+...@cobb.uk.net writes: On Thursday 10 September 2009 12:16:59 Marius Vollmer wrote: Also, you can make it so that maemo-optify only runs in debian/rules when it is present: which maemo-optify maemo-optify Small correction: that doesn't work (because it returns an error status when maemo-optify is not present). About the maemo-optify usage. The opkg package manager support a offline root mode that allows you to install packages using a different base. The idea is that one might want to install some content on a removable media. it doesn't require you to change the packages. perhaps this is a less intrusive option? Greetings ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Martin Grimme martin.gri...@gmail.com writes: I don't think /opt/package is a bad idea, and I wouldn't call it a hack. On other Unices, like Solaris, 3rd party software usually goes to /opt. In fact, by default, /usr is write-protected in virtual containers (mounted read-only from the host system). I like to see Maemo as a traditional, all-inclusive distribution. As such, the stuff in Maemo Extras is as 1st party as any other package. (True, some package are more equal than others, such as the OS meta-packages, and some repositories are also more equal than others, such as the System Software Updates repo, but these things are not really Us vs You.) The good thing IMHO about /opt is that all stuff resides in one place, not being scattered all across the filesystem. That's just on the surface. If you have a reasonable package management system, it doesn't make any difference either way. I do agree that the traditional Unix filesystem layout is not very clean, but it is also not broken and I don't want to try to fix anything about it within this excersize of finding more space for applications. When looking at Solaris, you can find directories like lib and bin inside /opt/package/. This way it's also possible for programs to bring in libraries that would otherwise break or mess up the root system. They can do this without /opt as well, of course. LD_LIBRARY_PATH, etc. can be setup by the start scripts of the programs accordingly. I'm all for keeping the root system clean of 3rd party stuff. Knock yourself out, I am not stopping you. :-) But I also don't want to make this optification any more complex than it needs to be, for the sake of something non-trivial and fuzzy as cleaning up the Unix filesystem layout. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Fri, Sep 11, 2009 at 10:32, Marius Vollmer marius.voll...@nokia.com wrote: Yes, but package is not in the same namespace as the distribution packages. As you point out, one should register the package names with Lanana. (I didn't know about this, thanks for the information.) No, according to the FHS provider needs to be registered with LANANA, not individual packages (and so only if you use /opt/provider/package). Cheers, Andrew -- Andrew Flegg -- mailto:and...@bleb.org | http://www.bleb.org/ Maemo Community Council chair ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Friday 11 September 2009 09:56:14 Marius Vollmer wrote: Hmm, I am really in a detail-oriented let's get this done mode. As I said, the question of whether or not and if so how to use /opt for distribution packages is bigger than this, and I don't think we should try to answer it here. So, given all the controversy and discussion, why don't we start with the simple hack. Let's decide that for the short term, any package (or group of packages forming an application) which is over (say) 500K SHOULD (not MUST) use the maemo-optify hack or some other mechanism to put most of its data somewhere in /opt. Yes, I hate all the symlinks but it will do the job for now. But don't try to force it on all applications, or do it automatically. Just get the maintainers of the big, commonly installed apps to agree to use the hack for now. I am assuming that there is no way that, at this late stage, Nokia would even consider any serious changes to filesystems (such as union filesystems, etc.). So, much of this discussion is about how it can get fixed in the medium term. The **really** interesting question is going to be whether someone can come up with a solution which Nokia can (and will agree to) apply in a Fremantle update! Graham ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Hello! Other thoughts included: * Use of /opt is perhaps now a QA requirement for Extras * Can we somehow add a /opt check into minimae/maemian? Is it possible, and is it sensible? Please recall that maemo5 is not the only maemo. Maemo4 is the latest availble for N800/N810 and maemo2 is the latest officailly available on 770. Many packages can compile from same source for all versions. Don't add artificial obstacles to force developers to make their packages incompatible with older versions. I also would like to request a solution that does not make packages differ from diablo or Mer. Up to now I was able to have the same packages for all OS versions. A solution that requires me to have different packages for Fremantle and the rest, where the differences are purly textual exchanges of /usr with /opt (which can possible be automated as shown by maemo-optify-deb) creates additional efforts for me (and likely other people packaging software). While the reasons for this are well argumented and need not to be discussed they have their origin in a technical defiency than in a planed architectual or design change and thus possibly should be handled by the package. I would kindly request a solution that: * Either changes packages automatically by the autobuilder. A functionality that seems in principle available. I have no problem with explicitely activating this feature from within the package if the trigger is for example an additional marker file that is ignored by earlier versions (and can be evaluated by Mer any time). * If a simple trigger (Use /opt!) is not sufficient because the underlying algorithm is not good enough I also have no problem with an additional file that states (sub)directories that do not need to be placed under /usr and then can be switched in the package scripts by maemo-optify. Currently I place data like icons, images etc... under /usr/share/... using configures $(pkgdatadir) to hand over a compile time define to the application do be used as search path for application data. With he requested move of application (and/or application data), will such a define still work? Will maemo-optify exchange the call to configure to explicitely set the non-default position of the package data dir to /opt/something...? Will it exchange configure --prefix=/usr with configure --prefix=/opt or something like configure --prefix=/usr --datadir=/opt (not sure if this is --datadir or --datarootdir...? Currently it does use links to fix this (so it should work without patching rules), but it looks like using links is under discussion, too :-/ If not, we are to different packaging anyway I fear... -- Gruß... Tim ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, 10 Sep 2009, Aniello Del Sorbo wrote: We developers already maintain different packages for different Maemo versions, at least Maemo 4 and Maemo 5 as there are too many differences. A Diablo/Chinook/Bora/Gregale package etc won't include this /opt packaging idea (that, indeed, I like). The Fremantle package can be totally different and it's only a matter of changing the destinations of the application files. At least my application will packaged differently for Chinook/Diablo and Fremantle. GUI applications might have big differences. Command line applications probably don't have any. Python apps might not need changes to work on both OS2008 and Fremantle. -- Matan. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, 10 Sep 2009, David Weinehall wrote: On ons, 2009-09-09 at 23:48 +0200, ext Matan Ziv-Av wrote: On Wed, 9 Sep 2009, Andrew Flegg wrote: On Wed, Sep 9, 2009 at 16:22, Attila Csipama...@csipa.in.rs wrote: On Wednesday 09 September 2009 17:09:17 Andrew Flegg wrote: Of course, if maemo-optify were really clever, it'd also handle patterns like the above and produce /opt/bob/share/foo, but I'd consider this an optional enhancement. I wonder if optification could be done on the level of the autobuilder... In that case most developers wouldn't even need to think about it (and/or mess it up :) at all. There was a strong feeling on #maemo (and one I agree with) that the auto-builder shouldn't be changing things in an uploaded package. Other thoughts included: * Use of /opt is perhaps now a QA requirement for Extras * Can we somehow add a /opt check into minimae/maemian? Is it possible, and is it sensible? Please recall that maemo5 is not the only maemo. Maemo4 is the latest availble for N800/N810 and maemo2 is the latest officailly available on 770. Many packages can compile from same source for all versions. Don't add artificial obstacles to force developers to make their packages incompatible with older versions. It's not an artificial obstacle, it's a way to work around the sad reality that VFAT is a.) de facto standard, b.) a seriously useless file system, c.) most users are unable to cope with better alternatives. But by all means, if you think this is an artificial obstacle, please suggest alternative solutions; we'd be very thankful if you can come up with any solution we might've missed. A simple solution is to let the installer handle this in a similar manner to the way doc files are removed after installation. You need: a script that runs after a package is installed and moves+sym links whatever it decides are big files or directories to some other location. You might want to allow package to indicate that some files should not be moved. Such an indication will be ignored by OS not running this script, thus allowing the package to be installed on multiple versions of operating system. -- Matan. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, Sep 10, 2009 at 07:10, Tim Teulings r...@edge.ping.de wrote: I also would like to request a solution that does not make packages differ from diablo or Mer. Up to now I was able to have the same packages for all OS versions. A solution that requires me to have different packages for Fremantle and the rest, where the differences are purly textual exchanges of /usr with /opt (which can possible be automated as shown by maemo-optify-deb) creates additional efforts for me (and likely other people packaging software). While the reasons for this are well argumented and need not to be discussed they have their origin in a technical defiency than in a planed architectual or design change and thus possibly should be handled by the package. The use of /opt/_package_/ on Diablo, Fremantle, Mer, Ubuntu is entirely concordant with the FHS: http://www.pathname.com/fhs/pub/fhs-2.3.html#OPTADDONAPPLICATIONSOFTWAREPACKAGES So, further to my earlier stance, I don't see a problem with building a package which is /opt aware for all the above platforms. Now, maemo-optifying a package and using it on other platforms (and even Maemo) is a hack, so I'd prefer developers used the autoconf approach of --datadir or similar, with only a few things (startup script, icons, service files, desktop files) being either directly installed to /usr (or symlinked to /opt). Cheers, Andrew -- Andrew Flegg -- mailto:and...@bleb.org | http://www.bleb.org/ Maemo Community Council chair ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Wed, 2009-09-09 at 22:22 +0200, Weinehall David (Nokia-D/Tampere) wrote: On ons, 2009-09-09 at 16:53 +0200, ext Andraž 'ruskie' Levstik wrote: :2009-09-09T17:11:Marius Vollmer: Hamalainen Kimmo (Nokia-D/Helsinki) kimmo.hamalai...@nokia.com writes: It would not be trivial task, perhaps it's easier to use some network protocol, or make ext3 support for Windows :) A file-based protocol is the way to go. With ext3, we would still need to unmount the partition to export it, which isn't nice. First time around and not really a dev just a user but: usbnet + scp as an option(winscp?)? there is an ext2/3 driver for windows... a few of them at that as well so maybe use one of those? The problem isn't finding a solution for the computer literati, but rather for normal users. Normal users that doesn't know what scp is. IMHO we have three options: - Real VFAT (with all the drawbacks it brings) - VVFAT - A separate program (PC Suite, most likely) to do the transfers (probably leaving Linux and MacOS users out in the cold) Or a tiny read only partition with winscp.exe and the rest is exposed via ssh/scp Only problem is ssh getting outdated and all the devices being exploited. Bad. -- Maemo Software Nokia Devices ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
:2009-09-09T23:22:David Weinehall: The problem isn't finding a solution for the computer literati, but rather for normal users. Normal users that doesn't know what scp is. IMHO we have three options: - Real VFAT (with all the drawbacks it brings) - VVFAT - A separate program (PC Suite, most likely) to do the transfers (probably leaving Linux and MacOS users out in the cold) I wasn't suggesting they need to know what scp is. It could all be hidden under some filesystem(like sshfs) so when they plug it in all they get is a new drive letter under windows. Of course it would require getting that support but it would work probably way better than any of the other solutions. Or samba as an option? That would be effective as well if not scp :) But basically I think and believe the best and easiest interop system would be networked. I've setup sshfs for regular users(this would be much easier with usbnet and predefined stuff) and nobody ever notices anything odd about it :) -- Andraž ruskie Levstik Source Mage GNU/Linux Games/Xorg grimoire guru Re-Alpine Coordinator http://sourceforge.net/projects/re-alpine/ Geek/Hacker/Tinker Be sure brain is in gear before engaging mouth.___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, 2009-09-10 at 08:48 +0200, ext Matan Ziv-Av wrote: On Thu, 10 Sep 2009, David Weinehall wrote: On ons, 2009-09-09 at 23:48 +0200, ext Matan Ziv-Av wrote: On Wed, 9 Sep 2009, Andrew Flegg wrote: On Wed, Sep 9, 2009 at 16:22, Attila Csipama...@csipa.in.rs wrote: On Wednesday 09 September 2009 17:09:17 Andrew Flegg wrote: Of course, if maemo-optify were really clever, it'd also handle patterns like the above and produce /opt/bob/share/foo, but I'd consider this an optional enhancement. I wonder if optification could be done on the level of the autobuilder... In that case most developers wouldn't even need to think about it (and/or mess it up :) at all. There was a strong feeling on #maemo (and one I agree with) that the auto-builder shouldn't be changing things in an uploaded package. Other thoughts included: * Use of /opt is perhaps now a QA requirement for Extras * Can we somehow add a /opt check into minimae/maemian? Is it possible, and is it sensible? Please recall that maemo5 is not the only maemo. Maemo4 is the latest availble for N800/N810 and maemo2 is the latest officailly available on 770. Many packages can compile from same source for all versions. Don't add artificial obstacles to force developers to make their packages incompatible with older versions. It's not an artificial obstacle, it's a way to work around the sad reality that VFAT is a.) de facto standard, b.) a seriously useless file system, c.) most users are unable to cope with better alternatives. But by all means, if you think this is an artificial obstacle, please suggest alternative solutions; we'd be very thankful if you can come up with any solution we might've missed. A simple solution is to let the installer handle this in a similar manner to the way doc files are removed after installation. You need: a script that runs after a package is installed and moves+sym links whatever it decides are big files or directories to some other location. You might want to allow package to indicate that some files should not be moved. Such an indication will be ignored by OS not running this script, thus allowing the package to be installed on multiple versions of operating system. And that indication is already a packaging difference. I'd rather do ./configure --prefix=/opt/package and that's it. Cleaner IMHO. Cheers, -- Maemo Software Nokia Devices ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
:2009-09-10T10:25:Mohammed Hassan: Or a tiny read only partition with winscp.exe and the rest is exposed via ssh/scp Only problem is ssh getting outdated and all the devices being exploited. Bad. Which is why it would only listen on usb0 by default :) So hard to exploit it don't you think :) with a user only login and scponly shell :) -- Andraž ruskie Levstik Source Mage GNU/Linux Games/Xorg grimoire guru Re-Alpine Coordinator http://sourceforge.net/projects/re-alpine/ Geek/Hacker/Tinker Don't forget: the future is now. It's just not widely distributed yet.___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
:2009-09-10T10:31:Mohammed Hassan: And that indication is already a packaging difference. I'd rather do ./configure --prefix=/opt/package and that's it. Frankly if this would become a standard I think there'll be atleast some people that will setup their own autobuilder to install to regular /usr or /usr/local. I know I'll remove any traces of vfat and expand existing partitions so I'll be able to install normally. But I can accept that some users will want/require vfat. I wouldn't be opposed to having two repos opt and usr so that a user can make their own choice on how to do it. That way nobody would be limited to one or the other. Just my 0.02€ ;) -- Andraž ruskie Levstik Source Mage GNU/Linux Games/Xorg grimoire guru Re-Alpine Coordinator http://sourceforge.net/projects/re-alpine/ Geek/Hacker/Tinker Ryle hira.___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, Sep 10, 2009 at 08:41, Andraž 'ruskie' Levstik rus...@codemages.net wrote: Frankly if this would become a standard I think there'll be atleast some people that will setup their own autobuilder to install to regular /usr or /usr/local. Why do you care, as a user, where the package contents get installed? I really struggle to think of any really compelling reason. I know I'll remove any traces of vfat and expand existing partitions so I'll be able to install normally. Things will be installed into /opt transparently to the user using App Mgr, apt or dpkg. The burden is on the packager, not the user. I wouldn't be opposed to having two repos opt and usr so that a user can make their own choice on how to do it. That way nobody would be limited to one or the other. I really can't see this working, and don't understand the limited to bit - or your desire to switch. Cheers, Andrew -- Andrew Flegg -- mailto:and...@bleb.org | http://www.bleb.org/ Maemo Community Council chair ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, 2009-09-10 at 09:30 +0200, ext Andraž 'ruskie' Levstik wrote: :2009-09-09T23:22:David Weinehall: The problem isn't finding a solution for the computer literati, but rather for normal users. Normal users that doesn't know what scp is. IMHO we have three options: - Real VFAT (with all the drawbacks it brings) - VVFAT - A separate program (PC Suite, most likely) to do the transfers (probably leaving Linux and MacOS users out in the cold) I wasn't suggesting they need to know what scp is. It could all be hidden under some filesystem(like sshfs) so when they plug it in all they get is a new drive letter under windows. Of course it would require getting that support but it would work probably way better than any of the other solutions. Or samba as an option? That would be effective as well if not scp :) But basically I think and believe the best and easiest interop system would be networked. I've setup sshfs for regular users(this would be much easier with usbnet and predefined stuff) and nobody ever notices anything odd about it :) USB networking does not work out-of-the-box, it requires driver installation. It would not work with PS3 or Xbox360 either. USB mass storage is still the most supported way to transfer files over USB. But yes, if you happen to have the drivers, then you could have something more convenient than USB mass storage. -Kimmo -- Andraž ruskie Levstik Source Mage GNU/Linux Games/Xorg grimoire guru Re-Alpine Coordinator http://sourceforge.net/projects/re-alpine/ Geek/Hacker/Tinker Be sure brain is in gear before engaging mouth. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, 2009-09-10 at 09:41 +0200, ext Andraž 'ruskie' Levstik wrote: :2009-09-10T10:31:Mohammed Hassan: And that indication is already a packaging difference. I'd rather do ./configure --prefix=/opt/package and that's it. Frankly if this would become a standard I think there'll be atleast some people that will setup their own autobuilder to install to regular /usr or /usr/local. I know I'll remove any traces of vfat and expand existing partitions so I'll be able to install normally. But I can accept that some users will want/require vfat. I wouldn't be opposed to having two repos opt and usr so that a user can make their own choice on how to do it. That way nobody would be limited to one or the other. What I meant was introducing a method to inform the installer that we some files need not be moved is a packaging change already. It's something fremantle specific and I'd rather do it the cleaner way then. Maybe we should have DEB_BUILD_OPTS += optify ? Problem is if you start using dh_install and friends, you will be in limbo. Cheers, -- Maemo Software Nokia Devices ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Graham Cobb g+...@cobb.uk.net writes: Hmm. Are there some objective criteria for what should go in opt? Not really. The proposed tool, maemo-optify, has a hard-coded, builtin heuristic to select which files to move. It is supposed to be a fire-and-forget action: you add a simple call to maemo-optify to debian/rules, and the heuristic does the right thing. The current heuristic moves files that are larger than 2 kiB, and directories that are larger than 2 kiB and are named like the package. This is the output for ecoach, for example: $ maemo-optify-deb ecoach_1.53beta3_i386.deb usr/bin/ecoach - opt/maemo/usr/bin/ecoach usr/share/ecoach - opt/maemo/usr/share/ecoach usr/share/icons/hicolor/scalable/hildon/ecoach.png - \ opt/maemo/usr/share/icons/hicolor/scalable/hildon/ecoach.png usr/share/pixmaps/ecoach - opt/maemo/usr/share/pixmaps/ecoach ecoach: optified 4 entries, saving about 546 kB. In light of the recent discussion, the 2 kiB threshold is maybe a bit low. Of course, we should add some way to control the heuristic of maemo-optify, but I think it is important that it does something reasonably by default. Can anyone suggest good tools for determining how much space my packages will be taking up, particularly by directory (e.g. I would like to see how much space they take up in /usr/bin, /usr/lib, etc.)? You can extract it with $ dpkg-deb -x package.deb and then investigate the ./package/ directory with du, etc. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
:2009-09-10T08:55:Andrew Flegg: On Thu, Sep 10, 2009 at 08:41, Andraž 'ruskie' Levstik rus...@codemages.net wrote: Frankly if this would become a standard I think there'll be atleast some people that will setup their own autobuilder to install to regular /usr or /usr/local. Why do you care, as a user, where the package contents get installed? I really struggle to think of any really compelling reason. Because I expect certain things to be where I expect them. Just like on a BSD system I expect anything I install to be in /usr/local so I would expect on any GNU+Linux system to have them in /usr I know I'll remove any traces of vfat and expand existing partitions so I'll be able to install normally. Things will be installed into /opt transparently to the user using App Mgr, apt or dpkg. The burden is on the packager, not the user. That's great for those users that will only really use the nice UI. I wouldn't be opposed to having two repos opt and usr so that a user can make their own choice on how to do it. That way nobody would be limited to one or the other. I really can't see this working, and don't understand the limited to bit - or your desire to switch. It's about the freedom to choose how one wishes to configure a system they own(you know I pay for it therefor I own the device and am free to do with it as I please, unlike certain other industries that want to rent it out to you only). As said I can understand there might be technical limitations to preferring /opt or any other location for this but I think it shouldn't be limited to that. Another solution which I don't know if anyone looked at it would be: LVM. Simply have a small basic root with everything needed to boot there and then have /usr and /home as LVM volumes with triggers in dpkg(I guess would be the lowest most layer) to re-adjust size dynamically to needed size. Optimal solution really. :) Regards -- Andraž ruskie Levstik Source Mage GNU/Linux Games/Xorg grimoire guru Re-Alpine Coordinator http://sourceforge.net/projects/re-alpine/ Geek/Hacker/Tinker Be sure brain is in gear before engaging mouth.___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Matan Ziv-Av wrote: On Wed, 9 Sep 2009, David Greaves wrote: Matan Ziv-Av wrote: On Wed, 9 Sep 2009, Andrew Flegg wrote: * Use of /opt is perhaps now a QA requirement for Extras * Can we somehow add a /opt check into minimae/maemian? Is it possible, and is it sensible? Please recall that maemo5 is not the only maemo. Maemo4 is the latest availble for N800/N810 and maemo2 is the latest officailly available on 770. Many packages can compile from same source for all versions. Don't add artificial obstacles to force developers to make their packages incompatible with older versions. *cough* Mer*cough* Call me when Mer is a reasonable replacement for OS2008 on N810. Until then, please don't try to fragment the community for no reason except for planned obsolescence. Had I ranted on how it was essential to do something different to provide definite support for Mer then that would have been a reasonable although impolite response. However since it was a gentle reminder that Mer *may* be on the horizon as yet another environment that packagers may like to support from the same tarball - and one that doesn't currently make use of /opt - then I really don't understand it. However I don't see much evidence that Nokia are considering the legacy community here - over time application writers are likely to package for the current platform and it's going to make supporting the N810 and older even harder for the few of us that are working very, very hard on a way to extend the useable life of those devices. Now this /opt thing may not affect packaging for Maemo4 or Maemo2 or Mer - but I'm not convinced that that is down to luck, not design. (However if the decision to use /opt and the current proposed solution *does* have supporting Maemo4 as a requirement and not just a side-effect then I apologise.) Oh, and Matan, having potential users tell us to shut up and call them when it's ready is frankly offensive. If you want your N810 (I assume) to have something that runs apps from fremantle-extras then come and lend a hand. If you are going to use symlinks anyway, let the package installer make them, so the same package can be used for various system. A lot of packages available for maemo are merely debian/ubuntu packages built in maemo environment. This /opt idea will reduce the amount of software available for maemo5 devices. So despite the comment above you appear to be in agreement with my comments supporting consistency with Debian a few mails above... It certainly does have the potential to make porting more complex packages harder than it needs to be. David -- Don't worry, you'll be fine; I saw it work in a cartoon once... ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, 2009-09-10 at 10:06 +0200, ext Andraž 'ruskie' Levstik wrote: :2009-09-10T08:55:Andrew Flegg: On Thu, Sep 10, 2009 at 08:41, Andraž 'ruskie' Levstik rus...@codemages.net wrote: Frankly if this would become a standard I think there'll be atleast some people that will setup their own autobuilder to install to regular /usr or /usr/local. Why do you care, as a user, where the package contents get installed? I really struggle to think of any really compelling reason. Because I expect certain things to be where I expect them. Just like on a BSD system I expect anything I install to be in /usr/local so I would expect on any GNU+Linux system to have them in /usr But you are not a normal user here :-) You know that stuff ends in /usr/local on BSD and in /usr on GNU/Linux and you can know that they end up in /opt in Maemo5 It's almost impossible to support everyone. Point is to support the important/common uses but not make the life of the minor uses miserable. If you already know that we have /usr or that there's something callled autobuilder and know you can set it up then you are not a normal user. And I'm not wearing my Nokia hat here. I'm wearing my FOSS hat :) Cheers, -- Maemo Software Nokia Devices ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, Sep 10, 2009 at 09:05, Marius Vollmer marius.voll...@nokia.com wrote: ext Graham Cobb g+...@cobb.uk.net writes: Hmm. Are there some objective criteria for what should go in opt? Not really. The proposed tool, maemo-optify, has a hard-coded, builtin heuristic to select which files to move. It is supposed to be a fire-and-forget action: you add a simple call to maemo-optify to debian/rules, and the heuristic does the right thing. The current heuristic moves files that are larger than 2 kiB, and directories that are larger than 2 kiB and are named like the package. This is the output for ecoach, for example: $ maemo-optify-deb ecoach_1.53beta3_i386.deb usr/bin/ecoach - opt/maemo/usr/bin/ecoach usr/share/ecoach - opt/maemo/usr/share/ecoach usr/share/icons/hicolor/scalable/hildon/ecoach.png - \ opt/maemo/usr/share/icons/hicolor/scalable/hildon/ecoach.png usr/share/pixmaps/ecoach - opt/maemo/usr/share/pixmaps/ecoach ecoach: optified 4 entries, saving about 546 kB. Definitely auto=adding a call to maemo-optify is something that MUD might do (as Graham already suggested), and I agree it's something which shouldn't be done at the autobuilder (at least until it becomes a problem that no-one's using /opt). However, I'd love to see the output above be: $ maemo-optify-deb ecoach_1.53beta3_i386.deb usr/bin/ecoach - opt/ecoach/bin/ecoach usr/share/ecoach - opt/ecoach/share/ecoach usr/share/icons/hicolor/scalable/hildon/ecoach.png - \ opt/ecoach/share/icons/hicolor/scalable/hildon/ecoach.png usr/share/pixmaps/ecoach - opt/ecoach/share/pixmaps/ecoach ecoach: optified 4 entries, saving about 546 kB. In this, I thought about usr/share/ecoach - opt/ecoach/share, but it introduces problems and over-complications with .../share/pixmaps and .../icons. What do you think Marius? In light of the recent discussion, the 2 kiB threshold is maybe a bit low. Indeed. What's the disk space usage of a symlink? ;-) Of course, we should add some way to control the heuristic of maemo-optify, but I think it is important that it does something reasonably by default. Absolutely (hence my keenness on it being more FHS compliant by using the package name). Cheers, Andrew -- Andrew Flegg -- mailto:and...@bleb.org | http://www.bleb.org/ Maemo Community Council chair ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Thomas Perl th.p...@gmail.com writes: 2009/9/9 Marius Vollmer marius.voll...@nokia.com: ext David Greaves da...@dgreaves.com writes: Hmm, seems like another solution would be to have the opt partition mounted as /usr and install all the 'standard' stuff into /root_usr/ and preinstall symlinks into /usr - /root_usr Yeah, that would work but we unfortunately can only install into the rootfs partition when creating FIASCO images, due to the tools that create these images. I think David's suggestion would be more sane, Yes, immensely so. as packages don't have to be changed (Debian packages normally install into /usr, so that's already standard and works well). Exactly. If we really want to split our roofs over two partitions, we should put / and the first and /usr on the second. No symlinks etc would be needed; we can mount /usr as early as needed. However, spreading the rootfs over two partitions makes things more complicated, of course. For example, preparing a FIASCO for the rootfs now must prepare two filesystem images, and flashing such a FIASCO must write to those two partitions. This is of course doable, but it requires changes that we are unfortunately not able to do for Fremantle. The shit hit the fan too late, you might say. Moving /opt to a second partition can be done now, since /opt is entirely empty and we do not need to include a filesystem image for it in the FIASCO files. It doesn't have to be /opt of course, we could have invented a new directory such as /space and moved that to the big flash. I think that would have been cleaner, since we do not want any of the usual semantics of /opt here, we just want to extent the size of the rootfs with some hacks. (That's why maemo-optify moves files into /opt/maemo/, to stay out of the way of the real inhabitants of /opt. More about this later.) Now, what about the future? Should we move /usr to a second partition? I would say no. Let's move the whole of / to the big flash. This keeps things simple; only one filesystem image to worry about. In my ideal world, we would only have one big filesystem for everything. This probably isn't possible, for two reasons: - We might want to continue to export /home/user/MyDocs as a block-device over USB for some time still, until we have a viable and proven replacement. This means that /home/usr/MyDocs should be on its own VFAT partition. - We want to reinstall the OS without destroying user content, and for robustness reasons, we want to reinstall the OS by writing a filesystem image to a partition (instead of deleting files and writing new ones in a existing filesystem). This means that /home should be in its own ext3 or ubifs partition. I have good hopes that we eventually can get rid of the /home/user/MyDocs partition, and that would leave us with a system with two partitions: / and /home, which is fairly standard, I'd say. Would that really be a problem? Using /opt just seems wrong and nonstandard, sorry :/ Yes, but it's just a name. :-) As I said, it would be better to do this hack with /space instead of /opt/maemo/, but I think that's just a minor detail. Think about http://wiki.maemo.org/Mainstream_Linux_Alignment, please :) We will get rid of this abuse of /opt as fast as we can. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Andrew Flegg and...@bleb.org writes: Limiting it to a hack for a large app results in a question of when is an app large: 1MB, 2MB, 4MB, 8MB, 16MB, ...? If we can have a consensus on *that*, that could be something we have a QA check for? I'd say we chould check directly for what we care about: size of files outside of /opt. Packages are only allowed to have a average size of files outside of /opt of, say, 20 kiB. I.e., they must satisfy (size of files not in /opt) / (number of files not in /opt) 20 kiB. The 20 kiB is of course debatable. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
:2009-09-10T11:29:Mohammed Hassan: But you are not a normal user here :-) I guess it depends what a normal user would be for this device. If it were for example my mom(I'm being very specific here) she would know to expect /mnt and /media and even /usr/bin and such things(and yes she uses Xubuntu). You know that stuff ends in /usr/local on BSD and in /usr on GNU/Linux and you can know that they end up in /opt in Maemo5 It's almost impossible to support everyone. Point is to support the important/common uses but not make the life of the minor uses miserable. I guess that's the point I'm arguing. It should be possible to do it without jumping through hoops. So I guess in the end I don't really care as long as it won't be to hard to adjust it back to /usr or /usr/local without symlinks. That's really the one solution I dislike - symlinks. If you already know that we have /usr or that there's something callled autobuilder and know you can set it up then you are not a normal user. And I'm not wearing my Nokia hat here. I'm wearing my FOSS hat :) I really don't care what hat someone wears ;) I react to everyone in the same manner. Regards -- Andraž ruskie Levstik Source Mage GNU/Linux Games/Xorg grimoire guru Re-Alpine Coordinator http://sourceforge.net/projects/re-alpine/ Geek/Hacker/Tinker Don't forget: the future is now. It's just not widely distributed yet.___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Graham Cobb wrote: I would suggest that Nokia add /opt/bin to the PATH, add /opt/lib to the LD_LIBRARY_PATH and add /opt/lib/pkgconfig to PKG_CONFIG_PATH (all on the device and in scratchbox) and that we ignore the FHS rule that packages should not install into those directories. That would make things a little easier and more robust. We can then build with --prefix=/opt and add a couple of softlinks if necessary for files which still need to be elsewhere. And if you depend on porting in a few other debian packages you'll repackage them for maemo? Doable - but to expect other devs to do that? At the moment 'porting' an app is usually a semi-automated process. Having to figure out the debian/rules for each and then hacking their ./configure to --prefix=/opt Oh boy PLEASE don't require a ./configure change optify is the lesser evil David -- Don't worry, you'll be fine; I saw it work in a cartoon once... ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, Sep 10, 2009 at 09:52, Graham Cobb g+...@cobb.uk.net wrote:
On Thursday 10 September 2009 07:53:37 Andrew Flegg wrote:
The use of /opt/_package_/ on Diablo, Fremantle, Mer, Ubuntu is
entirely concordant with the FHS:
Well, not really. That FHS section is clear that if you are using /opt,
nothing should be installed in (for example) /usr/bin (or even
/opt/bin).
True. I meant nomenclature and so on. But the following is my FHS++ idea:
/opt/foo/bin/foo
/opt/foo/share/some_big_data_file.wad
/opt/foo/lib/some_bespoke_library.{py,so}
/usr/bin/foo - /opt/foo/bin/foo
/usr/share/applications/hildon/foo.desktop
/usr/lib/dbus-1/services/foo.service
/usr/share/icons//foo.png
It *seems* much cleaner, architecturally, for end-user apps. Perhaps
with the mass of symlinks from maemo-optify as a solution for
libraries etc.
However, m-vo's apparently got reasons about just having a
/opt/maemo prefix over a /opt/_package_ structure. I look forward
to reading it ;-)
[snip sensible stuff]
I have no idea whether this will work for python -- how does python find
things like libraries (or whatever the equivalent concept is in python)?
There's a PYTHON_PATH. Python's particularly troublesome as lots of
additional packages (like python-twitter and python-evolution) are
extra dependencies of applications.
How well will dpkg cope when, say, python-core has created
/opt/maemo/usr/lib/python2.5/site-packages and /usr/lib/python2.5 -
/opt/maemo/usr/lib/python2.5 and then python-evolution comes along and
tries to put /usr/lib/python2.5/site-packages/evolution -
/opt/maemo/usr/lib/python2.5/site-packages/evolution in? I suspect,
this over symlinking could be problematic.
The real (and best) solution is a unionfs. IIRC, there isn't one in
the mainline, because of problems in some use cases. However for the
use case of we want to be able to install apps over a single
partition, the solutions are viable, IMHO.
Valerie Aurora recently wrote some LWN articles on the issues, which
make good reading:
http://lwn.net/Articles/324291/
http://lwn.net/Articles/325369/
http://lwn.net/Articles/327738/
Index and more thoughts at:
http://valerieaurora.org/union/
I would have thought that in the time still available compiling one of
these as a module and mounting the partition over / (or /usr) would be
entirely viable. m-vo, thoughts?
Cheers,
Andrew
--
Andrew Flegg -- mailto:and...@bleb.org | http://www.bleb.org/
Maemo Community Council chair
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Marius Vollmer wrote: ext Thomas Perl th.p...@gmail.com writes: 2009/9/9 Marius Vollmer marius.voll...@nokia.com: ext David Greaves da...@dgreaves.com writes: Hmm, seems like another solution would be to have the opt partition mounted as /usr and install all the 'standard' stuff into /root_usr/ and preinstall symlinks into /usr - /root_usr Yeah, that would work but we unfortunately can only install into the rootfs partition when creating FIASCO images, due to the tools that create these images. I think David's suggestion would be more sane, Yes, immensely so. as packages don't have to be changed (Debian packages normally install into /usr, so that's already standard and works well). Exactly. If we really want to split our roofs over two partitions, we should put / and the first and /usr on the second. No symlinks etc would be needed; we can mount /usr as early as needed. However, spreading the rootfs over two partitions makes things more complicated, of course. For example, preparing a FIASCO for the rootfs now must prepare two filesystem images, and flashing such a FIASCO must write to those two partitions. This is of course doable, but it requires changes that we are unfortunately not able to do for Fremantle. The shit hit the fan too late, you might say. Really? I would think you could prepare /usr_root and /usr on the single partition. /usr contains per-file symlinks to /usr_root and is hidden when the real /usr is mounted. Voila - one fs image for FIASCO and no need to mount /usr early. The problem is now getting the updated symlinks onto the real /usr Maybe something like: As the fiasco boots it mounts partition for /usr on /usr_old Checks /usr_old/version against /usr/version If there is a mismatch it tars up /usr/ and untars it to /usr_old Then remount /usr_old on /usr The tar thing could include a manifest from the previous image so you could tidy up properly. I realise we have optify for now... Just exploring for fun. We will get rid of this abuse of /opt as fast as we can. Thanks :) -- Don't worry, you'll be fine; I saw it work in a cartoon once... ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, Sep 10, 2009 at 09:48, Marius Vollmer marius.voll...@nokia.com wrote: We will get rid of this abuse of /opt as fast as we can. That changes things, if /opt is going away (in a Maemo Update?) - *no* package should be using it; and it _should_ be done as part of the auto-builder (however distasteful that may be). (Maybe) Cheers, Andrew -- Andrew Flegg -- mailto:and...@bleb.org | http://www.bleb.org/ Maemo Community Council chair ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, 10 Sep 2009, David Greaves wrote: Matan Ziv-Av wrote: On Wed, 9 Sep 2009, David Greaves wrote: Matan Ziv-Av wrote: On Wed, 9 Sep 2009, Andrew Flegg wrote: * Use of /opt is perhaps now a QA requirement for Extras * Can we somehow add a /opt check into minimae/maemian? Is it possible, and is it sensible? Please recall that maemo5 is not the only maemo. Maemo4 is the latest availble for N800/N810 and maemo2 is the latest officailly available on 770. Many packages can compile from same source for all versions. Don't add artificial obstacles to force developers to make their packages incompatible with older versions. *cough* Mer*cough* Call me when Mer is a reasonable replacement for OS2008 on N810. Until then, please don't try to fragment the community for no reason except for planned obsolescence. Had I ranted on how it was essential to do something different to provide definite support for Mer then that would have been a reasonable although impolite response. However since it was a gentle reminder that Mer *may* be on the horizon as yet another environment that packagers may like to support from the same tarball - and one that doesn't currently make use of /opt - then I really don't understand it. However I don't see much evidence that Nokia are considering the legacy community here - over time application writers are likely to package for the current platform and it's going to make supporting the N810 and older even harder for the few of us that are working very, very hard on a way to extend the useable life of those devices. Now this /opt thing may not affect packaging for Maemo4 or Maemo2 or Mer - but I'm not convinced that that is down to luck, not design. (However if the decision to use /opt and the current proposed solution *does* have supporting Maemo4 as a requirement and not just a side-effect then I apologise.) Oh, and Matan, having potential users tell us to shut up and call them when it's ready is frankly offensive. If you want your N810 (I assume) to have something that runs apps from fremantle-extras then come and lend a hand. There seems to be a misunderstanding here. I opined that the suggested change might make it harder to make packages compatible between fremantle and OS2008, and that this is a bad thing. I then interpreted your *cough* Mer*cough* comment as saying that compatibility with OS2008 is irrelevant, since Mer is expected to be installed on every N800/N810 device. My reply merely meant to say that Mer is not currently an acceptable replacement for most N800/N810 users, and that I do not expect it to be in the near future, thus having packages compatible with OS2008 whenever possible is an important goal. This is not a slight on Mer or Mer developers, but mere statment of facts as I see them. I am a user and developer of Mer, and it is my preferred OS on smartq devices, but is not yet a replacement for OS2008 on N810. Power management and compatibility with some applications are the main issues I see right now. We actually seem to be in agreement here: Fremantle packaging decisions should take other systems (OS2008, Mer, etc.) into account. -- Matan. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, 10 Sep 2009, Mohammed Hassan wrote: On Thu, 2009-09-10 at 08:48 +0200, ext Matan Ziv-Av wrote: On Thu, 10 Sep 2009, David Weinehall wrote: On ons, 2009-09-09 at 23:48 +0200, ext Matan Ziv-Av wrote: On Wed, 9 Sep 2009, Andrew Flegg wrote: On Wed, Sep 9, 2009 at 16:22, Attila Csipama...@csipa.in.rs wrote: On Wednesday 09 September 2009 17:09:17 Andrew Flegg wrote: Of course, if maemo-optify were really clever, it'd also handle patterns like the above and produce /opt/bob/share/foo, but I'd consider this an optional enhancement. I wonder if optification could be done on the level of the autobuilder... In that case most developers wouldn't even need to think about it (and/or mess it up :) at all. There was a strong feeling on #maemo (and one I agree with) that the auto-builder shouldn't be changing things in an uploaded package. Other thoughts included: * Use of /opt is perhaps now a QA requirement for Extras * Can we somehow add a /opt check into minimae/maemian? Is it possible, and is it sensible? Please recall that maemo5 is not the only maemo. Maemo4 is the latest availble for N800/N810 and maemo2 is the latest officailly available on 770. Many packages can compile from same source for all versions. Don't add artificial obstacles to force developers to make their packages incompatible with older versions. It's not an artificial obstacle, it's a way to work around the sad reality that VFAT is a.) de facto standard, b.) a seriously useless file system, c.) most users are unable to cope with better alternatives. But by all means, if you think this is an artificial obstacle, please suggest alternative solutions; we'd be very thankful if you can come up with any solution we might've missed. A simple solution is to let the installer handle this in a similar manner to the way doc files are removed after installation. You need: a script that runs after a package is installed and moves+sym links whatever it decides are big files or directories to some other location. You might want to allow package to indicate that some files should not be moved. Such an indication will be ignored by OS not running this script, thus allowing the package to be installed on multiple versions of operating system. And that indication is already a packaging difference. But such a packaging difference leaves a package compatible with OS2008, while moving some files to /opt might not be. I'd rather do ./configure --prefix=/opt/package and that's it. That means two packages needed - one for fremantle and one for OS2008, while in many cases it could be unnecessary. -- Matan. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Matan Ziv-Av wrote: I then interpreted your *cough* Mer*cough* comment as saying that compatibility with OS2008 is irrelevant, since Mer is expected to be installed on every N800/N810 device. Ah. That would be nice but we know we're not close to that yet. We actually seem to be in agreement here: Fremantle packaging decisions should take other systems (OS2008, Mer, etc.) into account. Sure - lets chalk it down to email missing the nuances :) David -- Don't worry, you'll be fine; I saw it work in a cartoon once... ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thursday 10 September 2009 10:15:44 David Greaves wrote: Graham Cobb wrote: I would suggest that Nokia add /opt/bin to the PATH, add /opt/lib to the LD_LIBRARY_PATH and add /opt/lib/pkgconfig to PKG_CONFIG_PATH (all on the device and in scratchbox) and that we ignore the FHS rule that packages should not install into those directories. That would make things a little easier and more robust. We can then build with --prefix=/opt and add a couple of softlinks if necessary for files which still need to be elsewhere. And if you depend on porting in a few other debian packages you'll repackage them for maemo? No, in most cases just leave them and let them install into /usr as normal -- most of them are small libraries or, occasionally, small command line apps. At the moment 'porting' an app is usually a semi-automated process. Having to figure out the debian/rules for each and then hacking their ./configure to --prefix=/opt Many of the Debian packages I have to port to get the libraries I need require a configure change. That is one of the things I normally expect. Along with changing the Maintainer, often changing the dependencies (usually linked with the configure change -- to remove a dependency on something not ported and not relevant for Maemo), changing the Makefile.am to remove building the docs, etc. Doesn't take long. PLEASE don't require a ./configure change optify is the lesser evil My position is completely the opposite. Where possible make a configure change -- only use optify in the cases where using configure doesn't work (like source code which has hard coded paths). How about a combination: 1) Allow use of optify -- not automatically in the autobuilder but as a single line in the debian/rules. Any package that wants to use that is fine. 2) Nokia sets up the environment (in both the SDK and on the device) to include /opt directories in PATH, LD_LIBRARY_PATH, PKG_CONFIG_PATH and PYTHON_PATH. In that case, I would encourage Mer to also implement this. 3) Allow use of --prefix=/opt. Any package that wants to use that is fine. In particular, allow installation into /opt/bin, /opt/lib, etc (not just into /opt/package). Note that this is not FHS compliant (FHS would require --prefix=/opt/package). An alternative to 2 and 3 is to use --prefix=/opt/package and make the package install symlinks where necessary (e.g. into /usr/lib/pkgconfig and into /usr/bin). More work for the maintainer but better FHS compliance and a better chance of working on old versions of Maemo. Graham ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, Sep 10, 2009 at 10:58, Matan Ziv-Av ma...@svgalib.org wrote: But such a packaging difference leaves a package compatible with OS2008, while moving some files to /opt might not be. Why would moving some files to /opt make it incompatible with OS2008? It may not be how OS2008 packages have worked to date, but then OS2005 packages ended up in /var/lib/install before OS2006 came along. Cheers, Andrew -- Andrew Flegg -- mailto:and...@bleb.org | http://www.bleb.org/ Maemo Community Council chair ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thursday 10 September 2009 05:45:22 quim@nokia.com wrote: Limiting it to a hack for a large app results in a question of when is an app large: 1MB, 2MB, 4MB, 8MB, 16MB, ...? If we can have a consensus on *that*, that could be something we have a QA check for? Developers are encouraged to make good use of them, specially for applications requiring more than 500KB, To reiterate, my main concern is that it has been said that for various (perfectly understandable) reasons, libraries are NOT to be optified. Now, the root partition has _far_ less free space than to accomodate all the libraries - for example a tiny 1KB facebook python-webkit app can set your limited root space back with 60MB worth of libs without blinking an eye (=conforming fully with the guidelines). Is this a valid concern or did I get it wrong ? including dependencies. You can't really count the size of those, as they might have dependencies of their own, some are parts of the OS, and most of the time you don't know how many other apps will use them (if 100 apps depend on a 400KB package, there might be more ratio to optify it than if it's just one app depending on a 600KB package). ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Attila Csipa wrote: To reiterate, my main concern is that it has been said that for various (perfectly understandable) reasons, libraries are NOT to be optified. I haven't followed all the email from this thread but until yesterday libraries were thought to be good candidates for the /opt and actually specific examples for Python or Qt were given. including dependencies. You can't really count the size of those, as they might have dependencies of their own, some are parts of the OS, and most of the time you don't know how many other apps will use them (if 100 apps depend on a 400KB package, there might be more ratio to optify it than if it's just one app depending on a 600KB package). The Application Manager does make a size estimate, I would say useful enough. Before installing the first Python app all the Python apps are very thick in the AM. Once you have the Python libraries most of them are quite thin actually. In practice, if the maintainers of libraries have done their homework optifying properly and we still agree with the 500kb rule, I beliebe this whole /opt thing will affect only a bunch of developers very conscious about the weight of their software and the problems it might cause to average users. -- Quim Gil open source advocate Maemo Devices @ Nokia ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Weinehall David (Nokia-D/Tampere) david.weineh...@nokia.com writes: IMHO we have three options: - Real VFAT (with all the drawbacks it brings) - VVFAT - A separate program (PC Suite, most likely) to do the transfers (probably leaving Linux and MacOS users out in the cold) There is also MTP over USB. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Aniello Del Sorbo ani...@gmail.com writes: A Diablo/Chinook/Bora/Gregale package etc won't include this /opt packaging idea (that, indeed, I like). There is no harm in using /opt also for non-Fremantle releases. That is, if you 'optify' your package for Fremantle, you can still ship it for the older releases with the optification in it. Also, you can make it so that maemo-optify only runs in debian/rules when it is present: which maemo-optify maemo-optify ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext David Greaves da...@dgreaves.com writes: (However if the decision to use /opt and the current proposed solution *does* have supporting Maemo4 as a requirement and not just a side-effect then I apologise.) No, it's just a side-effect, I am afraid. :-) However, as I wrote in another mail, optified packages will work on all systems, not just on Fremantle. They sure are ugly and I think it is fair to hate Fremantle for pouring its ugliness into Mer, but I hope it's possible to live with that... ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thursday 10 September 2009 13:06:03 Quim Gil wrote: ext Attila Csipa wrote: To reiterate, my main concern is that it has been said that for various (perfectly understandable) reasons, libraries are NOT to be optified. I haven't followed all the email from this thread but until yesterday libraries were thought to be good candidates for the /opt and actually specific examples for Python or Qt were given. I was referring to this message (unless the current interpretation is that Python and Qt are not system libraries): On Wednesday 09 September 2009 14:58:14 Kimmo Hämäläinen wrote: This was, of course, considered as well, but the problem is that concurrent I/O to/from the eMMC can (for reasons kernel guys know) cause high load in the system. Hence eMMC is avoided for system libraries and stuff, because it would cause a performance problem when these libraries are used and e.g. swap is active. There are other technical problems with eMMC as well but it should work for this application use case pretty well. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Wed, 2009-09-09 at 14:25 +0300, Quim Gil wrote: The maemo-optify tool helps developers to prepare Debian packages that use /opt. This tool moves selected files inside the package to locations under /opt, and will symbolically link from the original location to the new place of the file. An early version of the tool can be found at http://gitorious.org/maemo-af/maemo-optify - see the README file for details. So we can just run maemo-optify-deb on our package's debian/ directory, reupload and not worry about it again? (Assuming that maemo-optify-deb worked OK.) -- murr...@murrayc.com www.murrayc.com www.openismus.com ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Hassan Mohammed.2 (Nokia-D/Helsinki) mohammed.2.has...@nokia.com writes: And that indication is already a packaging difference. I'd rather do ./configure --prefix=/opt/package and that's it. And then you have to deal with the fallout from that. If you are ready to do that, more power to you, but I really would like to keep a very low profile on using /opt/maemo: there are better ways to give more space to the rootfs and we should make it easy to stop using /opt/maemo again. I really hope this is only temporary. If you want to use /opt for other reasons, for example because you like the cleanliness of storing all your files below /opt/package, then I don't want to stop you, of course. But very few packages do this in practice, and it doesn't look like a obviously good thing to me either. Executable programs and libraries _are_ in a flat namespace, and you can not avoid conflicts by storing them outside of the usual /bin, /lib etc directories. For me, it's a feature that two installed packages can't both contain libfoo.so.0, and I want the package management tools to tell me about this conflict. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Graham Cobb g+...@cobb.uk.net writes: 2) Nokia sets up the environment (in both the SDK and on the device) to include /opt directories in PATH, LD_LIBRARY_PATH, PKG_CONFIG_PATH and PYTHON_PATH. In that case, I would encourage Mer to also implement this. 3) Allow use of --prefix=/opt. Any package that wants to use that is fine. In particular, allow installation into /opt/bin, /opt/lib, etc (not just into /opt/package). Note that this is not FHS compliant (FHS would require --prefix=/opt/package). I think this is a essentialy useless exersize. Replace /opt with /usr in your text above, and you'll see that this has already been done. I don't see a need to add a third prefix into the picture. Instead, we should move /usr to a different partition if we really want to. I hope we can avoid that by moving all of / to the eMMC, but if that is not possible for whatever reason, we should move /usr. (If the eMMC is an all-around shitty place for files because it is too slow and too unreliable, then we really have a bigger problem. I hope that most of the perceived shittyness of the eMMC can actually be blamed on VFAT. On the other hand, it is likely true that the OneNAND is better in many ways, and we should make good use of it as long as we have it. I'm proposing to put swap there, and maybe /var. But really, if you have 32 GiB of storage, who cares about the 256 MiB in the corner? It's reasonable to ignore it completely.) ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Andrew Flegg and...@bleb.org writes: True. I meant nomenclature and so on. But the following is my FHS++ idea: This reminds me of http://cr.yp.to/slashpackage.html Don't know if there is anything interesting in it for us, but it's always good to listen to what djb has to say. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Gil Quim (Nokia-D/Helsinki) quim@nokia.com writes: Limiting it to a hack for a large app results in a question of when is an app large: 1MB, 2MB, 4MB, 8MB, 16MB, ...? If we can have a consensus on *that*, that could be something we have a QA check for? Developers are encouraged to make good use of them, specially for applications requiring more than 500KB, including dependencies. Well, that's just a marketing number. :-) ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, 2009-09-10 at 13:35 +0200, Vollmer Marius (Nokia-D/Helsinki)
wrote:
Hassan Mohammed.2 (Nokia-D/Helsinki) mohammed.2.has...@nokia.com
writes:
And that indication is already a packaging difference.
I'd rather do ./configure --prefix=/opt/package and that's it.
And then you have to deal with the fallout from that. If you are ready
to do that, more power to you, but I really would like to keep a very
low profile on using /opt/maemo: there are better ways to give more
space to the rootfs and we should make it easy to stop using /opt/maemo
again.
I really hope this is only temporary.
If you want to use /opt for other reasons, for example because you like
the cleanliness of storing all your files below /opt/package, then I
don't want to stop you, of course.
But very few packages do this in practice, and it doesn't look like a
obviously good thing to me either. Executable programs and libraries
_are_ in a flat namespace, and you can not avoid conflicts by storing
them outside of the usual /bin, /lib etc directories. For me, it's a
feature that two installed packages can't both contain libfoo.so.0, and
I want the package management tools to tell me about this conflict.
Well, ./configure --prefix=/opt/maemo then. I also don't like to
namespace packages unless I build them from source and I
use /usr/local/package from there but that's another issue.
I just feel that passing a prefix is a cleaner solution but that's IMHO.
By passing a prefix, you can:
1) add /opt/maemo/{bin,sbin} to $PATH
2) add /opt/maemo/lib to /etc/ld.so.conf (or /etc/ld.so.conf.d)
3) compile and install stuff and that's it.
Some applications still hardcode paths sometimes and will probably fail
but then they should be fixed. The advantage is that you get rid of the
symlinks and stuff.
Only problem is when we start getting libfoo1 and libfoo2. One of them
goes to /usr/lib and the other to /opt/maemo/lib :(
--
Maemo Software
Nokia Devices
___
maemo-developers mailing list
maemo-developers@maemo.org
https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext David Greaves da...@dgreaves.com writes: As the fiasco boots it mounts partition for /usr on /usr_old Checks /usr_old/version against /usr/version If there is a mismatch it tars up /usr/ and untars it to /usr_old Then remount /usr_old on /usr Yeah, that _could_ work, technically, but I am afraid it is still a lot more fragile than we can afford at this point. Also, it will take a looong time during the first boot and you can't really do anything in parallel. That's quite a high cost, and I don't think it is acceptable. We are pretty paranoid about boot time here. We could make this is an option, hiding it away somewhere in the Control Panel, say. Hitting the button would boot into a special mode and migrate /usr to the eMMC. But, again, too invasive to get this done before Fremantle has to ship, I would say. The beauty of modifiying packages in the Maemo Extras and elsewhere is that we are not tied to the Fremantle release schedule. If we screw up with maemo-optify, we can quickly fix it. In the worst case, we can simply do nothing until we have something that works well enough. Fixing the Fremantle OS ones it is out is very slow compared to uploading a new version of a package to Maemo Extras. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Andrew Flegg and...@bleb.org writes: On Thu, Sep 10, 2009 at 09:48, Marius Vollmer marius.voll...@nokia.com wrote: We will get rid of this abuse of /opt as fast as we can. That changes things, if /opt is going away (in a Maemo Update?) [...] Careful. :-) The /opt directory itself is of course not becoming illegal in some way, and your optified packages will continue to work, in the same way as they will work on Diablo, Chinook, etc. The optification will become pointless, but it will be quite harmless forever (except for the forest of symlinks, but I hope that they don't cause significant cost). ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thursday 10 September 2009 13:05:35 Marius Vollmer wrote: ext Graham Cobb g+...@cobb.uk.net writes: 2) Nokia sets up the environment (in both the SDK and on the device) to include /opt directories in PATH, LD_LIBRARY_PATH, PKG_CONFIG_PATH and PYTHON_PATH. In that case, I would encourage Mer to also implement this. 3) Allow use of --prefix=/opt. Any package that wants to use that is fine. In particular, allow installation into /opt/bin, /opt/lib, etc (not just into /opt/package). Note that this is not FHS compliant (FHS would require --prefix=/opt/package). I think this is a essentialy useless exersize. Replace /opt with /usr in your text above, and you'll see that this has already been done. Exactly! My proposal was to make /opt into another /usr. I don't see a need to add a third prefix into the picture. Only that we are being told we have no choice! Instead, we should move /usr to a different partition if we really want to. I hope we can avoid that by moving all of / to the eMMC, but if that is not possible for whatever reason, we should move /usr. If this is really temporary and can/will be fixed in the first update then I agree that putting maemo-optify in debian/rules (and maemo-optify in the Build-Depends) is the right answer. There is no need to do it automatically for all packages because it only matters for larger packages and they tend to have maintainers who are more able to test and fix any obscure problems it causes. Leave the semi-automatically ported Debian packages and most small packages unchanged. If it isn't going to be fixed until Harmattan (which is presumably some time away and may not ever run on current devices such as the N900) then I think it should be less hacky than a bunch of symlinks. That is why I wanted to see /opt promoted to an equivalent of /usr. Graham ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, 2009-09-10 at 14:05 +0200, Vollmer Marius (Nokia-D/Helsinki) wrote: ext Graham Cobb g+...@cobb.uk.net writes: slow and too unreliable, then we really have a bigger problem. I hope that most of the perceived shittyness of the eMMC can actually be blamed on VFAT. On the other hand, it is likely true that the OneNAND is Actually /opt will be using ext3 (/home). They found out that FAT is faster than ext3, though, that's why the flash player uses /home/user/MyDocs for its cache files... -Kimmo ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Murray Cumming murr...@murrayc.com writes: So we can just run maemo-optify-deb on our package's debian/ directory, reupload and not worry about it again? (Assuming that maemo-optify-deb worked OK.) No, it's a bit different: you need to manually put a call to maemo-optify into the right place in debian/rules, and then re-upload. The maemo-optify-deb tool works on binary packages. The idea was to put it into the buildbot or the queue manager in front of extras-devel. That might be a bit intrusive and the current consensus seems to be that we don't use maemo-optify-deb at all. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Graham Cobb g+...@cobb.uk.net writes: If it isn't going to be fixed until Harmattan (which is presumably some time away and may not ever run on current devices such as the N900) then I think it should be less hacky than a bunch of symlinks. That is why I wanted to see /opt promoted to an equivalent of /usr. Hrrmm... I am not very optimistic about fixing this with a Fremantle update. On balance, though, I still think that the maemo-optify approach gives the most gain for the least pain, even if we are in this for a little bit longer. Your approach of turning /opt into a proper prefix alonside /, /usr, and also /usr/local is quite intrusive, I would say, as it potentially affects all packages that have standard places to search. We did this excersize early on with /var/lib/install, and while it is possible, we never were sure whether we had succeeded completely and there was always something to fix somewhere that didn't work right. It was a big relief to stop doing that and to just go with established standards. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Hamalainen Kimmo (Nokia-D/Helsinki) kimmo.hamalai...@nokia.com writes: On Thu, 2009-09-10 at 14:05 +0200, Vollmer Marius (Nokia-D/Helsinki) wrote: ext Graham Cobb g+...@cobb.uk.net writes: slow and too unreliable, then we really have a bigger problem. I hope that most of the perceived shittyness of the eMMC can actually be blamed on VFAT. On the other hand, it is likely true that the OneNAND is Actually /opt will be using ext3 (/home). They found out that FAT is faster than ext3, though, that's why the flash player uses /home/user/MyDocs for its cache files... Ohh, interesting. It's high time for some benchmarks anyway. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thursday 10 September 2009 16:42:30 Marius Vollmer wrote: Actually /opt will be using ext3 (/home). They found out that FAT is faster than ext3, though, that's why the flash player uses /home/user/MyDocs for its cache files... Ohh, interesting. It's high time for some benchmarks anyway. Not exactly a fair fight (considering ext3 journalling and all), but I'm curious, though, much does FAT performance vary (if at all) as fragmentation increases... ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Andrew Flegg and...@bleb.org writes: I've a suggestion for Marius, after some discussion on #maemo. This suggestion should make maemo-optify more compatible with how Maemo-specific applications, aware of /opt, may use it (and closer to how /opt is traditionally used in upstream Linux). Instead of using a fixed prefix of /opt/maemo/path, use /opt/package/trimmed path. Hmm. My arguments against this are: - The names /opt/package do not belong to the distribution, they are for things that are _not_ in the distribution. I.e., a Debian package called matlab has no license to install into /opt/matlab. - We are not trying to clean up the filesystem, installing into /usr/lib would be perfectly fine if only there would be enough space there. - Now, cleaning up the filesystem is a good goal, but it can be done independently from this space-saving struggle. By all means, propose and implement ways to clean up the filesystem. In my opinion, installing distribution packages into /opt/package might look clean, but it is not a good idea. That opinion is not very strong, but I don't want to make the space saving here dependent on resolving this argument. - The /opt/maemo/ location moves the whole mess cleanly out-of-sight of all the naturaly inhabitants of /opt. It's unlikely that other software wants to install into /opt/maemo. I would prefer /space over /opt/maemo, but I also don't want to stirr up the waters more by proposing yet another change. Actually, the maemo-optify scripts could be changed to use /home/maemo/ instead of /opt/maemo... hmmm they would need to be called maemo-homeify then, of course. Anyway, for me, the /opt location is a red-herring. We are not using it because we want the FHS semantics of it, we are just using it because that's the first thing that came into someones mind, the train started rolling, and I made up my mind too late about this all. - Computing the trimmed path from path is an extra complication, and we must make sure that no collisions happen. It's doable of course, but in the light of the arguments above, why bother? ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, Sep 10, 2009 at 16:12, Marius Vollmer marius.voll...@nokia.com wrote: ext Andrew Flegg and...@bleb.org writes: Instead of using a fixed prefix of /opt/maemo/path, use /opt/package/trimmed path. [big snip] I'm not going to get into a point-by-point rebuttal of these. But installing stuff in /opt on Maemo by third-parties isn't really going to happen. We own the space, pretty much everything is going to be installed from packages, and we already make all manner of assumptions in a Linux system that there's some unique UNIX name for a package. Why *not* make the one-line change to maemo-optify to make its results slightly cleaner? - Computing the trimmed path from path is an extra complication, and we must make sure that no collisions happen. It's doable of course, but in the light of the arguments above, why bother? ...because /opt is a hack because no-one at Nokia had the foresight to imagine that users might want to install multiple applications, and large new frameworks like Qt. ...because /opt is a hack which should be *embarassing*. ...because maemo-optify creating a forest of symlinks is messy, unelegant and possibly prone to failure (see my earlier question about Python modules and sub-directories of optified packages). Mainly, though, because last minute fixes shouldn't throw good design out of the window. Cheers, Andrew -- Andrew Flegg -- mailto:and...@bleb.org | http://www.bleb.org/ Maemo Community Council chair ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Andrew Flegg and...@bleb.org writes: Although a unionfs solution would be a bit more further dev on Nokia's part, it will reduce the developer complexity and gives us a real world solution now. I'm sure the community would help as well, with patching/building/testing kernel modules (once again, Nokia should realise there are clever technical people in the community who could help design an optimal (= quick good) solution if engaged at the right time). Yes, agreed. Let's give this unionfs thing a shot. I was previously arguing against it, but only as a long-term solution. As a Fremantle-only hack, it might be better than the /opt hack. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thu, Sep 10, 2009 at 16:20, Marius Vollmer marius.voll...@nokia.com wrote: ext Andrew Flegg and...@bleb.org writes: Although a unionfs solution would be a bit more further dev on Nokia's part, it will reduce the developer complexity and gives us a real world solution now. [...] Yes, agreed. Let's give this unionfs thing a shot. I was previously arguing against it, but only as a long-term solution. As a Fremantle-only hack, it might be better than the /opt hack. Cool. I should also point out that when I say a unionfs solution, I mean a union FS solution; rather than advocating unionfs over aufs (for example). Thanks to Stskeeps/Carsten for picking me up on that. Cheers, Andrew -- Andrew Flegg -- mailto:and...@bleb.org | http://www.bleb.org/ Maemo Community Council chair ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Thursday 10 September 2009 12:16:59 Marius Vollmer wrote: There is no harm in using /opt also for non-Fremantle releases. That is, if you 'optify' your package for Fremantle, you can still ship it for the older releases with the optification in it. Also, you can make it so that maemo-optify only runs in debian/rules when it is present: which maemo-optify maemo-optify Small correction: that doesn't work (because it returns an error status when maemo-optify is not present). So you need to tell make to ignore the error. In which case there is no real point with the which, it is simpler just to say: - maemo-optify If you want the build to fail if maemo-optify is present but gives an error then use: if which maemo-optify ; then maemo-optify ; fi Is the idea that the autobuilder will automatically install maemo-optify if it exists for that version or do we have to add it to our Build-Depends? If the latter then we are back to needing different source packages for different Maemo versions. By the way, I have been experimenting with maemo-optify. I think it is currently generating too many links for quite small files. I think 20K would be a better default for the size and, if feasible, I would like to see the size settable as an option on the command line, to allow the developer to tune it for their particular package. Graham ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
I recently got two devices (Novatel Mifi and Cricket A600) which are network devices but first appear as CD ROMS of all things, but contain the installer for the driver and configuration program for the platforms. These are actually more annoying since you have to eject the CD to turn on the wireless modem. You could do a tiny (maybe ro) vfat or isofs with the usb network driver, updater, backup, sync, etc. programs for both mac and windows and anything else. Plug it in and it might autorun the driver/utility install (especially if it was pretending to be a CD/ISO). Probably too late now, but there are other ways to do this. And it would be nice to scan/index the files as they get uploaded instead of the long freeze-ups while the whole filesystem is rechecked. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Considering /opt and MyDocs in your packages
Hi Maemo developers, This is an important information specially for those handling large packages. You can find an online version updated at http://wiki.maemo.org/Documentation/Maemo_5_Developer_Guide/Packaging%2C_Deploying_and_Distributing/Installing_under_opt_and_MyDocs (or http://tr.im/yeWM in short) The N900 has about 100MB of free space in the root file system partition. This is not very much and would fill up quite quickly when installing additional applications. As a workaround, the /opt directory has been linked to a different partition with more space (about 1 GB) and /home/user/MyDocs is also available in certain cases, with even more space (about 25 GB). Developers are encouraged to make good use of them, specially for applications requiring more than 500KB, including dependencies. /opt as a good alternative The /opt directory is on an ext3 partition that is permanently mounted. Users cannot mess with it by accident. Of course, not everything needs to be moved to /opt: configuration files are best left in /etc, for example, and there is no point in moving small files like the various *.desktop and *.service files. The maemo-optify tool helps developers to prepare Debian packages that use /opt. This tool moves selected files inside the package to locations under /opt, and will symbolically link from the original location to the new place of the file. An early version of the tool can be found at http://gitorious.org/maemo-af/maemo-optify - see the README file for details. [edit] Considering /home/user/MyDocs The /home/user/MyDocs directory is another candidate for large collections of data files included in applications such as game graphics or maps. Using /opt is mostly transparent to package maintainers and end-users, but /home/user/MyDocs needs more careful consideration. The /home/user/MyDocs partition uses VFAT as its filesystem (which has some limitations compared to ext3) and it is removable: it will be unmounted and exported over USB when the user plugs in the cable. Also, /home/user/MyDocs is visible in the File manager. This all means: * You need to be aware that VFAT isn't really a POSIX filesystem. Things like symbolic links, permission bits, etc wont work as nicely as they do on a real POSIX filesystem. * You should be prepared that your files aren't there at all while the partition is unmounted. You should also not prevent the partition from being unmounted because you keep some of the files open. * User might see your files, could get confused and try to delete or rename them. In summary, you can't really put programs, libraries or theme graphics into /home/user/MyDocs. Instead, use /opt for these. -- Quim Gil open source advocate Maemo Devices @ Nokia ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
That's good news. I suppose that for /home/user/MyDocs is also available in certain cases you mean what you specifiy in more details later on the mail, that is that the directory may not be available when the tablet is connected via USB and the partition is made available to the PC/master tablet (if USB2Go works)? Aniello 2009/9/9 Quim Gil quim@nokia.com: Hi Maemo developers, This is an important information specially for those handling large packages. You can find an online version updated at http://wiki.maemo.org/Documentation/Maemo_5_Developer_Guide/Packaging%2C_Deploying_and_Distributing/Installing_under_opt_and_MyDocs (or http://tr.im/yeWM in short) The N900 has about 100MB of free space in the root file system partition. This is not very much and would fill up quite quickly when installing additional applications. As a workaround, the /opt directory has been linked to a different partition with more space (about 1 GB) and /home/user/MyDocs is also available in certain cases, with even more space (about 25 GB). Developers are encouraged to make good use of them, specially for applications requiring more than 500KB, including dependencies. /opt as a good alternative The /opt directory is on an ext3 partition that is permanently mounted. Users cannot mess with it by accident. Of course, not everything needs to be moved to /opt: configuration files are best left in /etc, for example, and there is no point in moving small files like the various *.desktop and *.service files. The maemo-optify tool helps developers to prepare Debian packages that use /opt. This tool moves selected files inside the package to locations under /opt, and will symbolically link from the original location to the new place of the file. An early version of the tool can be found at http://gitorious.org/maemo-af/maemo-optify - see the README file for details. [edit] Considering /home/user/MyDocs The /home/user/MyDocs directory is another candidate for large collections of data files included in applications such as game graphics or maps. Using /opt is mostly transparent to package maintainers and end-users, but /home/user/MyDocs needs more careful consideration. The /home/user/MyDocs partition uses VFAT as its filesystem (which has some limitations compared to ext3) and it is removable: it will be unmounted and exported over USB when the user plugs in the cable. Also, /home/user/MyDocs is visible in the File manager. This all means: * You need to be aware that VFAT isn't really a POSIX filesystem. Things like symbolic links, permission bits, etc wont work as nicely as they do on a real POSIX filesystem. * You should be prepared that your files aren't there at all while the partition is unmounted. You should also not prevent the partition from being unmounted because you keep some of the files open. * User might see your files, could get confused and try to delete or rename them. In summary, you can't really put programs, libraries or theme graphics into /home/user/MyDocs. Instead, use /opt for these. -- Quim Gil open source advocate Maemo Devices @ Nokia ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers -- -- anidel Sent from London, Eng, United Kingdom ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Wednesday 09 September 2009 13:25:49 Quim Gil wrote: The N900 has about 100MB of free space in the root file system partition. This is not very much and would fill up quite quickly when installing additional applications. As a workaround, the /opt directory has been linked to a different partition with more space (about 1 GB) and /home/user/MyDocs is also available in certain cases, with even more space (about 25 GB). Developers are encouraged to make good use of them, specially for applications requiring more than 500KB, including dependencies. Can we get some info on how this relates to backups ? E.g. if someone does a backup, what of these gets backed up (if any), or if we can somehow signal that some directories should be included in the backup ? ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Quim Gil wrote: Hi Maemo developers, This is an important information specially for those handling large packages. You can find an online version updated at http://wiki.maemo.org/Documentation/Maemo_5_Developer_Guide/Packaging%2C_Deploying_and_Distributing/Installing_under_opt_and_MyDocs (or http://tr.im/yeWM in short) The N900 has about 100MB of free space in the root file system partition. This is not very much and would fill up quite quickly when installing additional applications. As a workaround, the /opt directory has been linked to a different partition with more space (about 1 GB) and /home/user/MyDocs is also available in certain cases, with even more space (about 25 GB). Developers are encouraged to make good use of them, specially for applications requiring more than 500KB, including dependencies. /opt as a good alternative Hmm, seems like another solution would be to have the opt partition mounted as /usr and install all the 'standard' stuff into /root_usr/ and preinstall symlinks into /usr - /root_usr You could have the root partition's /usr full of symlinks to /root_usr/ and then it works without /usr mounted and when it is overmounted by the other partition it wouldn't change. Then /usr would be much like Debian upstream - the place to install larger data files David -- Don't worry, you'll be fine; I saw it work in a cartoon once... ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Wednesday 09 September 2009 13:25:49 Quim Gil wrote: installing additional applications. As a workaround, the /opt directory has been linked to a different partition with more space (about 1 GB) And, forgot to ask, is the 1GB size configurable, or is it hardcoded ? I guess could be grown with resize2fs but the bigger issue is changing the VFAT partition in that case. I'm a bit concerned with limiting something like this to 1GB for two at least two reasons: - if we expect an influx of large, more complex games and apps with a lot of data files, 1 GB is not nearly as much as it was on Diablo - 1GB also can be easily filled if the unit is used for development (compiling Qt can surprise you sometimes :) ). I understand in most cases you can use the VFAT partition for that, but there are caveats you yourself mentioned in your post. Case in point - df tells me I already use 1.4 GB for / on my Diablo setup (cloned to internal mmc). ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Hi, Am Mittwoch, den 09.09.2009, 14:25 +0300 schrieb Quim Gil: As a workaround, the /opt directory has been linked to a different partition with more space (about 1 GB) [snipped] In summary, you can't really put programs, libraries or theme graphics into /home/user/MyDocs. Instead, use /opt for these. 1 GB might be a bit limiting in that case, no? Perhaps 2 GB would be more appropiate. At least that would allow the /usr/share of my box to fit into it (yeah, comparing apples with oranges ... maybe). regards, Michael ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Wed, 2009-09-09 at 14:07 +0200, ext David Greaves wrote: ... Hmm, seems like another solution would be to have the opt partition mounted as /usr and install all the 'standard' stuff into /root_usr/ and preinstall symlinks into /usr - /root_usr This was, of course, considered as well, but the problem is that concurrent I/O to/from the eMMC can (for reasons kernel guys know) cause high load in the system. Hence eMMC is avoided for system libraries and stuff, because it would cause a performance problem when these libraries are used and e.g. swap is active. There are other technical problems with eMMC as well but it should work for this application use case pretty well. -Kimmo ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
ext Attila Csipa ma...@csipa.in.rs writes: And, forgot to ask, is the 1GB size configurable, or is it hardcoded? It will be hardcoded, but I think it is still negotiable. The partition itself is actually 2 GB, but it is also used for the Meta Tracker database and other things. What about making it 4 GB? Would that feel big enough? ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Wed, Sep 9, 2009 at 14:07, Marius Vollmermarius.voll...@nokia.com wrote: ext Attila Csipa ma...@csipa.in.rs writes: And, forgot to ask, is the 1GB size configurable, or is it hardcoded? It will be hardcoded, but I think it is still negotiable. The partition itself is actually 2 GB, but it is also used for the Meta Tracker database and other things. What about making it 4 GB? Would that feel big enough? As it gets bigger, the usable space for a user's music collection gets smaller (AIUI). If the N900 is advertised as being 32GB, but then a user finds they can only store 25GB of music, they're going to be understandably upset. 1-2GB sounds plenty for the typical use case. Large graphics/cut scenes/additional data for games could be put in /home/user/MyDocs/.data/_package_. This wouldn't show up in the file manager, and Tracker shouldn't be indexing stuff in *random* .something directories (apart from the known i18n'ed ones) Cheers, Andrew -- Andrew Flegg -- mailto:and...@bleb.org | http://www.bleb.org/ Maemo Community Council chair ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Wednesday 09 September 2009 13:20:48 Attila Csipa wrote: - 1GB also can be easily filled if the unit is used for development (compiling Qt can surprise you sometimes :) ). I understand in most cases you can use the VFAT partition for that, but there are caveats you yourself mentioned in your post. I wouldn't expect this use case to be a problem -- developers can create a new ext2/3 partition on a removeable memory card. I already do that on my N810 and I am looking forward to the N900 being an excuse to get a new, even larger memory card! Graham ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Wed, 2009-09-09 at 15:11 +0200, ext Andrew Flegg wrote: On Wed, Sep 9, 2009 at 14:07, Marius Vollmermarius.voll...@nokia.com wrote: ext Attila Csipa ma...@csipa.in.rs writes: And, forgot to ask, is the 1GB size configurable, or is it hardcoded? It will be hardcoded, but I think it is still negotiable. The partition itself is actually 2 GB, but it is also used for the Meta Tracker database and other things. What about making it 4 GB? Would that feel big enough? As it gets bigger, the usable space for a user's music collection gets smaller (AIUI). If the N900 is advertised as being 32GB, but then a user finds they can only store 25GB of music, they're going to be understandably upset. Yes, could be too late to change the partition layout because of the marketing etc. bullshit reasons. But that does not stop you from repartitioning eMMC later (as long as the order of partitions remains the same). -Kimmo ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Marius Vollmer wrote: It will be hardcoded, but I think it is still negotiable. The partition itself is actually 2 GB, but it is also used for the Meta Tracker database and other things. What about making it 4 GB? Would that feel big enough? 2 GB, 4 GB... I think there is always an issue when it is hard coded. Why is the ancient VFAT and fixed partitioning still used? Would it be possible to partition eMMC into one big ext3 partition and just use some kind of loopdevice or similar when exposing a part of it as an USB storage in VFAT format? That way also the annoying not mounted right now issue would be fixed, since an USB host and the device could use the same files at the same time. I do not see technical limits, but maybe someone should just code a relevant kernel module (the virtual VFAT loopdevice ;) if that does not exist. BR, Henrik -- Henrik Hedberg - http://www.henrikhedberg.net/ ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
2009/9/9 Kimmo Hämäläinen kimmo.hamalai...@nokia.com: Yes, could be too late to change the partition layout because of the marketing etc. bullshit reasons. But that does not stop you from repartitioning eMMC later (as long as the order of partitions remains the same). Well, I don't want a 4GB /opt so if someone does, they can repartition as you say. 1GB sounds about the right point for now. Is there already a gparted port to Maemo; if not - should be straightforward given it's ext2 rather than ubifs/jffs2. Cheers, Andrew -- Andrew Flegg -- mailto:and...@bleb.org | http://www.bleb.org/ Maemo Community Council chair ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On ons, 2009-09-09 at 15:20 +0200, ext Henrik Hedberg wrote: Marius Vollmer wrote: It will be hardcoded, but I think it is still negotiable. The partition itself is actually 2 GB, but it is also used for the Meta Tracker database and other things. What about making it 4 GB? Would that feel big enough? 2 GB, 4 GB... I think there is always an issue when it is hard coded. Why is the ancient VFAT and fixed partitioning still used? Would it be possible to partition eMMC into one big ext3 partition and just use some kind of loopdevice or similar when exposing a part of it as an USB storage in VFAT format? That way also the annoying not mounted right now issue would be fixed, since an USB host and the device could use the same files at the same time. I do not see technical limits, but maybe someone should just code a relevant kernel module (the virtual VFAT loopdevice ;) if that does not exist. Patches happily accepted! regards: David ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
2009/9/9 Andrew Flegg and...@bleb.org: 2009/9/9 Kimmo Hämäläinen kimmo.hamalai...@nokia.com: Yes, could be too late to change the partition layout because of the marketing etc. bullshit reasons. But that does not stop you from repartitioning eMMC later (as long as the order of partitions remains the same). Well, I don't want a 4GB /opt so if someone does, they can repartition as you say. 1GB sounds about the right point for now. Is there already a gparted port to Maemo; if not - should be straightforward given it's ext2 rather than ubifs/jffs2. Cheers, Andrew I do agree, most people will be more than happy with the current partitioning. I for one don't know what to do with all that space anyway (I'll have 40Gb with an 8Gb microSD I already own) and I will repartition it to allow much more space to /opt and / But this will be me. -- anidel Sent from London, Eng, United Kingdom ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
On Wed, 2009-09-09 at 15:20 +0200, ext Henrik Hedberg wrote: Marius Vollmer wrote: It will be hardcoded, but I think it is still negotiable. The partition itself is actually 2 GB, but it is also used for the Meta Tracker database and other things. What about making it 4 GB? Would that feel big enough? 2 GB, 4 GB... I think there is always an issue when it is hard coded. Why is the ancient VFAT and fixed partitioning still used? Would it VFAT is used because of MS Windows support. be possible to partition eMMC into one big ext3 partition and just use some kind of loopdevice or similar when exposing a part of it as an USB storage in VFAT format? That way also the annoying not mounted right now issue would be fixed, since an USB host and the device could use Yes, these would be really nice if there would be such translation software written and tested by someone. It would not be trivial task, perhaps it's easier to use some network protocol, or make ext3 support for Windows :) -Kimmo the same files at the same time. I do not see technical limits, but maybe someone should just code a relevant kernel module (the virtual VFAT loopdevice ;) if that does not exist. BR, Henrik ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
Hamalainen Kimmo (Nokia-D/Helsinki) kimmo.hamalai...@nokia.com writes: It would not be trivial task, perhaps it's easier to use some network protocol, or make ext3 support for Windows :) A file-based protocol is the way to go. With ext3, we would still need to unmount the partition to export it, which isn't nice. ___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
Re: Considering /opt and MyDocs in your packages
:2009-09-09T17:11:Marius Vollmer: Hamalainen Kimmo (Nokia-D/Helsinki) kimmo.hamalai...@nokia.com writes: It would not be trivial task, perhaps it's easier to use some network protocol, or make ext3 support for Windows :) A file-based protocol is the way to go. With ext3, we would still need to unmount the partition to export it, which isn't nice. First time around and not really a dev just a user but: usbnet + scp as an option(winscp?)? there is an ext2/3 driver for windows... a few of them at that as well so maybe use one of those? Just my 0.02€ -- Andraž ruskie Levstik Source Mage GNU/Linux Games/Xorg grimoire guru Re-Alpine Coordinator http://sourceforge.net/projects/re-alpine/ Geek/Hacker/Tinker Quis custodiet ipsos custodes?___ maemo-developers mailing list maemo-developers@maemo.org https://lists.maemo.org/mailman/listinfo/maemo-developers
