Re: QtMoko Virtual Memory
Sure. Let me dig them up again... -Sean -Original Message- From: Nashvin Gangaram nashv...@gmail.com Sender: community-boun...@lists.openmoko.org Date: Wed, 1 Sep 2010 10:50:56 To: Openmoko_Communitycommunity@lists.openmoko.org Reply-To: List for Openmoko community discussion community@lists.openmoko.org Subject: QtMoko Virtual Memory ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
QtMoko Virtual Memory
Hi I was just experimenting the other day, trying to configure my Neo Freerunner with QtMoko v24 to use Virtual Memory. I created a swap partition on my MicroSDHC Card, using GParted on PC. I notice than on startup, there is something like a Activating Swap Configuration... failed message. So, I just had a couple of questions: - Is it possible for QtMoko to use a swap partition on the SD Card? - How can this be done? - Does swap cause a significant performance increase with QtMoko? Kind Regards, Nashvin ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: QtMoko Virtual Memory
Dnia 2010-09-01, śro o godzinie 10:50 +0200, Nashvin Gangaram pisze: Hi Hi, this question should go to supp...@lists.openmoko.org I was just experimenting the other day, trying to configure my Neo Freerunner with QtMoko v24 to use Virtual Memory. I created a swap partition on my MicroSDHC Card, using GParted on PC. I notice than on startup, there is something like a Activating Swap Configuration... failed message. So, I just had a couple of questions: * Is it possible for QtMoko to use a swap partition on the SD Card? IIRC, yes * How can this be done? AFAIR, you can set this in /etc/fstab as /dev/mmcblk0p4 * Does swap cause a significant performance increase with QtMoko? IMHO, just a little performance gain. If you use logging, set /var/log to your uSD. -- Patryk LeadMan Benderz Linux Registered User #377521 () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments Email secured by Check Point ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: QtMoko Virtual Memory
On 9/1/10, Nashvin Gangaram nashv...@gmail.com wrote: Hi I was just experimenting the other day, trying to configure my Neo Freerunner with QtMoko v24 to use Virtual Memory. I created a swap partition on my MicroSDHC Card, using GParted on PC. I notice than on startup, there is something like a Activating Swap Configuration... failed message. So, I just had a couple of questions: - Is it possible for QtMoko to use a swap partition on the SD Card? - How can this be done? - Does swap cause a significant performance increase with QtMoko? Hi, the message is displayed because the default setting in the /etc/fstab is probably a swap partition that does not match the partition you created (edit this file to change the settings in order to match your partitions). Personally I just needed swap sometimes when installing with opkg some packages because memory was not enough. In terms of speed that won't change a lot. So it will just prevent the case that the system is running out of memory. Personally, I use a swap file instead of a swap partition on the sd card. Using swap may however reduce the lifetime of our sd card. Regards, Michele ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: QtMoko Virtual Memory
On Miércoles, 1 de Septiembre de 2010 11:22:45 Gennady Kupava escribió: В Срд, 01/09/2010 в 10:50 +0200, Nashvin Gangaram пишет: * Does swap cause a significant performance increase with QtMoko? No, i think it may cause significant performance decrease instead. But this just educated guess and it would be better to do some testing to prove this. It's not true for me, and it's a common misunderstanding IMHO. For testing, simply try to download tiles at zoom level 11, for the upper 6 zoom levels in tangoGPS. Better if you make it twice (moving on the map) Without swap, it will catch all available memory, and FR will get really slow, to the limit of appearing to hang, and even sometimes oomkill will start killing some random process. With swap, linux can swap unused pages (other daemon pages, not tangogps ones) and tangogps will continue running, and FR will be responsible. You only will notice some 1-4 seconds slowdown from time to time, when pages are swapped out swap is not only for creating more memory. If FR starts to massively trashing pages to swap, it will really SLOW things a lot, for sure (the same is true for your PC) But it will help *a lot* to have more memory available for running apps. Think on swap as a place where put unused memory pages, and use real RAM for currently used apps or caching files from slow uSD Swap will *ALLWAYS* help, but will help a lot more on a limited memory device, as FR Here [1] you can read more about what I'm saying. [1] http://kerneltrap.org/node/3202 signature.asc Description: This is a digitally signed message part. ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: QtMoko Virtual Memory
В Срд, 01/09/2010 в 12:38 +0200, David Garabana Barro пишет: It's not true for me, and it's a common misunderstanding IMHO. This is not misunderstanding, this is just one of questions which can't have only one answer. For some tasks swap is good, for others - bad. I have strong (enough for me) arguments too. Yours are strong for you, so this is just matter of situation. For testing, simply try to download tiles at zoom level 11, for the upper 6 zoom levels in tangoGPS. Better if you make it twice (moving on the map) Without swap, it will catch all available memory, and FR will get really slow, to the limit of appearing to hang, and even sometimes oomkill will start killing some random process. With swap, linux can swap unused pages (other daemon pages, not tangogps ones) and tangogps will continue running, and FR will be responsible. You only will notice some 1-4 seconds slowdown from time to time, when pages are swapped out swap is not only for creating more memory. If FR starts to massively trashing pages to swap, it will really SLOW things a lot, for sure (the same is true for your PC) But it will help *a lot* to have more memory available for running apps. Think on swap as a place where put unused memory pages, and use real RAM for currently used apps or caching files from slow uSD Swap will *ALLWAYS* help, but will help a lot more on a limited memory device, as FR Here [1] you can read more about what I'm saying. [1] http://kerneltrap.org/node/3202 Thanks for description, now my position: First of all, i do not use swap on any Linux system i am using with amount of memory =512Mb. I even do not use swap on desktop where i have tmpfs mounted to /tmp. Having something published on kerneltrap is not meant to be only possible answer, i can provide an example. Few years ago Linus believed that moving everything to userspace is good idea and this is way to go. Now i see everything is included to kernel (devfs vs udev, evdev vs tslib, kms vs userspace mode switching). But i am still at the point that every possible thing should be in userspace. I think, both points of view very extensivly published to mailing lists and both have strong grounds :) First, about swap in general. IMO, swap were introduced in absolutely different context. In 80's memory situation were completely different. --- now, real situations. 1. main swap problem is in it's nature. it will push to disk less-frequently used pages from memory but use 'freed memory'. But for me it turned out that it is impossible to predict which page is useful which is not _in future_, and it turns out that freed memory on all modern systems used to keep relatively useless huge disk cache. it's question (for me) is _huge_ disk cache is better than _meduim-sized_ disk cache in many situations. note that I found that using tmpfs is times faster than using such 'buffer' (i tested qtmoko build). having random process swapped-out also makes system very unpredictive. you may never know how long some application will start, this annoying for me, i like low latency. 1a. to apply (1) to FR. imagine you have phone app in background. it got swapped out as you did last call 8 hours ago. now, you recieve call and what? you should wait for your app to be paged back even to start hearing ring! considering fr sd io speed is 2.6M/s, and your app is for example 10M, this will take 4 seconds in best case. something have to be also discarded from memory too free up that 10M before loading your phone app. 2. situation (2) is described in [1] you pointed. i personally face it tons of times - some program (last were firefox and midming commander) just run out memory due to bug or other reasons and starts incrementally requesting swap. if you unlicky (and do not kill app in 30 seconds), you may get your terminal and most of X be unloaded to swap, and only thing you can do after that is hard reset. introducing limits will kill only really useful feature of swap - being able to load something larger than memory (3) 3. about your favorite gps application. i think it should create file and map it to memory instead of using extreme amounts of ram to store all data. In older day of swap not all systems had such ability. It's strange that sometimg became 'slow'. without swap it should never become slow, it should be just oomkilled. in fact, i do not understand this problem very well, as i think that just next malloc should return 0, or new throw bad_alloc. 4. swap on nand is special story. recently i saw really interesting article on this topic (by DocScrutinizer i think). as nand has block size of 4kb, writing (and reading) scatter pages (4k) to it may be very slow process. 5. if your distribution is on internal NAND, it may be several times faster to reload application from internal flash than to swap from sd. So, i can see that i only had problems with swap and never had real _need_ to use it. And, because of all
Re: QtMoko Virtual Memory
yeah long but thx! 2010/9/1 Gennady Kupava g...@bsdmn.com В Срд, 01/09/2010 в 12:38 +0200, David Garabana Barro пишет: It's not true for me, and it's a common misunderstanding IMHO. This is not misunderstanding, this is just one of questions which can't have only one answer. For some tasks swap is good, for others - bad. I have strong (enough for me) arguments too. Yours are strong for you, so this is just matter of situation. For testing, simply try to download tiles at zoom level 11, for the upper 6 zoom levels in tangoGPS. Better if you make it twice (moving on the map) Without swap, it will catch all available memory, and FR will get really slow, to the limit of appearing to hang, and even sometimes oomkill will start killing some random process. With swap, linux can swap unused pages (other daemon pages, not tangogps ones) and tangogps will continue running, and FR will be responsible. You only will notice some 1-4 seconds slowdown from time to time, when pages are swapped out swap is not only for creating more memory. If FR starts to massively trashing pages to swap, it will really SLOW things a lot, for sure (the same is true for your PC) But it will help *a lot* to have more memory available for running apps. Think on swap as a place where put unused memory pages, and use real RAM for currently used apps or caching files from slow uSD Swap will *ALLWAYS* help, but will help a lot more on a limited memory device, as FR Here [1] you can read more about what I'm saying. [1] http://kerneltrap.org/node/3202 Thanks for description, now my position: First of all, i do not use swap on any Linux system i am using with amount of memory =512Mb. I even do not use swap on desktop where i have tmpfs mounted to /tmp. Having something published on kerneltrap is not meant to be only possible answer, i can provide an example. Few years ago Linus believed that moving everything to userspace is good idea and this is way to go. Now i see everything is included to kernel (devfs vs udev, evdev vs tslib, kms vs userspace mode switching). But i am still at the point that every possible thing should be in userspace. I think, both points of view very extensivly published to mailing lists and both have strong grounds :) First, about swap in general. IMO, swap were introduced in absolutely different context. In 80's memory situation were completely different. --- now, real situations. 1. main swap problem is in it's nature. it will push to disk less-frequently used pages from memory but use 'freed memory'. But for me it turned out that it is impossible to predict which page is useful which is not _in future_, and it turns out that freed memory on all modern systems used to keep relatively useless huge disk cache. it's question (for me) is _huge_ disk cache is better than _meduim-sized_ disk cache in many situations. note that I found that using tmpfs is times faster than using such 'buffer' (i tested qtmoko build). having random process swapped-out also makes system very unpredictive. you may never know how long some application will start, this annoying for me, i like low latency. 1a. to apply (1) to FR. imagine you have phone app in background. it got swapped out as you did last call 8 hours ago. now, you recieve call and what? you should wait for your app to be paged back even to start hearing ring! considering fr sd io speed is 2.6M/s, and your app is for example 10M, this will take 4 seconds in best case. something have to be also discarded from memory too free up that 10M before loading your phone app. 2. situation (2) is described in [1] you pointed. i personally face it tons of times - some program (last were firefox and midming commander) just run out memory due to bug or other reasons and starts incrementally requesting swap. if you unlicky (and do not kill app in 30 seconds), you may get your terminal and most of X be unloaded to swap, and only thing you can do after that is hard reset. introducing limits will kill only really useful feature of swap - being able to load something larger than memory (3) 3. about your favorite gps application. i think it should create file and map it to memory instead of using extreme amounts of ram to store all data. In older day of swap not all systems had such ability. It's strange that sometimg became 'slow'. without swap it should never become slow, it should be just oomkilled. in fact, i do not understand this problem very well, as i think that just next malloc should return 0, or new throw bad_alloc. 4. swap on nand is special story. recently i saw really interesting article on this topic (by DocScrutinizer i think). as nand has block size of 4kb, writing (and reading) scatter pages (4k) to it may be very slow process. 5. if your distribution is on internal NAND, it may be several times faster to reload application from internal flash
Re: QtMoko Virtual Memory
hi all, here I am too ;) 2010-09...@15:41 Gennady Kupava В Срд, 01/09/2010 в 12:38 +0200, David Garabana Barro пишет: whoa! a lot of stuff... interesting thread I'd like to add just my 2c: what about reducing the swappiness? since qtmoko v19 I use a swap partition (I'm not for the swapfile solution) with a vm.swappiness value = 0 I think this is ok for my freerunner use case because: 1 - I use the real ram as soon as it's free 2 - I've the swap extra memory too 3 - I'm worried about uSD usage and I'm not sure if my memory has the wear leveling feature when I check my free memory I usually see used the ram, instead the swap is fully available. (obviously things are different if I do apt-get or other jobs not typical in a mobile phone scenario and my swap is here to willing be used) kind regards! -- ALFA21 IS PROVIDED AS IS AND WITHOUT WARRANTIES OF ANY KIND, EXPRESS OR IMPLIED. ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
