Re: QtMoko Virtual Memory

2010-09-16 Thread Sean Moss-Pultz
Sure. Let me dig them up again... 


  -Sean

-Original Message-
From: Nashvin Gangaram nashv...@gmail.com
Sender: community-boun...@lists.openmoko.org
Date: Wed, 1 Sep 2010 10:50:56 
To: Openmoko_Communitycommunity@lists.openmoko.org
Reply-To: List for Openmoko community discussion community@lists.openmoko.org
Subject: QtMoko Virtual Memory

___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community



___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community


Re: QtMoko Virtual Memory

2010-09-01 Thread Patryk Benderz
Dnia 2010-09-01, śro o godzinie 10:50 +0200, Nashvin Gangaram pisze:
 Hi
Hi, this question should go to supp...@lists.openmoko.org

 I was just experimenting the other day, trying to configure my Neo
 Freerunner with QtMoko v24 to use Virtual Memory.  I created a swap
 partition on my MicroSDHC Card, using GParted on PC.  I notice than on
 startup, there is something like a Activating Swap Configuration...
 failed message.  So, I just had a couple of questions:
   * Is it possible for QtMoko to use a swap partition on the SD
 Card?
IIRC, yes

   * How can this be done?
AFAIR, you can set this in /etc/fstab as /dev/mmcblk0p4

   * Does swap cause a significant performance increase with
 QtMoko?
IMHO, just a little performance gain. If you use logging, set /var/log
to your uSD.

-- 
Patryk LeadMan Benderz
Linux Registered User #377521
()  ascii ribbon campaign - against html e-mail 
/\  www.asciiribbon.org   - against proprietary attachments


Email secured by Check Point

___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community


Re: QtMoko Virtual Memory

2010-09-01 Thread Michele Brocco
On 9/1/10, Nashvin Gangaram nashv...@gmail.com wrote:
 Hi

 I was just experimenting the other day, trying to configure my Neo
 Freerunner with QtMoko v24 to use Virtual Memory.  I created a swap
 partition on my MicroSDHC Card, using GParted on PC.  I notice than on
 startup, there is something like a Activating Swap Configuration... failed
 message.  So, I just had a couple of questions:

- Is it possible for QtMoko to use a swap partition on the SD Card?
- How can this be done?
- Does swap cause a significant performance increase with QtMoko?

Hi,

the message is displayed because the default setting in the /etc/fstab
is probably a swap partition that does not match the partition you
created (edit this file to change the settings in order to match your
partitions). Personally I just needed swap sometimes when installing
with opkg some packages because memory was not enough. In terms of
speed that won't change a lot. So it will just prevent the case that
the system is running out of memory. Personally, I use a swap file
instead of a swap partition on the sd card. Using swap may however
reduce the lifetime of our sd card.

Regards,
Michele

___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community


Re: QtMoko Virtual Memory

2010-09-01 Thread David Garabana Barro
On Miércoles, 1 de Septiembre de 2010 11:22:45 Gennady Kupava escribió:
 В Срд, 01/09/2010 в 10:50 +0200, Nashvin Gangaram пишет:
 
* Does swap cause a significant performance increase with

  QtMoko?
 
 No, i think it may cause significant performance decrease instead. But
 this just educated guess and it would be better to do some testing to
 prove this.

It's not true for me, and it's a common misunderstanding IMHO. 

For testing, simply try to download tiles at zoom level 11, for the upper 6 
zoom levels in tangoGPS. Better if you make it twice (moving on the map)

Without swap, it will catch all available memory, and FR will get really 
slow, to the limit of appearing to hang, and even sometimes oomkill will 
start killing some random process.
With swap, linux can swap unused pages (other daemon pages, not tangogps ones) 
and tangogps will continue running, and FR will be responsible. You only will 
notice some 1-4 seconds slowdown from time to time, when pages are swapped out

swap is not only for creating more memory. If FR starts to massively trashing 
pages to swap, it will really SLOW things a lot, for sure (the same is true for 
your PC)

But it will help *a lot* to have more memory available for running apps.
Think on swap as a place where put unused memory pages, and use real RAM for 
currently used apps or caching files from slow uSD

Swap will *ALLWAYS* help, but will help a lot more on a limited memory device, 
as FR

Here [1] you can read more about what I'm saying.

[1] http://kerneltrap.org/node/3202


signature.asc
Description: This is a digitally signed message part.
___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community


Re: QtMoko Virtual Memory

2010-09-01 Thread Gennady Kupava
В Срд, 01/09/2010 в 12:38 +0200, David Garabana Barro пишет:
 
 It's not true for me, and it's a common misunderstanding IMHO. 

This is not misunderstanding, this is just one of questions which can't
have only one answer. For some tasks swap is good, for others - bad. I
have strong (enough for me) arguments too. Yours are strong for you, so
this is just matter of situation.

 
 For testing, simply try to download tiles at zoom level 11, for the upper 6 
 zoom levels in tangoGPS. Better if you make it twice (moving on the map)
 
 Without swap, it will catch all available memory, and FR will get really 
 slow, to the limit of appearing to hang, and even sometimes oomkill will 
 start killing some random process.
 With swap, linux can swap unused pages (other daemon pages, not tangogps 
 ones) 
 and tangogps will continue running, and FR will be responsible. You only will 
 notice some 1-4 seconds slowdown from time to time, when pages are swapped out
 
 swap is not only for creating more memory. If FR starts to massively 
 trashing 
 pages to swap, it will really SLOW things a lot, for sure (the same is true 
 for 
 your PC)
 
 But it will help *a lot* to have more memory available for running apps.
 Think on swap as a place where put unused memory pages, and use real RAM for 
 currently used apps or caching files from slow uSD
 
 Swap will *ALLWAYS* help, but will help a lot more on a limited memory 
 device, 
 as FR
 
 Here [1] you can read more about what I'm saying.
 
 [1] http://kerneltrap.org/node/3202

Thanks for description, now my position:

First of all, i do not use swap on any Linux system i am using with
amount of memory =512Mb. I even do not use swap on desktop where i have
tmpfs mounted to /tmp.

Having something published on kerneltrap is not meant to be only
possible answer, i can provide an example. Few years ago Linus believed
that moving everything to userspace is good idea and this is way to go.
Now i see everything is included to kernel (devfs vs udev, evdev vs
tslib, kms vs userspace mode switching). But i am still at the point
that every possible thing should be in userspace. I think, both points
of view very extensivly published to mailing lists and both have strong
grounds :)

First, about swap in general. IMO, swap were introduced in absolutely
different context. In 80's memory situation were completely different. 

---
now, real situations.

1. main swap problem is in it's nature. it will push to disk
less-frequently used pages from memory but use 'freed memory'. But for
me it turned out that it is impossible to predict which page is useful
which is not _in future_, and it turns out that freed memory on all
modern systems used to keep relatively useless huge disk cache. it's
question (for me) is _huge_ disk cache is better than _meduim-sized_
disk cache in many situations. note that I found that using tmpfs is
times faster than using such 'buffer' (i tested qtmoko build). having
random process swapped-out also makes system very unpredictive. you may
never know how long some application will start, this annoying for me, i
like low latency.

1a. to apply (1) to FR. imagine you have phone app in background. it got
swapped out as you did last call 8 hours ago. now, you recieve call and
what? you should wait for your app to be paged back even to start
hearing ring! considering fr sd io speed is 2.6M/s, and your app is for
example 10M, this will take 4 seconds in best case. something have to be
also discarded from memory too free up that 10M before loading your
phone app.

2. situation (2) is described in [1] you pointed. i personally face it
tons of times - some program (last were firefox and midming commander)
just run out memory due to bug or other reasons and starts incrementally
requesting swap. if you unlicky (and do not kill app in 30 seconds), you
may get your terminal and most of X be unloaded to swap, and only thing
you can do after that is hard reset. introducing limits will kill only
really useful feature of swap - being able to load something larger than
memory (3)

3. about your favorite gps application. i think it should create file
and map it to memory instead of using extreme amounts of ram to store
all data. In older day of swap not all systems had such ability. It's
strange that sometimg became 'slow'. without swap it should never become
slow, it should be just oomkilled. in fact, i do not understand this
problem very well, as i think that just next malloc should return 0, or
new throw bad_alloc.

4. swap on nand is special story. recently i saw really interesting
article on this topic (by DocScrutinizer i think). as nand has block
size of 4kb, writing (and reading) scatter pages (4k) to it may be very
slow process.

5. if your distribution is on internal NAND, it may be several times
faster to reload application from internal flash than to swap from sd.

So, i can see that i only had problems with swap and never had real
_need_ to use it.
And, because of all 

Re: QtMoko Virtual Memory

2010-09-01 Thread Sylvain Paré
yeah long but thx!

2010/9/1 Gennady Kupava g...@bsdmn.com

 В Срд, 01/09/2010 в 12:38 +0200, David Garabana Barro пишет:
 
  It's not true for me, and it's a common misunderstanding IMHO.

 This is not misunderstanding, this is just one of questions which can't
 have only one answer. For some tasks swap is good, for others - bad. I
 have strong (enough for me) arguments too. Yours are strong for you, so
 this is just matter of situation.

 
  For testing, simply try to download tiles at zoom level 11, for the upper
 6
  zoom levels in tangoGPS. Better if you make it twice (moving on the map)
 
  Without swap, it will catch all available memory, and FR will get really
  slow, to the limit of appearing to hang, and even sometimes oomkill
 will
  start killing some random process.
  With swap, linux can swap unused pages (other daemon pages, not tangogps
 ones)
  and tangogps will continue running, and FR will be responsible. You only
 will
  notice some 1-4 seconds slowdown from time to time, when pages are
 swapped out
 
  swap is not only for creating more memory. If FR starts to massively
 trashing
  pages to swap, it will really SLOW things a lot, for sure (the same is
 true for
  your PC)
 
  But it will help *a lot* to have more memory available for running apps.
  Think on swap as a place where put unused memory pages, and use real RAM
 for
  currently used apps or caching files from slow uSD
 
  Swap will *ALLWAYS* help, but will help a lot more on a limited memory
 device,
  as FR
 
  Here [1] you can read more about what I'm saying.
 
  [1] http://kerneltrap.org/node/3202

 Thanks for description, now my position:

 First of all, i do not use swap on any Linux system i am using with
 amount of memory =512Mb. I even do not use swap on desktop where i have
 tmpfs mounted to /tmp.

 Having something published on kerneltrap is not meant to be only
 possible answer, i can provide an example. Few years ago Linus believed
 that moving everything to userspace is good idea and this is way to go.
 Now i see everything is included to kernel (devfs vs udev, evdev vs
 tslib, kms vs userspace mode switching). But i am still at the point
 that every possible thing should be in userspace. I think, both points
 of view very extensivly published to mailing lists and both have strong
 grounds :)

 First, about swap in general. IMO, swap were introduced in absolutely
 different context. In 80's memory situation were completely different.

 ---
 now, real situations.

 1. main swap problem is in it's nature. it will push to disk
 less-frequently used pages from memory but use 'freed memory'. But for
 me it turned out that it is impossible to predict which page is useful
 which is not _in future_, and it turns out that freed memory on all
 modern systems used to keep relatively useless huge disk cache. it's
 question (for me) is _huge_ disk cache is better than _meduim-sized_
 disk cache in many situations. note that I found that using tmpfs is
 times faster than using such 'buffer' (i tested qtmoko build). having
 random process swapped-out also makes system very unpredictive. you may
 never know how long some application will start, this annoying for me, i
 like low latency.

 1a. to apply (1) to FR. imagine you have phone app in background. it got
 swapped out as you did last call 8 hours ago. now, you recieve call and
 what? you should wait for your app to be paged back even to start
 hearing ring! considering fr sd io speed is 2.6M/s, and your app is for
 example 10M, this will take 4 seconds in best case. something have to be
 also discarded from memory too free up that 10M before loading your
 phone app.

 2. situation (2) is described in [1] you pointed. i personally face it
 tons of times - some program (last were firefox and midming commander)
 just run out memory due to bug or other reasons and starts incrementally
 requesting swap. if you unlicky (and do not kill app in 30 seconds), you
 may get your terminal and most of X be unloaded to swap, and only thing
 you can do after that is hard reset. introducing limits will kill only
 really useful feature of swap - being able to load something larger than
 memory (3)

 3. about your favorite gps application. i think it should create file
 and map it to memory instead of using extreme amounts of ram to store
 all data. In older day of swap not all systems had such ability. It's
 strange that sometimg became 'slow'. without swap it should never become
 slow, it should be just oomkilled. in fact, i do not understand this
 problem very well, as i think that just next malloc should return 0, or
 new throw bad_alloc.

 4. swap on nand is special story. recently i saw really interesting
 article on this topic (by DocScrutinizer i think). as nand has block
 size of 4kb, writing (and reading) scatter pages (4k) to it may be very
 slow process.

 5. if your distribution is on internal NAND, it may be several times
 faster to reload application from internal flash 

Re: QtMoko Virtual Memory

2010-09-01 Thread Alfa21
hi all, here I am too ;)

2010-09...@15:41 Gennady Kupava
 В Срд, 01/09/2010 в 12:38 +0200, David Garabana Barro пишет:

whoa! a lot of stuff... interesting thread
I'd like to add just my 2c:

what about reducing the swappiness?
since qtmoko v19 I use a swap partition (I'm not for the swapfile solution) 
with a vm.swappiness value = 0

I think this is ok for my freerunner use case because:
1 - I use the real ram as soon as it's free
2 - I've the swap extra memory too
3 - I'm worried about uSD usage and I'm not sure if my memory has the wear 
leveling feature

when I check my free memory I usually see used the ram, instead the swap is 
fully available.
(obviously things are different if I do apt-get or other jobs not typical in a 
mobile phone scenario and my swap is here to willing be used)

kind regards!
-- 
ALFA21 IS PROVIDED AS IS AND WITHOUT WARRANTIES OF ANY KIND, EXPRESS OR 
IMPLIED.

___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community