Re: Why would some process swap in place of reclaiming free(cached) memory
On Wed, Feb 20, 2013 at 10:01 PM, Soham Chakraborty sohamwonderpik...@gmail.com wrote: The question is salivating and simple. When we have free and lotsa cached memory in a system (irrespective of distro and kernel), why would some process end up in swap space, I understand the overcommit mode of vm, the default value of 0 and what it does. I also know that the cache might be dirty and we don't might not want to allocate free pages thinking that we might go under the watermark of low. But, I mainly end up getting this question from end users - why would things swap if I have free memory. I support michael's answer. basically because those (anonymous) pages are inactive for quite long time. So instead of letting them sit in RAM for nothing, they are swapped out. IMHO, it is better to do this earlier than waiting for free RAM getting tighter and doing swap out right at the moment some apps hit page fault and swapping out inactive pages. If we did this way, memory allocation will go slow (maybe really slow). Any reference to source where it gives the condition to transfer some pages to swap, despite of having free memory, would be highly appreciated. i think it's in mm/vmscan.c I tried to find in mm/vmscan.c but didn't get much success. really? hmm I remember there are codes there that scan inactive pages and swap them out. -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Why would some process swap in place of reclaiming free(cached) memory
On Thu, Feb 21, 2013 at 12:19 AM, Soham Chakraborty sohamwonderpik...@gmail.com wrote: On Thu, Feb 21, 2013 at 10:45 AM, Mandeep Sandhu mandeepsandhu@gmail.com wrote: I know the pages are kept on disk initially and then swapped in as needed/referenced. Thus if there are code pages where none of the code in the page has been executed since starting the app, then that page will remain on disk until the application logic eventually invokes it. At that point it will be demand loaded from disk. During that early stage, do the never used pages show as swapped out? If so, why would anyone want all those never used pages to be brought into ram just to sit there unused? Are you saying that pages of code segment (of an executable) which have not been accessed yet, are copied onto the swap space from their location on the disk when the program is exec'ed? Or do they remain on disk (in their original location) and loaded into memory only when accessed (and _then_ possibly swapped out)? If they remain on disk, then that wouldn't show up in swap space, right? If the code segment of an executable which hasn't been accessed, they will remain on disk because they have fixed filesystem backing. Only when they are referenced, they will go to cache and then when inactive and/or while reclaiming, they will end up in disk again. But, afaik, anything which has a filesystem backing, a fixed storage backing, doesn't go to swap. This part is pretty clear to me. Only anon pages go to swap and something which is on disk, is not anon in first place. Soham, I agree with the details you describe. My question was if the pages not in RAM, but available in filesystem backing were being included in the count of pages swapped out that you were asking about. Let me rephrase my answer: Total addressable pages = Allocated RAM pages + filesystem Backing Storage pages + pages swapped out I don't see the filesystem backing storage pages typically included in performance tools like top. Therefore my question was if those userspace reporting tools use the term swap to represent both the truely swapped out pages and the filesystem backing storage pages. Seems like you know the MM system better than me, so I won't argue for or against my theory. I'll just leave it as a question. Greg ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Why would some process swap in place of reclaiming free(cached) memory
Hi all, I posted this question in linux-mm list as well but it didn't spawn much interest. So, I am putting the same question here also. Would love if someone can put some traction. The question is salivating and simple. When we have free and lotsa cached memory in a system (irrespective of distro and kernel), why would some process end up in swap space, I understand the overcommit mode of vm, the default value of 0 and what it does. I also know that the cache might be dirty and we don't might not want to allocate free pages thinking that we might go under the watermark of low. But, I mainly end up getting this question from end users - why would things swap if I have free memory. I guess, technically we can't stop swapping even if we set vm.swappiness to 0 but well, I think I have said what I need to said. If you need data, I can provide those. Any reference to source where it gives the condition to transfer some pages to swap, despite of having free memory, would be highly appreciated. I tried to find in mm/vmscan.c but didn't get much success. Soham ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Why would some process swap in place of reclaiming free(cached) memory
Hi! On 20:31 Wed 20 Feb , Soham Chakraborty wrote: Hi all, I posted this question in linux-mm list as well but it didn't spawn much interest. So, I am putting the same question here also. Would love if someone can put some traction. The question is salivating and simple. When we have free and lotsa cached memory in a system (irrespective of distro and kernel), why would some process end up in swap space, Freeing cache memory does not need to be faster than swapping. The cached files might be accessed more often than memory of some process. It is a hard balancing act. I understand the overcommit mode of vm, the default value of 0 and what it does. I also know that the cache might be dirty and we don't might not want to allocate free pages thinking that we might go under the watermark of low. But, I mainly end up getting this question from end users - why would things swap if I have free memory. I think this might indeed make sense in some cases. If there is lots of memory used which is never accessed, it could be swapped out early. This would prevent write activity if memory gets tight later. Also note that there is a entry SwapCached in /proc/meminfo. It might be possible that the pages swapped out pages are present in both swap *and* memory. Just a possibility... I guess, technically we can't stop swapping even if we set vm.swappiness to 0 but well, I think I have said what I need to said. If you need data, I can provide those. Swapping can be prevented. 1) You can disable all swap space in the system 2) Alternatively, you can call mlock() if you only care about few processes. ... -Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Why would some process swap in place of reclaiming free(cached) memory
Soham Chakraborty sohamwonderpik...@gmail.com wrote: Hi all, I posted this question in linux-mm list as well but it didn't spawn much interest. So, I am putting the same question here also. Would love if someone can put some traction. The question is salivating and simple. When we have free and lotsa cached memory in a system (irrespective of distro and kernel), why would some process end up in swap space, I understand the overcommit mode of vm, the default value of 0 and what it does. I also know that the cache might be dirty and we don't might not want to allocate free pages thinking that we might go under the watermark of low. But, I mainly end up getting this question from end users - why would things swap if I have free memory. I guess, technically we can't stop swapping even if we set vm.swappiness to 0 but well, I think I have said what I need to said. If you need data, I can provide those. Any reference to source where it gives the condition to transfer some pages to swap, despite of having free memory, would be highly appreciated. I tried to find in mm/vmscan.c but didn't get much success. Soham I am not a mm expert, but let me pose a question and let someone else answer it. What happens to the paging counts when a large program is exec'ed. I know the pages are kept on disk initially and then swapped in as needed/referenced. Thus if there are code pages where none of the code in the page has been executed since starting the app, then that page will remain on disk until the application logic eventually invokes it. At that point it will be demand loaded from disk. During that early stage, do the never used pages show as swapped out? If so, why would anyone want all those never used pages to be brought into ram just to sit there unused? The end result is that that swap'ed pages represent pages available to be swapped in, but not the number of pages that were actually swapped out at some point in time. Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Why would some process swap in place of reclaiming free(cached) memory
I know the pages are kept on disk initially and then swapped in as needed/referenced. Thus if there are code pages where none of the code in the page has been executed since starting the app, then that page will remain on disk until the application logic eventually invokes it. At that point it will be demand loaded from disk. During that early stage, do the never used pages show as swapped out? If so, why would anyone want all those never used pages to be brought into ram just to sit there unused? Are you saying that pages of code segment (of an executable) which have not been accessed yet, are copied onto the swap space from their location on the disk when the program is exec'ed? Or do they remain on disk (in their original location) and loaded into memory only when accessed (and _then_ possibly swapped out)? If they remain on disk, then that wouldn't show up in swap space, right? -mandeep The end result is that that swap'ed pages represent pages available to be swapped in, but not the number of pages that were actually swapped out at some point in time. Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Re: Why would some process swap in place of reclaiming free(cached) memory
On Thu, Feb 21, 2013 at 10:45 AM, Mandeep Sandhu mandeepsandhu@gmail.com wrote: I know the pages are kept on disk initially and then swapped in as needed/referenced. Thus if there are code pages where none of the code in the page has been executed since starting the app, then that page will remain on disk until the application logic eventually invokes it. At that point it will be demand loaded from disk. During that early stage, do the never used pages show as swapped out? If so, why would anyone want all those never used pages to be brought into ram just to sit there unused? Are you saying that pages of code segment (of an executable) which have not been accessed yet, are copied onto the swap space from their location on the disk when the program is exec'ed? Or do they remain on disk (in their original location) and loaded into memory only when accessed (and _then_ possibly swapped out)? If they remain on disk, then that wouldn't show up in swap space, right? If the code segment of an executable which hasn't been accessed, they will remain on disk because they have fixed filesystem backing. Only when they are referenced, they will go to cache and then when inactive and/or while reclaiming, they will end up in disk again. But, afaik, anything which has a filesystem backing, a fixed storage backing, doesn't go to swap. This part is pretty clear to me. Only anon pages go to swap and something which is on disk, is not anon in first place. Soham -mandeep The end result is that that swap'ed pages represent pages available to be swapped in, but not the number of pages that were actually swapped out at some point in time. Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies ___ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
