Re: DOS by unprivileged user
On Mon, Apr 30, 2018 at 12:00 PM, Ferry Tothwrote: > Op woensdag 25 april 2018 16:54:59 CEST schreef Alan Cox: >> > > I think memory allocation and io waits can't be decoupled from >> > > scheduling as they are now. >> > >> > The scheduler is not decoupled from either, it is intimately involved >> > in both. However, none of the decision making smarts for either reside >> > in the scheduler, nor should they. >> >> It belongs in both. >> >> Classical Unix systems never had this problem because they respond to >> thrashing by ensuring that all processes consumed CPU and made some >> progress. Linux handles it by thrashing itself to dealth while BSD always >> handled it by moving from paging more towards swapping and behaving like >> a swap bound batch machine. >> >> Linux thrashes itself to death, the classic BSD algorithn instead throws >> fairness out of the window under extreme load to prevent it. It might take >> a few seconds but at least you will get your prompt back. >> >> Alan >> > I haven t tried BSD. > > But when I was young I allocated 10MB on a HP9000 (UX) with 1MB of RAM. > People wanted to launch me out of the window (18th floor). > > I did not want to say Unix was better, only with so much emphasis on security > I' m surprised how easy it is for a regular user to bring linux to on it s > knees. While it is true that things can be improved/tweaked for typical desktop/single user usage; this isn't really a security issue. For shared systems, there are a few ways to soft/hard limit resources: nice, *limit, cgroups, systemd limits, containers/VMs... Cheers, Miguel
Re: DOS by unprivileged user
On Mon, Apr 30, 2018 at 12:00 PM, Ferry Toth wrote: > Op woensdag 25 april 2018 16:54:59 CEST schreef Alan Cox: >> > > I think memory allocation and io waits can't be decoupled from >> > > scheduling as they are now. >> > >> > The scheduler is not decoupled from either, it is intimately involved >> > in both. However, none of the decision making smarts for either reside >> > in the scheduler, nor should they. >> >> It belongs in both. >> >> Classical Unix systems never had this problem because they respond to >> thrashing by ensuring that all processes consumed CPU and made some >> progress. Linux handles it by thrashing itself to dealth while BSD always >> handled it by moving from paging more towards swapping and behaving like >> a swap bound batch machine. >> >> Linux thrashes itself to death, the classic BSD algorithn instead throws >> fairness out of the window under extreme load to prevent it. It might take >> a few seconds but at least you will get your prompt back. >> >> Alan >> > I haven t tried BSD. > > But when I was young I allocated 10MB on a HP9000 (UX) with 1MB of RAM. > People wanted to launch me out of the window (18th floor). > > I did not want to say Unix was better, only with so much emphasis on security > I' m surprised how easy it is for a regular user to bring linux to on it s > knees. While it is true that things can be improved/tweaked for typical desktop/single user usage; this isn't really a security issue. For shared systems, there are a few ways to soft/hard limit resources: nice, *limit, cgroups, systemd limits, containers/VMs... Cheers, Miguel
Re: DOS by unprivileged user
Op woensdag 25 april 2018 16:54:59 CEST schreef Alan Cox: > > > I think memory allocation and io waits can't be decoupled from > > > scheduling as they are now. > > > > The scheduler is not decoupled from either, it is intimately involved > > in both. However, none of the decision making smarts for either reside > > in the scheduler, nor should they. > > It belongs in both. > > Classical Unix systems never had this problem because they respond to > thrashing by ensuring that all processes consumed CPU and made some > progress. Linux handles it by thrashing itself to dealth while BSD always > handled it by moving from paging more towards swapping and behaving like > a swap bound batch machine. > > Linux thrashes itself to death, the classic BSD algorithn instead throws > fairness out of the window under extreme load to prevent it. It might take > a few seconds but at least you will get your prompt back. > > Alan > I haven t tried BSD. But when I was young I allocated 10MB on a HP9000 (UX) with 1MB of RAM. People wanted to launch me out of the window (18th floor). I did not want to say Unix was better, only with so much emphasis on security I' m surprised how easy it is for a regular user to bring linux to on it s knees.
Re: DOS by unprivileged user
Op woensdag 25 april 2018 16:54:59 CEST schreef Alan Cox: > > > I think memory allocation and io waits can't be decoupled from > > > scheduling as they are now. > > > > The scheduler is not decoupled from either, it is intimately involved > > in both. However, none of the decision making smarts for either reside > > in the scheduler, nor should they. > > It belongs in both. > > Classical Unix systems never had this problem because they respond to > thrashing by ensuring that all processes consumed CPU and made some > progress. Linux handles it by thrashing itself to dealth while BSD always > handled it by moving from paging more towards swapping and behaving like > a swap bound batch machine. > > Linux thrashes itself to death, the classic BSD algorithn instead throws > fairness out of the window under extreme load to prevent it. It might take > a few seconds but at least you will get your prompt back. > > Alan > I haven t tried BSD. But when I was young I allocated 10MB on a HP9000 (UX) with 1MB of RAM. People wanted to launch me out of the window (18th floor). I did not want to say Unix was better, only with so much emphasis on security I' m surprised how easy it is for a regular user to bring linux to on it s knees.
Re: DOS by unprivileged user
On Wed, 2018-04-25 at 15:54 +0100, Alan Cox wrote: > > Classical Unix systems never had this problem because they respond to > thrashing by ensuring that all processes consumed CPU and made some > progress. Linux handles it by thrashing itself to dealth while BSD always > handled it by moving from paging more towards swapping and behaving like > a swap bound batch machine. Memcg constrained the gitk hog nicely, forcing it do dig into swap after hitting its limit. Dunno if there are any userspace bits that can use it wisely though. -Mike
Re: DOS by unprivileged user
On Wed, 2018-04-25 at 15:54 +0100, Alan Cox wrote: > > Classical Unix systems never had this problem because they respond to > thrashing by ensuring that all processes consumed CPU and made some > progress. Linux handles it by thrashing itself to dealth while BSD always > handled it by moving from paging more towards swapping and behaving like > a swap bound batch machine. Memcg constrained the gitk hog nicely, forcing it do dig into swap after hitting its limit. Dunno if there are any userspace bits that can use it wisely though. -Mike
Re: DOS by unprivileged user
On Wed, 2018-04-25 at 15:54 +0100, Alan Cox wrote: > > > I think memory allocation and io waits can't be decoupled from > > > scheduling as they are now. > > > > The scheduler is not decoupled from either, it is intimately involved > > in both. However, none of the decision making smarts for either reside > > in the scheduler, nor should they. > > It belongs in both. If mm decision making belongs within the process scheduler, it follows that IO requests, dirty page writeback etc. do as well. Nope, I don't think we want to create a squid-uler, with tentacles extending all over the dang kernel. The thrashing problem could use some attention, but we'll have to agree to disagree about the scheduler growing mm, io (etc) smarts. -Mike
Re: DOS by unprivileged user
On Wed, 2018-04-25 at 15:54 +0100, Alan Cox wrote: > > > I think memory allocation and io waits can't be decoupled from > > > scheduling as they are now. > > > > The scheduler is not decoupled from either, it is intimately involved > > in both. However, none of the decision making smarts for either reside > > in the scheduler, nor should they. > > It belongs in both. If mm decision making belongs within the process scheduler, it follows that IO requests, dirty page writeback etc. do as well. Nope, I don't think we want to create a squid-uler, with tentacles extending all over the dang kernel. The thrashing problem could use some attention, but we'll have to agree to disagree about the scheduler growing mm, io (etc) smarts. -Mike
Re: DOS by unprivileged user
> > I think memory allocation and io waits can't be decoupled from > > scheduling as they are now. > > The scheduler is not decoupled from either, it is intimately involved > in both. However, none of the decision making smarts for either reside > in the scheduler, nor should they. It belongs in both. Classical Unix systems never had this problem because they respond to thrashing by ensuring that all processes consumed CPU and made some progress. Linux handles it by thrashing itself to dealth while BSD always handled it by moving from paging more towards swapping and behaving like a swap bound batch machine. Linux thrashes itself to death, the classic BSD algorithn instead throws fairness out of the window under extreme load to prevent it. It might take a few seconds but at least you will get your prompt back. Alan
Re: DOS by unprivileged user
> > I think memory allocation and io waits can't be decoupled from > > scheduling as they are now. > > The scheduler is not decoupled from either, it is intimately involved > in both. However, none of the decision making smarts for either reside > in the scheduler, nor should they. It belongs in both. Classical Unix systems never had this problem because they respond to thrashing by ensuring that all processes consumed CPU and made some progress. Linux handles it by thrashing itself to dealth while BSD always handled it by moving from paging more towards swapping and behaving like a swap bound batch machine. Linux thrashes itself to death, the classic BSD algorithn instead throws fairness out of the window under extreme load to prevent it. It might take a few seconds but at least you will get your prompt back. Alan
Re: DOS by unprivileged user
On Sun, 2018-04-22 at 21:37 +0200, Ferry Toth wrote: > > Yes your memory hog scenario thoroughly wrecks the user experience, but > > the process scheduler in not the source of that wreckage, it's a memory > > management issue. With no constraints in place, anybody can just keep > > on allocating until the entire system starts grinding itself to dust. > > > > -Mike > > That is exactly the issue I think. It is not just a user experience, > they is no distinction between crashing the kernel and grinding it to > dust. The effect we have is: any user on a multi user system can > crash the system. > > Memory management / constraints can't solve the problem either: it > might be intentional to alloc such amounts of memory. Constraints definitely can solve this particular problem instance. Plain old ulimit can stop the memory hog (wish) dead in its tracks, or you can use memory cgroup controller to constrain it in a more friendly manner. I started gitk in a 4G constrained cgroup, which redirected its greedy fingers to the swap bin for the remainder of its needs. But yes, there is currently no wonderful one size fits all fully automatic solution to running low on memory that doesn't involve running to the store. > I think memory allocation and io waits can't be decoupled from > scheduling as they are now. The scheduler is not decoupled from either, it is intimately involved in both. However, none of the decision making smarts for either reside in the scheduler, nor should they. -Mike
Re: DOS by unprivileged user
On Sun, 2018-04-22 at 21:37 +0200, Ferry Toth wrote: > > Yes your memory hog scenario thoroughly wrecks the user experience, but > > the process scheduler in not the source of that wreckage, it's a memory > > management issue. With no constraints in place, anybody can just keep > > on allocating until the entire system starts grinding itself to dust. > > > > -Mike > > That is exactly the issue I think. It is not just a user experience, > they is no distinction between crashing the kernel and grinding it to > dust. The effect we have is: any user on a multi user system can > crash the system. > > Memory management / constraints can't solve the problem either: it > might be intentional to alloc such amounts of memory. Constraints definitely can solve this particular problem instance. Plain old ulimit can stop the memory hog (wish) dead in its tracks, or you can use memory cgroup controller to constrain it in a more friendly manner. I started gitk in a 4G constrained cgroup, which redirected its greedy fingers to the swap bin for the remainder of its needs. But yes, there is currently no wonderful one size fits all fully automatic solution to running low on memory that doesn't involve running to the store. > I think memory allocation and io waits can't be decoupled from > scheduling as they are now. The scheduler is not decoupled from either, it is intimately involved in both. However, none of the decision making smarts for either reside in the scheduler, nor should they. -Mike
Re: DOS by unprivileged user
On Sun 2018-04-22 18:27:38, Michal Hocko wrote: > On Sun 22-04-18 10:43:00, vcap...@pengaru.com wrote: > > On Sun, Apr 22, 2018 at 12:16:54PM +0200, Pavel Machek wrote: > > > On Thu 2018-04-19 21:13:35, Ferry Toth wrote: > > > > It appears any ordinary user can easily create a DOS on linux. > > > > > > > > One sure way to reproduce this is to open gitk on the linux kernel repo > > > > (SIC) on a machine with 8GB RAM 16 GB swap on a HDD with btrfs and quad > > > > core > > > > + hyperthreading. But I will be easy enough to get the same effect with > > > > more > > > > RAM, other fs etc. > > > > > > You may want to disable swap. > > > > > > > I run without swap on my laptops, and still observe long periods of > > thrashing on the road towards OOM. What seems to occur is the active > > file-backed mappings of executables/libraries become a sort of swap > > area, repeatedly being discarded and faulted back in as the context > > switches occur. > > > > If there's any good way to prevent this, I'd like to know. > > I am afraid there is none yet. Johannes had some ground work for > page cache trashing detection > https://marc.info/?i=20170727153010.23347-1-hannes%40cmpxchg.org > but there was no version of the patchseries for quite some time and > there was no integration into the oom detection which would be > non-trivial as well. > > I realize this sucks. But the reality is that this is far from trivial > to resolve without introducing pre-mature OOM killer invocations. Another problem is that what "unusable machine" in X/web browser situation may be normal load for build server... I guess one way would be "hey, this is my X server; if it is waiting for disk for more than 10 seconds, you probably want to OOM kill someone. Ouch and same goes for my window manager". Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html signature.asc Description: Digital signature
Re: DOS by unprivileged user
On Sun 2018-04-22 18:27:38, Michal Hocko wrote: > On Sun 22-04-18 10:43:00, vcap...@pengaru.com wrote: > > On Sun, Apr 22, 2018 at 12:16:54PM +0200, Pavel Machek wrote: > > > On Thu 2018-04-19 21:13:35, Ferry Toth wrote: > > > > It appears any ordinary user can easily create a DOS on linux. > > > > > > > > One sure way to reproduce this is to open gitk on the linux kernel repo > > > > (SIC) on a machine with 8GB RAM 16 GB swap on a HDD with btrfs and quad > > > > core > > > > + hyperthreading. But I will be easy enough to get the same effect with > > > > more > > > > RAM, other fs etc. > > > > > > You may want to disable swap. > > > > > > > I run without swap on my laptops, and still observe long periods of > > thrashing on the road towards OOM. What seems to occur is the active > > file-backed mappings of executables/libraries become a sort of swap > > area, repeatedly being discarded and faulted back in as the context > > switches occur. > > > > If there's any good way to prevent this, I'd like to know. > > I am afraid there is none yet. Johannes had some ground work for > page cache trashing detection > https://marc.info/?i=20170727153010.23347-1-hannes%40cmpxchg.org > but there was no version of the patchseries for quite some time and > there was no integration into the oom detection which would be > non-trivial as well. > > I realize this sucks. But the reality is that this is far from trivial > to resolve without introducing pre-mature OOM killer invocations. Another problem is that what "unusable machine" in X/web browser situation may be normal load for build server... I guess one way would be "hey, this is my X server; if it is waiting for disk for more than 10 seconds, you probably want to OOM kill someone. Ouch and same goes for my window manager". Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html signature.asc Description: Digital signature
Re: DOS by unprivileged user
On Sun 22-04-18 10:43:00, vcap...@pengaru.com wrote: > On Sun, Apr 22, 2018 at 12:16:54PM +0200, Pavel Machek wrote: > > On Thu 2018-04-19 21:13:35, Ferry Toth wrote: > > > It appears any ordinary user can easily create a DOS on linux. > > > > > > One sure way to reproduce this is to open gitk on the linux kernel repo > > > (SIC) on a machine with 8GB RAM 16 GB swap on a HDD with btrfs and quad > > > core > > > + hyperthreading. But I will be easy enough to get the same effect with > > > more > > > RAM, other fs etc. > > > > You may want to disable swap. > > > > I run without swap on my laptops, and still observe long periods of > thrashing on the road towards OOM. What seems to occur is the active > file-backed mappings of executables/libraries become a sort of swap > area, repeatedly being discarded and faulted back in as the context > switches occur. > > If there's any good way to prevent this, I'd like to know. I am afraid there is none yet. Johannes had some ground work for page cache trashing detection https://marc.info/?i=20170727153010.23347-1-hannes%40cmpxchg.org but there was no version of the patchseries for quite some time and there was no integration into the oom detection which would be non-trivial as well. I realize this sucks. But the reality is that this is far from trivial to resolve without introducing pre-mature OOM killer invocations. -- Michal Hocko SUSE Labs
Re: DOS by unprivileged user
On Sun 22-04-18 10:43:00, vcap...@pengaru.com wrote: > On Sun, Apr 22, 2018 at 12:16:54PM +0200, Pavel Machek wrote: > > On Thu 2018-04-19 21:13:35, Ferry Toth wrote: > > > It appears any ordinary user can easily create a DOS on linux. > > > > > > One sure way to reproduce this is to open gitk on the linux kernel repo > > > (SIC) on a machine with 8GB RAM 16 GB swap on a HDD with btrfs and quad > > > core > > > + hyperthreading. But I will be easy enough to get the same effect with > > > more > > > RAM, other fs etc. > > > > You may want to disable swap. > > > > I run without swap on my laptops, and still observe long periods of > thrashing on the road towards OOM. What seems to occur is the active > file-backed mappings of executables/libraries become a sort of swap > area, repeatedly being discarded and faulted back in as the context > switches occur. > > If there's any good way to prevent this, I'd like to know. I am afraid there is none yet. Johannes had some ground work for page cache trashing detection https://marc.info/?i=20170727153010.23347-1-hannes%40cmpxchg.org but there was no version of the patchseries for quite some time and there was no integration into the oom detection which would be non-trivial as well. I realize this sucks. But the reality is that this is far from trivial to resolve without introducing pre-mature OOM killer invocations. -- Michal Hocko SUSE Labs
Re: DOS by unprivileged user
On Sun, Apr 22, 2018 at 12:16:54PM +0200, Pavel Machek wrote: > On Thu 2018-04-19 21:13:35, Ferry Toth wrote: > > It appears any ordinary user can easily create a DOS on linux. > > > > One sure way to reproduce this is to open gitk on the linux kernel repo > > (SIC) on a machine with 8GB RAM 16 GB swap on a HDD with btrfs and quad > > core > > + hyperthreading. But I will be easy enough to get the same effect with > > more > > RAM, other fs etc. > > You may want to disable swap. > I run without swap on my laptops, and still observe long periods of thrashing on the road towards OOM. What seems to occur is the active file-backed mappings of executables/libraries become a sort of swap area, repeatedly being discarded and faulted back in as the context switches occur. If there's any good way to prevent this, I'd like to know. Regards, Vito Caputo
Re: DOS by unprivileged user
On Sun, Apr 22, 2018 at 12:16:54PM +0200, Pavel Machek wrote: > On Thu 2018-04-19 21:13:35, Ferry Toth wrote: > > It appears any ordinary user can easily create a DOS on linux. > > > > One sure way to reproduce this is to open gitk on the linux kernel repo > > (SIC) on a machine with 8GB RAM 16 GB swap on a HDD with btrfs and quad > > core > > + hyperthreading. But I will be easy enough to get the same effect with > > more > > RAM, other fs etc. > > You may want to disable swap. > I run without swap on my laptops, and still observe long periods of thrashing on the road towards OOM. What seems to occur is the active file-backed mappings of executables/libraries become a sort of swap area, repeatedly being discarded and faulted back in as the context switches occur. If there's any good way to prevent this, I'd like to know. Regards, Vito Caputo
Re: DOS by unprivileged user
On Thu 2018-04-19 21:13:35, Ferry Toth wrote: > It appears any ordinary user can easily create a DOS on linux. > > One sure way to reproduce this is to open gitk on the linux kernel repo > (SIC) on a machine with 8GB RAM 16 GB swap on a HDD with btrfs and quad core > + hyperthreading. But I will be easy enough to get the same effect with more > RAM, other fs etc. You may want to disable swap. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html signature.asc Description: Digital signature
Re: DOS by unprivileged user
On Thu 2018-04-19 21:13:35, Ferry Toth wrote: > It appears any ordinary user can easily create a DOS on linux. > > One sure way to reproduce this is to open gitk on the linux kernel repo > (SIC) on a machine with 8GB RAM 16 GB swap on a HDD with btrfs and quad core > + hyperthreading. But I will be easy enough to get the same effect with more > RAM, other fs etc. You may want to disable swap. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html signature.asc Description: Digital signature
Re: DOS by unprivileged user
On Fri, 2018-04-20 at 10:39 +0200, Ferry Toth wrote: > > Nevertheless I feel one process should not be allowed to harm other > processes by denying them resources. Even if when btrfs makes it easy > abuse I think the scheduler should have throttled gitk. Memory management is not in it's job description. -Mike
Re: DOS by unprivileged user
On Fri, 2018-04-20 at 10:39 +0200, Ferry Toth wrote: > > Nevertheless I feel one process should not be allowed to harm other > processes by denying them resources. Even if when btrfs makes it easy > abuse I think the scheduler should have throttled gitk. Memory management is not in it's job description. -Mike
Re: DOS by unprivileged user
Op vrijdag 20 april 2018 06:46:58 CEST schreef Mike Galbraith: > On Thu, 2018-04-19 at 21:13 +0200, Ferry Toth wrote: > > It appears any ordinary user can easily create a DOS on linux. > > > > One sure way to reproduce this is to open gitk on the linux kernel repo > > (SIC) on a machine with 8GB RAM 16 GB swap on a HDD with btrfs and quad > > core > > + hyperthreading. But I will be easy enough to get the same effect with > > more > > RAM, other fs etc. > > > > In this case gitk allocates more and more memory (until my system freezes > > 6.5GB of 7.5GB avaiable), the system starts swapping or writing to tmp > > files > > (can't investigate as there is no time until it freezes) and the io wait > > goes to 100% on all cores. At this point it is impossible to login from > > remote and local keyboard and mouse are frozen. Hard reset is the only way > > out at this point. > > datapoint: my i4790/ext4 box running master.yesterday booted mem=8G > became highly unpleasant to use, but I retained control, and the all > cores going to 100% thing did not happen at any time. > > I didn't try constraining on the gitk user, just turned it loose a few > times to see if it managed to render box effectively dead. It failed > to kill my box, but (expectedly) did make it suck rocks. > > -Mike > Yes, might be less dramatic with ext4 than with btrfs (COW icw fsync on hdd' s destroys performance of things like virtualbox, databases, dpkg). Nevertheless I feel one process should not be allowed to harm other processes by denying them resources. Even if when btrfs makes it easy abuse I think the scheduler should have throttled gitk.
Re: DOS by unprivileged user
Op vrijdag 20 april 2018 06:46:58 CEST schreef Mike Galbraith: > On Thu, 2018-04-19 at 21:13 +0200, Ferry Toth wrote: > > It appears any ordinary user can easily create a DOS on linux. > > > > One sure way to reproduce this is to open gitk on the linux kernel repo > > (SIC) on a machine with 8GB RAM 16 GB swap on a HDD with btrfs and quad > > core > > + hyperthreading. But I will be easy enough to get the same effect with > > more > > RAM, other fs etc. > > > > In this case gitk allocates more and more memory (until my system freezes > > 6.5GB of 7.5GB avaiable), the system starts swapping or writing to tmp > > files > > (can't investigate as there is no time until it freezes) and the io wait > > goes to 100% on all cores. At this point it is impossible to login from > > remote and local keyboard and mouse are frozen. Hard reset is the only way > > out at this point. > > datapoint: my i4790/ext4 box running master.yesterday booted mem=8G > became highly unpleasant to use, but I retained control, and the all > cores going to 100% thing did not happen at any time. > > I didn't try constraining on the gitk user, just turned it loose a few > times to see if it managed to render box effectively dead. It failed > to kill my box, but (expectedly) did make it suck rocks. > > -Mike > Yes, might be less dramatic with ext4 than with btrfs (COW icw fsync on hdd' s destroys performance of things like virtualbox, databases, dpkg). Nevertheless I feel one process should not be allowed to harm other processes by denying them resources. Even if when btrfs makes it easy abuse I think the scheduler should have throttled gitk.
Re: DOS by unprivileged user
On Thu, 2018-04-19 at 21:13 +0200, Ferry Toth wrote: > It appears any ordinary user can easily create a DOS on linux. > > One sure way to reproduce this is to open gitk on the linux kernel repo > (SIC) on a machine with 8GB RAM 16 GB swap on a HDD with btrfs and quad core > + hyperthreading. But I will be easy enough to get the same effect with more > RAM, other fs etc. > > In this case gitk allocates more and more memory (until my system freezes > 6.5GB of 7.5GB avaiable), the system starts swapping or writing to tmp files > (can't investigate as there is no time until it freezes) and the io wait > goes to 100% on all cores. At this point it is impossible to login from > remote and local keyboard and mouse are frozen. Hard reset is the only way > out at this point. datapoint: my i4790/ext4 box running master.yesterday booted mem=8G became highly unpleasant to use, but I retained control, and the all cores going to 100% thing did not happen at any time. I didn't try constraining on the gitk user, just turned it loose a few times to see if it managed to render box effectively dead. It failed to kill my box, but (expectedly) did make it suck rocks. -Mike
Re: DOS by unprivileged user
On Thu, 2018-04-19 at 21:13 +0200, Ferry Toth wrote: > It appears any ordinary user can easily create a DOS on linux. > > One sure way to reproduce this is to open gitk on the linux kernel repo > (SIC) on a machine with 8GB RAM 16 GB swap on a HDD with btrfs and quad core > + hyperthreading. But I will be easy enough to get the same effect with more > RAM, other fs etc. > > In this case gitk allocates more and more memory (until my system freezes > 6.5GB of 7.5GB avaiable), the system starts swapping or writing to tmp files > (can't investigate as there is no time until it freezes) and the io wait > goes to 100% on all cores. At this point it is impossible to login from > remote and local keyboard and mouse are frozen. Hard reset is the only way > out at this point. datapoint: my i4790/ext4 box running master.yesterday booted mem=8G became highly unpleasant to use, but I retained control, and the all cores going to 100% thing did not happen at any time. I didn't try constraining on the gitk user, just turned it loose a few times to see if it managed to render box effectively dead. It failed to kill my box, but (expectedly) did make it suck rocks. -Mike
DOS by unprivileged user
It appears any ordinary user can easily create a DOS on linux. One sure way to reproduce this is to open gitk on the linux kernel repo (SIC) on a machine with 8GB RAM 16 GB swap on a HDD with btrfs and quad core + hyperthreading. But I will be easy enough to get the same effect with more RAM, other fs etc. In this case gitk allocates more and more memory (until my system freezes 6.5GB of 7.5GB avaiable), the system starts swapping or writing to tmp files (can't investigate as there is no time until it freezes) and the io wait goes to 100% on all cores. At this point it is impossible to login from remote and local keyboard and mouse are frozen. Hard reset is the only way out at this point. IMHO there is something wrong in how the kernel hands out resources, in this case memory, CPU time and disk accesses. It should be easily measurable that a single application is allocating all memory and consequently all CPU time is spent in io wait. I'm pretty sure that if the kernel would stop or reduce allocating CPU time to the hogging (user) process causing excessive io wait, the machine would continue just fine. And probably a single process should not even be allowed to allocate this amount of memory. Or at least should not force other applications to get swapped out. In effect gitk should have beeb forced to run slow or be killed off (which doesn't happen because probably the kernel itself doesn't receive enough CPU time). As a positive side effect maybe someone would dig into gitk and make it less memory hungry (qgit is half as hungry, still a lot). -- Ferry Toth
DOS by unprivileged user
It appears any ordinary user can easily create a DOS on linux. One sure way to reproduce this is to open gitk on the linux kernel repo (SIC) on a machine with 8GB RAM 16 GB swap on a HDD with btrfs and quad core + hyperthreading. But I will be easy enough to get the same effect with more RAM, other fs etc. In this case gitk allocates more and more memory (until my system freezes 6.5GB of 7.5GB avaiable), the system starts swapping or writing to tmp files (can't investigate as there is no time until it freezes) and the io wait goes to 100% on all cores. At this point it is impossible to login from remote and local keyboard and mouse are frozen. Hard reset is the only way out at this point. IMHO there is something wrong in how the kernel hands out resources, in this case memory, CPU time and disk accesses. It should be easily measurable that a single application is allocating all memory and consequently all CPU time is spent in io wait. I'm pretty sure that if the kernel would stop or reduce allocating CPU time to the hogging (user) process causing excessive io wait, the machine would continue just fine. And probably a single process should not even be allowed to allocate this amount of memory. Or at least should not force other applications to get swapped out. In effect gitk should have beeb forced to run slow or be killed off (which doesn't happen because probably the kernel itself doesn't receive enough CPU time). As a positive side effect maybe someone would dig into gitk and make it less memory hungry (qgit is half as hungry, still a lot). -- Ferry Toth