Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-21 Thread Pavel Machek
Hi! If it is only one place, why not pre-allocate one I'm sick now skb and hold onto it. Any bigger solution seems to snowball into a huge mess. But the problem is even sending/receiving a single packet can cause multiple dynamic allocations in the networking path all the way from the

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-21 Thread David Stevens
Why not do it the other way? If you don't hear from me for 2 minutes, do a switchover. Then all you have to do is _not_ to send a packet -- easier to do. Anything else seems overkill. Pavel Because in some of the scenarios, including ours, it isn't a simple

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-16 Thread Bodo Eggert
David S. Miller [EMAIL PROTECTED] wrote: The idea to mark, for example, IPSEC key management daemon's sockets as critical is flawed, because the key management daemon could hit a swap page over the iSCSI device. Don't even start with the idea to lock the IPSEC key management daemon into ram

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread David S. Miller
From: Sridhar Samudrala [EMAIL PROTECTED] Date: Wed, 14 Dec 2005 23:37:37 -0800 (PST) Instead, you seem to be suggesting in_emergency to be set dynamically when we are about to run out of ATOMIC memory. Is this right? Not when we run out, but rather when we reach some low water mark, the

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread Arjan van de Ven
On Thu, 2005-12-15 at 00:21 -0800, David S. Miller wrote: From: Sridhar Samudrala [EMAIL PROTECTED] Date: Wed, 14 Dec 2005 23:37:37 -0800 (PST) Instead, you seem to be suggesting in_emergency to be set dynamically when we are about to run out of ATOMIC memory. Is this right? Not when we

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread David Stevens
Also, all this stuff is just a band aid because linux OOM behavior is so fucked up. In our internal discussions, characterizing this as OOM came up a lot, and I don't think of it as that at all. OOM is exactly what the scheme is trying to avoid! The actual situation we have in mind is a swap

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread David Stevens
David S. Miller [EMAIL PROTECTED] wrote on 12/15/2005 12:58:05 AM: From: David Stevens [EMAIL PROTECTED] Date: Thu, 15 Dec 2005 00:44:52 -0800 In our internal discussions I really wish this hadn't been discussed internally before being implemented. Any such internal discussions are

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread James Courtier-Dutton
Mitchell Blank Jr wrote: James Courtier-Dutton wrote: When I had the conversation with Matt at KS, the problem we were trying to solve was Memory pressure with network attached swap space. s/swap space/writable filesystems/ You can hit these problems even if you have no swap. Too much of

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread Arjan van de Ven
You are using the wrong hammer to crack your nut. You should instead approach your problem of why the ARP entry gets lost. For example, you could give as critical priority to your TCP session, but that still won't cure your ARP problem. I would suggest that the best way to cure your arp

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread jamal
On Thu, 2005-15-12 at 12:47 +0100, Arjan van de Ven wrote: You are using the wrong hammer to crack your nut. You should instead approach your problem of why the ARP entry gets lost. For example, you could give as critical priority to your TCP session, but that still won't cure your ARP

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread Arjan van de Ven
On Thu, 2005-12-15 at 08:00 -0500, jamal wrote: On Thu, 2005-15-12 at 12:47 +0100, Arjan van de Ven wrote: You are using the wrong hammer to crack your nut. You should instead approach your problem of why the ARP entry gets lost. For example, you could give as critical priority to

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-15 Thread Sridhar Samudrala
On Thu, 2005-12-15 at 00:21 -0800, David S. Miller wrote: From: Sridhar Samudrala [EMAIL PROTECTED] Date: Wed, 14 Dec 2005 23:37:37 -0800 (PST) Instead, you seem to be suggesting in_emergency to be set dynamically when we are about to run out of ATOMIC memory. Is this right? Not when we

[RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Sridhar Samudrala
These set of patches provide a TCP/IP emergency communication mechanism that could be used to guarantee high priority communications over a critical socket to succeed even under very low memory conditions that last for a couple of minutes. It uses the critical page pool facility provided by

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Andi Kleen
I would appreciate any feedback or comments on this approach. Maybe I'm missing something but wouldn't you need an own critical pool (or at least reservation) for each socket to be safe against deadlocks? Otherwise if a critical sockets needs e.g. 2 pages to finish something and 2 critical

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Sridhar Samudrala
On Wed, 2005-12-14 at 10:22 +0100, Andi Kleen wrote: I would appreciate any feedback or comments on this approach. Maybe I'm missing something but wouldn't you need an own critical pool (or at least reservation) for each socket to be safe against deadlocks? Otherwise if a critical sockets

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread David Stevens
It has a lot more users that compete true, but likely the set of GFP_CRITICAL users would grow over time too and it would develop the same problem. No, because the critical set is determined by the user (by setting the socket flag). The receive side has some things marked as

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Jesper Juhl
On 12/14/05, Sridhar Samudrala [EMAIL PROTECTED] wrote: These set of patches provide a TCP/IP emergency communication mechanism that could be used to guarantee high priority communications over a critical socket to succeed even under very low memory conditions that last for a couple of

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Ben Greear
Jesper Juhl wrote: To be a little serious, it sounds like something that could be used to cause trouble and something that will lose its usefulness once enough people start using it (for valid or invalid reasons), so what's the point... It could easily be a user-configurable option in an

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread James Courtier-Dutton
Jesper Juhl wrote: On 12/14/05, Sridhar Samudrala [EMAIL PROTECTED] wrote: These set of patches provide a TCP/IP emergency communication mechanism that could be used to guarantee high priority communications over a critical socket to succeed even under very low memory conditions that last for

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Sridhar Samudrala
On Wed, 2005-12-14 at 20:49 +, James Courtier-Dutton wrote: Jesper Juhl wrote: On 12/14/05, Sridhar Samudrala [EMAIL PROTECTED] wrote: These set of patches provide a TCP/IP emergency communication mechanism that could be used to guarantee high priority communications over a critical

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread James Courtier-Dutton
Sridhar Samudrala wrote: On Wed, 2005-12-14 at 20:49 +, James Courtier-Dutton wrote: Jesper Juhl wrote: On 12/14/05, Sridhar Samudrala [EMAIL PROTECTED] wrote: These set of patches provide a TCP/IP emergency communication mechanism that could be used to guarantee high priority

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Ben Greear
James Courtier-Dutton wrote: Have you actually thought about what would happen in a real world senario? There is no real world requirement for this sort of user land feature. In memory pressure mode, you don't care about user applications. In fact, under memory pressure no user applications

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Sridhar Samudrala
On Wed, 2005-12-14 at 14:39 -0800, Ben Greear wrote: James Courtier-Dutton wrote: Have you actually thought about what would happen in a real world senario? There is no real world requirement for this sort of user land feature. In memory pressure mode, you don't care about user

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Matt Mackall
On Wed, Dec 14, 2005 at 09:55:45AM -0800, Sridhar Samudrala wrote: On Wed, 2005-12-14 at 10:22 +0100, Andi Kleen wrote: I would appreciate any feedback or comments on this approach. Maybe I'm missing something but wouldn't you need an own critical pool (or at least reservation) for each

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Matt Mackall
On Wed, Dec 14, 2005 at 08:30:23PM -0800, David S. Miller wrote: From: Matt Mackall [EMAIL PROTECTED] Date: Wed, 14 Dec 2005 19:39:37 -0800 I think we need a global receive pool and per-socket send pools. Mind telling everyone how you plan to make use of the global receive pool when the

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread David S. Miller
From: Matt Mackall [EMAIL PROTECTED] Date: Wed, 14 Dec 2005 21:02:50 -0800 There needs to be two rules: iff global memory critical flag is set - allocate from the global critical receive pool on receive - return packet to global pool if not destined for a socket with an attached send

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Andi Kleen
On Wed, Dec 14, 2005 at 08:30:23PM -0800, David S. Miller wrote: From: Matt Mackall [EMAIL PROTECTED] Date: Wed, 14 Dec 2005 19:39:37 -0800 I think we need a global receive pool and per-socket send pools. Mind telling everyone how you plan to make use of the global receive pool when the

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Nick Piggin
David S. Miller wrote: From: Matt Mackall [EMAIL PROTECTED] Date: Wed, 14 Dec 2005 21:02:50 -0800 There needs to be two rules: iff global memory critical flag is set - allocate from the global critical receive pool on receive - return packet to global pool if not destined for a socket with

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Stephen Hemminger
On Wed, 14 Dec 2005 21:23:09 -0800 (PST) David S. Miller [EMAIL PROTECTED] wrote: From: Matt Mackall [EMAIL PROTECTED] Date: Wed, 14 Dec 2005 21:02:50 -0800 There needs to be two rules: iff global memory critical flag is set - allocate from the global critical receive pool on receive

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Stephen Hemminger
On Thu, 15 Dec 2005 06:42:45 +0100 Andi Kleen [EMAIL PROTECTED] wrote: On Wed, Dec 14, 2005 at 08:30:23PM -0800, David S. Miller wrote: From: Matt Mackall [EMAIL PROTECTED] Date: Wed, 14 Dec 2005 19:39:37 -0800 I think we need a global receive pool and per-socket send pools. Mind

Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism

2005-12-14 Thread Sridhar Samudrala
On Wed, 14 Dec 2005, David S. Miller wrote: From: Matt Mackall [EMAIL PROTECTED] Date: Wed, 14 Dec 2005 19:39:37 -0800 I think we need a global receive pool and per-socket send pools. Mind telling everyone how you plan to make use of the global receive pool when the allocation happens in