Re: Scheduling in interrupt BUG. [Patch]

2001-05-14 Thread Andi Kleen

On Mon, May 14, 2001 at 08:32:33AM -0400, Michal Ostrowski wrote:
> Having looked at the code for locking sockets I am concerned that the
> locking procedures for tcp may be wrong.   __release_sock releases the
> socket spinlock before calling sk->backlog_rcv() (== tcp_v4_do_rcv),
> however the comments at the top of tcp_v4_do_rcv() assert that the
> socket's spinlock is held (which is definitely not the case).
> 
> Anybody care to comment on this?

Looks ok for me.

The user socket lock (lock.users>0) is held while __release_sock runs, 
which is also sufficient to protect it as all new packets will go into backlog.
The spinlock comment only applies to bottom halves.

-Andi

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/



Re: Scheduling in interrupt BUG. [Patch]

2001-05-14 Thread Michal Ostrowski

Marcell Gal writes:
 > Hi,
 > 
 > This patch solved the problem. Should be ready for inclusion in 2.4.
 > No more 'Scheduling in interrupt' under those conditions.
 > Thanx for the thoughts, solution and the amazing speed.
 > You guys are doing a really great job!
 > 

Alexey pointed out a much nicer/correct/elegant/efficient
solution to this problem and I think that that's the way to go.

New patch below.

Michal Ostrowski
[EMAIL PROTECTED]


--- drivers/net/pppoe.c~Tue Mar  6 22:44:35 2001
+++ drivers/net/pppoe.c Mon May 14 14:10:49 2001
@@ -5,7 +5,7 @@
  * PPPoE --- PPP over Ethernet (RFC 2516)
  *
  *
- * Version:0.6.5
+ * Version:0.6.6
  *
  * 030700 : Fixed connect logic to allow for disconnect.
  * 270700 :Fixed potential SMP problems; we must protect against
@@ -19,6 +19,7 @@
  * 051000 :Initialization cleanup.
  * 00 :Fix recvmsg.
  * 050101 :Fix PADT procesing.
+ * 140501 :Use pppoe_rcv_core to handle all backlog. (Alexey)
  *
  * Author: Michal Ostrowski <[EMAIL PROTECTED]>
  * Contributors:
@@ -376,22 +377,6 @@
return ret;
 }
 
-
-/
- *
- * Receive wrapper called in process context.
- *
- ***/
-int pppoe_backlog_rcv(struct sock *sk, struct sk_buff *skb)
-{
-   lock_sock(sk);
-   pppoe_rcv_core(sk, skb);
-   release_sock(sk);
-   return 0;
-}
-
-
-
 /
  *
  * Receive a PPPoE Discovery frame.
@@ -481,7 +466,7 @@
sk->protocol = PX_PROTO_OE;
sk->family = PF_PPPOX;
 
-   sk->backlog_rcv = pppoe_backlog_rcv;
+   sk->backlog_rcv = pppoe_rcv_core;
sk->next = NULL;
sk->pprev = NULL;
sk->state = PPPOX_NONE;

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/



Re: Scheduling in interrupt BUG. [Patch]

2001-05-14 Thread Marcell Gal

Hi,

This patch solved the problem. Should be ready for inclusion in 2.4.
No more 'Scheduling in interrupt' under those conditions.
Thanx for the thoughts, solution and the amazing speed.
You guys are doing a really great job!

I hope we can get the earlier mentioned NULL ptr in all_ppp_units list
straight
soon. (I have a simple workaround - the mentioned hash, that even improves
speed,
but I a real fix would be more satisfaction. The relevant part of
ppp_generic.c
is so simple that it's really strange it is not correct.. ).

thanx:
Cell

Michal Ostrowski wrote:

> Anybody care to comment on this?
> [EMAIL PROTECTED]

--- linuxold/drivers/net/pppoe.cMon May 14 22:06:44 2001
+++ linux/drivers/net/pppoe.c   Mon May 14 22:11:25 2001
@@ -4,9 +4,9 @@
  * PPPoX --- Generic PPP encapsulation socket family
  * PPPoE --- PPP over Ethernet (RFC 2516)
  *
  *
- * Version:0.6.5
+ * Version:0.6.6
  *
  * 030700 : Fixed connect logic to allow for disconnect.
  * 270700 :Fixed potential SMP problems; we must protect against
  * simultaneous invocation of ppp_input
@@ -18,8 +18,9 @@
  * in pppoe_release.
  * 051000 :Initialization cleanup.
  * 00 :Fix recvmsg.
  * 050101 :Fix PADT procesing.
+ * 140501 :pppoe_backlog_rcv must call bh_lock_sock, not lock_sock.
  *
  * Author: Michal Ostrowski <[EMAIL PROTECTED]>
  * Contributors:
  * Arnaldo Carvalho de Melo <[EMAIL PROTECTED]>
@@ -383,11 +384,11 @@
  *
  ***/
 int pppoe_backlog_rcv(struct sock *sk, struct sk_buff *skb)
 {
-   lock_sock(sk);
+   bh_lock_sock(sk);
pppoe_rcv_core(sk, skb);
-   release_sock(sk);
+   bh_unlock_sock(sk);
return 0;
 }


--
You'll never see all the places, or read all the books, but fortunately,
they're not all recommended.



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/



Re: Scheduling in interrupt BUG.

2001-05-14 Thread Andrew Morton

Marcell GAL wrote:
> 
> int pppoe_backlog_rcv(struct sock *sk, struct sk_buff *skb)
> {
> lock_sock(sk);
> pppoe_rcv_core(sk, skb);
> release_sock(sk);
> return 0;
> }
> 

The backlog_rcv() method is called inside local_bh_disable()
and so cannot call lock_sock().   Definitely a bug in pppoe.

It looks like pppoe_backlog_rcv() should be using bh_lock_sock().
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/



Scheduling in interrupt BUG. [Patch]

2001-05-14 Thread Michal Ostrowski


Marcell GAL writes:
 > Hi Guys,
 > 
 > Once upon a time on my
 > x86 UP box, UP kernel 2.4.4, (64M ram, 260M swap)
 > http://home.sch.bme.hu/~cell/.config
 > I hit a reproducable "Scheduling in interrupt" BUG.
 > Also reproduced with 128M ram and low memory pressure
 > (first I suspected it is related to swapping)
 > Running lots of pppd version 2.4.0 (pppoe) sessions almost at the same
 > time. 
 > (before the crash the pppoe sessions work fine)
 > It crashed after 89 sessions, 473 another time.. (depending
 > on the phase of Jupiter moons I guess .. I still have to verify this),
 > usually much before memory is exhausted (30k mem/pppd process).
 > To do this you have to patch ppp_generic.c
 > http://x-dsl.hu/~cell/ppp_generic_hash/, because
 > otherwise we hit 'NULL ptr in all_ppp_units list'
 > BUG much _more likely_ than this 'sched.c line 709 thingy'..
 > 
 > EIP: c010faa4    <= sched.c schedule(), line 709:
 > which is ~ printk("Scheduling in interrupt");BUG();

>From what I've seen, you have a call chain consisting of:

__release_sock -> pppoe_backlog_rcv -> __lock_sock

This is going to be bad, because when __release_sock calls
sk->backlog_rcv, lock.users is still non-zero and thus the lock_sock
operation will block (leading to deadlock).  This problem is fixed
with the patch that I've added below.

You're seeing the "Scheduling in interrupt" message because the
combined effect of the various spin_lock/unlock calls in release_sock
and __release_sock at the time of the call to sk->backlog_rcv is to
increase the local bh count.

Having looked at the code for locking sockets I am concerned that the
locking procedures for tcp may be wrong.   __release_sock releases the
socket spinlock before calling sk->backlog_rcv() (== tcp_v4_do_rcv),
however the comments at the top of tcp_v4_do_rcv() assert that the
socket's spinlock is held (which is definitely not the case).

Anybody care to comment on this?

Michal Ostrowski
[EMAIL PROTECTED]

--- drivers/net/pppoe.c~Tue Mar  6 22:44:35 2001
+++ drivers/net/pppoe.c Mon May 14 08:24:06 2001
@@ -5,7 +5,7 @@
  * PPPoE --- PPP over Ethernet (RFC 2516)
  *
  *
- * Version:0.6.5
+ * Version:0.6.6
  *
  * 030700 : Fixed connect logic to allow for disconnect.
  * 270700 :Fixed potential SMP problems; we must protect against
@@ -19,6 +19,7 @@
  * 051000 :Initialization cleanup.
  * 00 :Fix recvmsg.
  * 050101 :Fix PADT procesing.
+ * 140501 :pppoe_backlog_rcv must call bh_lock_sock, not lock_sock.
  *
  * Author: Michal Ostrowski <[EMAIL PROTECTED]>
  * Contributors:
@@ -384,9 +385,9 @@
  ***/
 int pppoe_backlog_rcv(struct sock *sk, struct sk_buff *skb)
 {
-   lock_sock(sk);
+   bh_lock_sock(sk);
pppoe_rcv_core(sk, skb);
-   release_sock(sk);
+   bh_unlock_sock(sk);
return 0;
 }
 

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/



Scheduling in interrupt BUG.

2001-05-14 Thread Marcell GAL

Hi Guys,

Once upon a time on my
x86 UP box, UP kernel 2.4.4, (64M ram, 260M swap)
http://home.sch.bme.hu/~cell/.config
I hit a reproducable "Scheduling in interrupt" BUG.
Also reproduced with 128M ram and low memory pressure
(first I suspected it is related to swapping)
Running lots of pppd version 2.4.0 (pppoe) sessions almost at the same
time. 
(before the crash the pppoe sessions work fine)
It crashed after 89 sessions, 473 another time.. (depending
on the phase of Jupiter moons I guess .. I still have to verify this),
usually much before memory is exhausted (30k mem/pppd process).
To do this you have to patch ppp_generic.c
http://x-dsl.hu/~cell/ppp_generic_hash/, because
otherwise we hit 'NULL ptr in all_ppp_units list'
BUG much _more likely_ than this 'sched.c line 709 thingy'..

EIP: c010faa4<= sched.c schedule(), line 709:
which is ~ printk("Scheduling in interrupt");BUG();

Trace:

0xc01ddac5 <__lock_sock+53>:movl   $0x0,0x1c(%esp,1)
0xc01ddacd <__lock_sock+61>:mov%ebx,0x20(%esp,1)
0xc01ddad1 <__lock_sock+65>:movl   $0x0,0x24(%esp,1)
0xc01ddad9 <__lock_sock+73>:movl   $0x0,0x28(%esp,1)
0xc01ddae1 <__lock_sock+81>:lea0x1c(%esp,1),%esi
0xc01ddae5 <__lock_sock+85>:lea0x34(%edi),%eax
0xc01ddae8 <__lock_sock+88>:mov%esi,%edx
0xc01ddaea <__lock_sock+90>:call   0xc0110598

0xc01ddaef <__lock_sock+95>:nop
0xc01ddaf0 <__lock_sock+96>:movl   $0x2,(%ebx)
0xc01ddaf6 <__lock_sock+102>:   decl   0xc02f75ec
0xc01ddafc <__lock_sock+108>:   call   0xc010f71c  
*
0xc01ddb01 <__lock_sock+113>:   incl   0xc02f75ec
0xc01ddb07 <__lock_sock+119>:   cmpl   $0x0,0x30(%edi)
0xc01ddb0b <__lock_sock+123>:   jne0xc01ddaf0 <__lock_sock+96>

-
0xc01a315c : push   %esi
0xc01a315d :   push   %ebx
0xc01a315e :   mov0xc(%esp,1),%ebx
0xc01a3162 :   incl   0xc02f75ec
0xc01a3168 :  cmpl   $0x0,0x30(%ebx)
0xc01a316c :
je 0xc01a3177 
0xc01a316e :  push   %ebx
0xc01a316f :  call   0xc01dda90
<__lock_sock>   
0xc01a3174 :  add$0x4,%esp
0xc01a3177 :  movl   $0x1,0x30(%ebx)
0xc01a317e :  decl   0xc02f75ec
0xc01a3184 :  mov0x10(%esp,1),%eax

0xc01ddb2c <__release_sock>:push   %esi
0xc01ddb2d <__release_sock+1>:  push   %ebx
0xc01ddb2e <__release_sock+2>:  mov0xc(%esp,1),%esi
0xc01ddb32 <__release_sock+6>:  mov0xb8(%esi),%eax
0xc01ddb38 <__release_sock+12>: movl   $0x0,0xbc(%esi)
0xc01ddb42 <__release_sock+22>: movl   $0x0,0xb8(%esi)
0xc01ddb4c <__release_sock+32>: lea0x0(%esi,1),%esi
0xc01ddb50 <__release_sock+36>: mov(%eax),%ebx
0xc01ddb52 <__release_sock+38>: movl   $0x0,(%eax)
0xc01ddb58 <__release_sock+44>: push   %eax
0xc01ddb59 <__release_sock+45>: push   %esi
0xc01ddb5a <__release_sock+46>: mov0x31c(%esi),%eax
0xc01ddb60 <__release_sock+52>: call   *%eax
0xc01ddb62 <__release_sock+54>: mov%ebx,%eax
0xc01ddb64 <__release_sock+56>: add$0x8,%esp
0xc01ddb67 <__release_sock+59>: test   %eax,%eax


int pppoe_backlog_rcv(struct sock *sk, struct sk_buff *skb)
{
lock_sock(sk);
pppoe_rcv_core(sk, skb);
release_sock(sk);
return 0;
}


What else should I check? How can we fix it?
PPPoE is more and more frequently used nowadays because of ADSL
services. I think this can effect its stability (guess which direction
;-)
even with one session (though probably not that bad as with many
sessions).
 
Have a nice week:
Cell

-- 
Alice? WTFIA?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/



Scheduling in interrupt BUG.

2001-05-14 Thread Marcell GAL

Hi Guys,

Once upon a time on my
x86 UP box, UP kernel 2.4.4, (64M ram, 260M swap)
http://home.sch.bme.hu/~cell/.config
I hit a reproducable Scheduling in interrupt BUG.
Also reproduced with 128M ram and low memory pressure
(first I suspected it is related to swapping)
Running lots of pppd version 2.4.0 (pppoe) sessions almost at the same
time. 
(before the crash the pppoe sessions work fine)
It crashed after 89 sessions, 473 another time.. (depending
on the phase of Jupiter moons I guess .. I still have to verify this),
usually much before memory is exhausted (30k mem/pppd process).
To do this you have to patch ppp_generic.c
http://x-dsl.hu/~cell/ppp_generic_hash/, because
otherwise we hit 'NULL ptr in all_ppp_units list'
BUG much _more likely_ than this 'sched.c line 709 thingy'..

EIP: c010faa4 schedule+388/394   = sched.c schedule(), line 709:
which is ~ printk(Scheduling in interrupt);BUG();

Trace:

0xc01ddac5 __lock_sock+53:movl   $0x0,0x1c(%esp,1)
0xc01ddacd __lock_sock+61:mov%ebx,0x20(%esp,1)
0xc01ddad1 __lock_sock+65:movl   $0x0,0x24(%esp,1)
0xc01ddad9 __lock_sock+73:movl   $0x0,0x28(%esp,1)
0xc01ddae1 __lock_sock+81:lea0x1c(%esp,1),%esi
0xc01ddae5 __lock_sock+85:lea0x34(%edi),%eax
0xc01ddae8 __lock_sock+88:mov%esi,%edx
0xc01ddaea __lock_sock+90:call   0xc0110598
add_wait_queue_exclusive
0xc01ddaef __lock_sock+95:nop
0xc01ddaf0 __lock_sock+96:movl   $0x2,(%ebx)
0xc01ddaf6 __lock_sock+102:   decl   0xc02f75ec
0xc01ddafc __lock_sock+108:   call   0xc010f71c schedule 
*
0xc01ddb01 __lock_sock+113:   incl   0xc02f75ec
0xc01ddb07 __lock_sock+119:   cmpl   $0x0,0x30(%edi)
0xc01ddb0b __lock_sock+123:   jne0xc01ddaf0 __lock_sock+96

-
0xc01a315c pppoe_backlog_rcv: push   %esi
0xc01a315d pppoe_backlog_rcv+1:   push   %ebx
0xc01a315e pppoe_backlog_rcv+2:   mov0xc(%esp,1),%ebx
0xc01a3162 pppoe_backlog_rcv+6:   incl   0xc02f75ec
0xc01a3168 pppoe_backlog_rcv+12:  cmpl   $0x0,0x30(%ebx)
0xc01a316c pppoe_backlog_rcv+16:
je 0xc01a3177 pppoe_backlog_rcv+27
0xc01a316e pppoe_backlog_rcv+18:  push   %ebx
0xc01a316f pppoe_backlog_rcv+19:  call   0xc01dda90
__lock_sock   
0xc01a3174 pppoe_backlog_rcv+24:  add$0x4,%esp
0xc01a3177 pppoe_backlog_rcv+27:  movl   $0x1,0x30(%ebx)
0xc01a317e pppoe_backlog_rcv+34:  decl   0xc02f75ec
0xc01a3184 pppoe_backlog_rcv+40:  mov0x10(%esp,1),%eax

0xc01ddb2c __release_sock:push   %esi
0xc01ddb2d __release_sock+1:  push   %ebx
0xc01ddb2e __release_sock+2:  mov0xc(%esp,1),%esi
0xc01ddb32 __release_sock+6:  mov0xb8(%esi),%eax
0xc01ddb38 __release_sock+12: movl   $0x0,0xbc(%esi)
0xc01ddb42 __release_sock+22: movl   $0x0,0xb8(%esi)
0xc01ddb4c __release_sock+32: lea0x0(%esi,1),%esi
0xc01ddb50 __release_sock+36: mov(%eax),%ebx
0xc01ddb52 __release_sock+38: movl   $0x0,(%eax)
0xc01ddb58 __release_sock+44: push   %eax
0xc01ddb59 __release_sock+45: push   %esi
0xc01ddb5a __release_sock+46: mov0x31c(%esi),%eax
0xc01ddb60 __release_sock+52: call   *%eax
0xc01ddb62 __release_sock+54: mov%ebx,%eax
0xc01ddb64 __release_sock+56: add$0x8,%esp
0xc01ddb67 __release_sock+59: test   %eax,%eax


int pppoe_backlog_rcv(struct sock *sk, struct sk_buff *skb)
{
lock_sock(sk);
pppoe_rcv_core(sk, skb);
release_sock(sk);
return 0;
}


What else should I check? How can we fix it?
PPPoE is more and more frequently used nowadays because of ADSL
services. I think this can effect its stability (guess which direction
;-)
even with one session (though probably not that bad as with many
sessions).
 
Have a nice week:
Cell

-- 
Alice? WTFIA?
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/



Re: Scheduling in interrupt BUG.

2001-05-14 Thread Andrew Morton

Marcell GAL wrote:
 
 int pppoe_backlog_rcv(struct sock *sk, struct sk_buff *skb)
 {
 lock_sock(sk);
 pppoe_rcv_core(sk, skb);
 release_sock(sk);
 return 0;
 }
 

The backlog_rcv() method is called inside local_bh_disable()
and so cannot call lock_sock().   Definitely a bug in pppoe.

It looks like pppoe_backlog_rcv() should be using bh_lock_sock().
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/



Re: Scheduling in interrupt BUG. [Patch]

2001-05-14 Thread Michal Ostrowski

Marcell Gal writes:
  Hi,
  
  This patch solved the problem. Should be ready for inclusion in 2.4.
  No more 'Scheduling in interrupt' under those conditions.
  Thanx for the thoughts, solution and the amazing speed.
  You guys are doing a really great job!
  

Alexey pointed out a much nicer/correct/elegant/efficient
solution to this problem and I think that that's the way to go.

New patch below.

Michal Ostrowski
[EMAIL PROTECTED]


--- drivers/net/pppoe.c~Tue Mar  6 22:44:35 2001
+++ drivers/net/pppoe.c Mon May 14 14:10:49 2001
@@ -5,7 +5,7 @@
  * PPPoE --- PPP over Ethernet (RFC 2516)
  *
  *
- * Version:0.6.5
+ * Version:0.6.6
  *
  * 030700 : Fixed connect logic to allow for disconnect.
  * 270700 :Fixed potential SMP problems; we must protect against
@@ -19,6 +19,7 @@
  * 051000 :Initialization cleanup.
  * 00 :Fix recvmsg.
  * 050101 :Fix PADT procesing.
+ * 140501 :Use pppoe_rcv_core to handle all backlog. (Alexey)
  *
  * Author: Michal Ostrowski [EMAIL PROTECTED]
  * Contributors:
@@ -376,22 +377,6 @@
return ret;
 }
 
-
-/
- *
- * Receive wrapper called in process context.
- *
- ***/
-int pppoe_backlog_rcv(struct sock *sk, struct sk_buff *skb)
-{
-   lock_sock(sk);
-   pppoe_rcv_core(sk, skb);
-   release_sock(sk);
-   return 0;
-}
-
-
-
 /
  *
  * Receive a PPPoE Discovery frame.
@@ -481,7 +466,7 @@
sk-protocol = PX_PROTO_OE;
sk-family = PF_PPPOX;
 
-   sk-backlog_rcv = pppoe_backlog_rcv;
+   sk-backlog_rcv = pppoe_rcv_core;
sk-next = NULL;
sk-pprev = NULL;
sk-state = PPPOX_NONE;

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/



Re: Scheduling in interrupt BUG. [Patch]

2001-05-14 Thread Andi Kleen

On Mon, May 14, 2001 at 08:32:33AM -0400, Michal Ostrowski wrote:
 Having looked at the code for locking sockets I am concerned that the
 locking procedures for tcp may be wrong.   __release_sock releases the
 socket spinlock before calling sk-backlog_rcv() (== tcp_v4_do_rcv),
 however the comments at the top of tcp_v4_do_rcv() assert that the
 socket's spinlock is held (which is definitely not the case).
 
 Anybody care to comment on this?

Looks ok for me.

The user socket lock (lock.users0) is held while __release_sock runs, 
which is also sufficient to protect it as all new packets will go into backlog.
The spinlock comment only applies to bottom halves.

-Andi

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/



Scheduling in interrupt BUG. [Patch]

2001-05-14 Thread Michal Ostrowski


Marcell GAL writes:
  Hi Guys,
  
  Once upon a time on my
  x86 UP box, UP kernel 2.4.4, (64M ram, 260M swap)
  http://home.sch.bme.hu/~cell/.config
  I hit a reproducable Scheduling in interrupt BUG.
  Also reproduced with 128M ram and low memory pressure
  (first I suspected it is related to swapping)
  Running lots of pppd version 2.4.0 (pppoe) sessions almost at the same
  time. 
  (before the crash the pppoe sessions work fine)
  It crashed after 89 sessions, 473 another time.. (depending
  on the phase of Jupiter moons I guess .. I still have to verify this),
  usually much before memory is exhausted (30k mem/pppd process).
  To do this you have to patch ppp_generic.c
  http://x-dsl.hu/~cell/ppp_generic_hash/, because
  otherwise we hit 'NULL ptr in all_ppp_units list'
  BUG much _more likely_ than this 'sched.c line 709 thingy'..
  
  EIP: c010faa4 schedule+388/394   = sched.c schedule(), line 709:
  which is ~ printk(Scheduling in interrupt);BUG();

From what I've seen, you have a call chain consisting of:

__release_sock - pppoe_backlog_rcv - __lock_sock

This is going to be bad, because when __release_sock calls
sk-backlog_rcv, lock.users is still non-zero and thus the lock_sock
operation will block (leading to deadlock).  This problem is fixed
with the patch that I've added below.

You're seeing the Scheduling in interrupt message because the
combined effect of the various spin_lock/unlock calls in release_sock
and __release_sock at the time of the call to sk-backlog_rcv is to
increase the local bh count.

Having looked at the code for locking sockets I am concerned that the
locking procedures for tcp may be wrong.   __release_sock releases the
socket spinlock before calling sk-backlog_rcv() (== tcp_v4_do_rcv),
however the comments at the top of tcp_v4_do_rcv() assert that the
socket's spinlock is held (which is definitely not the case).

Anybody care to comment on this?

Michal Ostrowski
[EMAIL PROTECTED]

--- drivers/net/pppoe.c~Tue Mar  6 22:44:35 2001
+++ drivers/net/pppoe.c Mon May 14 08:24:06 2001
@@ -5,7 +5,7 @@
  * PPPoE --- PPP over Ethernet (RFC 2516)
  *
  *
- * Version:0.6.5
+ * Version:0.6.6
  *
  * 030700 : Fixed connect logic to allow for disconnect.
  * 270700 :Fixed potential SMP problems; we must protect against
@@ -19,6 +19,7 @@
  * 051000 :Initialization cleanup.
  * 00 :Fix recvmsg.
  * 050101 :Fix PADT procesing.
+ * 140501 :pppoe_backlog_rcv must call bh_lock_sock, not lock_sock.
  *
  * Author: Michal Ostrowski [EMAIL PROTECTED]
  * Contributors:
@@ -384,9 +385,9 @@
  ***/
 int pppoe_backlog_rcv(struct sock *sk, struct sk_buff *skb)
 {
-   lock_sock(sk);
+   bh_lock_sock(sk);
pppoe_rcv_core(sk, skb);
-   release_sock(sk);
+   bh_unlock_sock(sk);
return 0;
 }
 

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/