Re: -current lock warning...

2002-03-18 Thread John Baldwin 


On 18-Mar-2002 Alfred Perlstein wrote:
> * John Baldwin <[EMAIL PROTECTED]> [020318 10:24] wrote:
>> 
>> On 17-Mar-2002 Robert Watson wrote:
>> > 
>> > On Sun, 17 Mar 2002, Alfred Perlstein wrote:
>> > 
>> >> * Munehiro Matsuda <[EMAIL PROTECTED]> [020317 06:36] wrote:
>> >> > 
>> >> > PS. I got another message that happend when I ^C'ed a buildworld
>> >> > earlier, 
>> >> > with same kernel. May be it should go to Alfred Perlstein?
>> >> > 
>> >> > lock order reversal
>> >> >  1st 0xc198eec0 pipe mutex @ ../../../kern/sys_pipe.c:779
>> >> >  2nd 0xc0367fe0 Giant @ ../../../i386/i386/trap.c:716
>> >> 
>> >> I think there's a place where the pipe can fault on an address while
>> >> copying, I'll take a look at this. 
>> > 
>> > Are there any assertions that should be in place for copyin/copyout
>> > requring fault handling?  It sounds like somewhere we need to assert that
>> > Giant is held...
>> 
>> More correct is that probably no locks other than Giant should be held for
>> copyin/copyout.
> 
> s/probably/definetly.
> 
> Can you please provide a "blessed" API for raising and lowering a
> "can't block" count in the thread?  This can be used in copyout,
> copyin, and a bunch of vm and buffer ops to make sure we aren't calling
> them with mutexes held.

*sigh*  I've never objected to it and even provided you with tips on how to
implement it.  What more do you want?   I'm not really sure we need it though
to be honest as the implicit checks done by witness will probably ensure this
for us.

-- 

John Baldwin <[EMAIL PROTECTED]>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve!"  -  http://www.FreeBSD.org/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: -current lock warning...

2002-03-18 Thread Alfred Perlstein 

* John Baldwin <[EMAIL PROTECTED]> [020318 10:24] wrote:
> 
> On 17-Mar-2002 Robert Watson wrote:
> > 
> > On Sun, 17 Mar 2002, Alfred Perlstein wrote:
> > 
> >> * Munehiro Matsuda <[EMAIL PROTECTED]> [020317 06:36] wrote:
> >> > 
> >> > PS. I got another message that happend when I ^C'ed a buildworld earlier, 
> >> > with same kernel. May be it should go to Alfred Perlstein?
> >> > 
> >> > lock order reversal
> >> >  1st 0xc198eec0 pipe mutex @ ../../../kern/sys_pipe.c:779
> >> >  2nd 0xc0367fe0 Giant @ ../../../i386/i386/trap.c:716
> >> 
> >> I think there's a place where the pipe can fault on an address while
> >> copying, I'll take a look at this. 
> > 
> > Are there any assertions that should be in place for copyin/copyout
> > requring fault handling?  It sounds like somewhere we need to assert that
> > Giant is held...
> 
> More correct is that probably no locks other than Giant should be held for
> copyin/copyout.

s/probably/definetly.

Can you please provide a "blessed" API for raising and lowering a
"can't block" count in the thread?  This can be used in copyout,
copyin, and a bunch of vm and buffer ops to make sure we aren't calling
them with mutexes held.

-- 
-Alfred Perlstein [[EMAIL PROTECTED]]
'Instead of asking why a piece of software is using "1970s technology,"
 start asking why software is ignoring 30 years of accumulated wisdom.'
Tax deductible donations for FreeBSD: http://www.freebsdfoundation.org/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: -current lock warning...

2002-03-18 Thread John Baldwin 


On 17-Mar-2002 Robert Watson wrote:
> 
> On Sun, 17 Mar 2002, Alfred Perlstein wrote:
> 
>> * Munehiro Matsuda <[EMAIL PROTECTED]> [020317 06:36] wrote:
>> > 
>> > PS. I got another message that happend when I ^C'ed a buildworld earlier, 
>> > with same kernel. May be it should go to Alfred Perlstein?
>> > 
>> > lock order reversal
>> >  1st 0xc198eec0 pipe mutex @ ../../../kern/sys_pipe.c:779
>> >  2nd 0xc0367fe0 Giant @ ../../../i386/i386/trap.c:716
>> 
>> I think there's a place where the pipe can fault on an address while
>> copying, I'll take a look at this. 
> 
> Are there any assertions that should be in place for copyin/copyout
> requring fault handling?  It sounds like somewhere we need to assert that
> Giant is held...

More correct is that probably no locks other than Giant should be held for
copyin/copyout.

-- 

John Baldwin <[EMAIL PROTECTED]>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve!"  -  http://www.FreeBSD.org/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: -current lock warning...

2002-03-17 Thread Jake Burkholder 

Apparently, On Sun, Mar 17, 2002 at 09:17:22AM -0800,
Alfred Perlstein said words to the effect of;

> * Robert Watson <[EMAIL PROTECTED]> [020317 09:08] wrote:
> > 
> > On Sun, 17 Mar 2002, Alfred Perlstein wrote:
> > 
> > > * Munehiro Matsuda <[EMAIL PROTECTED]> [020317 06:36] wrote:
> > > > 
> > > > PS. I got another message that happend when I ^C'ed a buildworld earlier, 
> > > > with same kernel. May be it should go to Alfred Perlstein?
> > > > 
> > > > lock order reversal
> > > >  1st 0xc198eec0 pipe mutex @ ../../../kern/sys_pipe.c:779
> > > >  2nd 0xc0367fe0 Giant @ ../../../i386/i386/trap.c:716
> > > 
> > > I think there's a place where the pipe can fault on an address while
> > > copying, I'll take a look at this. 
> > 
> > Are there any assertions that should be in place for copyin/copyout
> > requring fault handling?  It sounds like somewhere we need to assert that
> > Giant is held...
> 
> No, you need to assert that no other mutex other than Giant is held.
> 
> It would be nice... :)

You can do this like at the bottom of syscall and trap, with witness_list().
It'll even print out what the other locks are and where they were acquired.

But yeah, if you're going to access pageable memory in kernel mode you pretty
much have to have no other locks held.

Good work on pipe locking btw.

Jake

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: -current lock warning...

2002-03-17 Thread Alfred Perlstein 

* Robert Watson <[EMAIL PROTECTED]> [020317 09:08] wrote:
> 
> On Sun, 17 Mar 2002, Alfred Perlstein wrote:
> 
> > * Munehiro Matsuda <[EMAIL PROTECTED]> [020317 06:36] wrote:
> > > 
> > > PS. I got another message that happend when I ^C'ed a buildworld earlier, 
> > > with same kernel. May be it should go to Alfred Perlstein?
> > > 
> > > lock order reversal
> > >  1st 0xc198eec0 pipe mutex @ ../../../kern/sys_pipe.c:779
> > >  2nd 0xc0367fe0 Giant @ ../../../i386/i386/trap.c:716
> > 
> > I think there's a place where the pipe can fault on an address while
> > copying, I'll take a look at this. 
> 
> Are there any assertions that should be in place for copyin/copyout
> requring fault handling?  It sounds like somewhere we need to assert that
> Giant is held...

No, you need to assert that no other mutex other than Giant is held.

It would be nice... :)

-- 
-Alfred Perlstein [[EMAIL PROTECTED]]
'Instead of asking why a piece of software is using "1970s technology,"
 start asking why software is ignoring 30 years of accumulated wisdom.'
Tax deductible donations for FreeBSD: http://www.freebsdfoundation.org/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: -current lock warning...

2002-03-17 Thread Robert Watson 


On Sun, 17 Mar 2002, Alfred Perlstein wrote:

> * Munehiro Matsuda <[EMAIL PROTECTED]> [020317 06:36] wrote:
> > 
> > PS. I got another message that happend when I ^C'ed a buildworld earlier, 
> > with same kernel. May be it should go to Alfred Perlstein?
> > 
> > lock order reversal
> >  1st 0xc198eec0 pipe mutex @ ../../../kern/sys_pipe.c:779
> >  2nd 0xc0367fe0 Giant @ ../../../i386/i386/trap.c:716
> 
> I think there's a place where the pipe can fault on an address while
> copying, I'll take a look at this. 

Are there any assertions that should be in place for copyin/copyout
requring fault handling?  It sounds like somewhere we need to assert that
Giant is held...

Robert N M Watson FreeBSD Core Team, TrustedBSD Project
[EMAIL PROTECTED]  NAI Labs, Safeport Network Services



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: -current lock warning...

2002-03-17 Thread Alfred Perlstein 

* Munehiro Matsuda <[EMAIL PROTECTED]> [020317 06:36] wrote:
> 
> PS. I got another message that happend when I ^C'ed a buildworld earlier, 
> with same kernel. May be it should go to Alfred Perlstein?
> 
> lock order reversal
>  1st 0xc198eec0 pipe mutex @ ../../../kern/sys_pipe.c:779
>  2nd 0xc0367fe0 Giant @ ../../../i386/i386/trap.c:716

I think there's a place where the pipe can fault on an address
while copying, I'll take a look at this.

-- 
-Alfred Perlstein [[EMAIL PROTECTED]]
'Instead of asking why a piece of software is using "1970s technology,"
 start asking why software is ignoring 30 years of accumulated wisdom.'
Tax deductible donations for FreeBSD: http://www.freebsdfoundation.org/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: -current lock warning...

2002-03-17 Thread Munehiro Matsuda 

From: Seigo Tanimura <[EMAIL PROTECTED]>
Date: Sun, 17 Mar 2002 01:17:21 +0900
::Seigo> On Sat, 16 Mar 2002 10:22:22 +0100,
::Seigo>   Poul-Henning Kamp <[EMAIL PROTECTED]> said:
::
::Seigo> Poul-Henning> acquiring duplicate lock of same type: "thrd_sleep"
::Seigo> Poul-Henning>  1st @ ../../../vm/vm_map.c:2288
::Seigo> Poul-Henning>  2nd @ ../../../vm/vm_kern.c:172

::The patch attached below renames the lock of the kernel_map to
::"kernel_map" once witness gets ready to run. This eliminated all of
::the lock order reversals on my PC.

Hello Seigo,

Thanks for the patch.
That eliminated the 'duplicate lock' warning and worked fine on my UP system
with 'make -j2 buildworld'.

Thank you,
  Haro

PS. I got another message that happend when I ^C'ed a buildworld earlier, 
with same kernel. May be it should go to Alfred Perlstein?

lock order reversal
 1st 0xc198eec0 pipe mutex @ ../../../kern/sys_pipe.c:779
 2nd 0xc0367fe0 Giant @ ../../../i386/i386/trap.c:716

=--
   _ _Munehiro (haro) Matsuda
 -|- /_\  |_|_|   Business Incubation Dept., Kubota Corp.
 /|\ |_|  |_|_|   1-3 Nihonbashi-Muromachi 3-Chome
  Chuo-ku Tokyo 103-8310, Japan
  Tel: +81-3-3245-3318  Fax: +81-3-3245-3315
  Email: [EMAIL PROTECTED]

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: -current lock warning...

2002-03-16 Thread Bill Fenner 


> Although I am still getting the following lock problems when I shut
> the system down:
>
> lock order reversal
>  1st 0xc036afc0 allproc @ ../../../kern/vfs_syscalls.c:452
>  2nd 0xc7ecce34 filedesc structure @ ../../../kern/vfs_syscalls.c:457

I've been seeing this since Feb 4.

  Bill

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: -current lock warning...

2002-03-16 Thread Lamont Granquist 


I've seen this as well, -current from about 5 days ago, dual proc 1.4GHz
K7 A7M266D with a 13GB IBM UDMA66 drive, GENERIC kernel + hints.

On Sat, 16 Mar 2002, Hiten Pandya wrote:
> --- Poul-Henning Kamp <[EMAIL PROTECTED]> wrote:
> > >I haven't seen this.  I built a kernel today, and I have a dual processor
> > >machine.  Are you using any special kernel options, such as VFS_BIO_DEBUG
> > >or something, or am I talking nuts? :)
> >
> > Well, I have.  On a single CPU net-booting -current.  No.  Yes :-)
>
> Although I am still getting the following lock problems when I shut
> the system down:
>
> lock order reversal
>  1st 0xc036afc0 allproc @ ../../../kern/vfs_syscalls.c:452
>  2nd 0xc7ecce34 filedesc structure @ ../../../kern/vfs_syscalls.c:457
>
> I think I will have to check out lock problems you are getting today,
> once I finish my breakfast. :-)
>
> __
> Do You Yahoo!?
> Yahoo! Sports - live college hoops coverage
> http://sports.yahoo.com/
>
> To Unsubscribe: send mail to [EMAIL PROTECTED]
> with "unsubscribe freebsd-current" in the body of the message
>
>


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: -current lock warning...

2002-03-16 Thread Seigo Tanimura 

On Sat, 16 Mar 2002 22:56:58 +0900,
  Seigo Tanimura <[EMAIL PROTECTED]> said:

Seigo> On Sat, 16 Mar 2002 10:22:22 +0100,
Seigo>   Poul-Henning Kamp <[EMAIL PROTECTED]> said:

Seigo> Poul-Henning> acquiring duplicate lock of same type: "thrd_sleep"
Seigo> Poul-Henning>  1st @ ../../../vm/vm_map.c:2288
Seigo> Poul-Henning>  2nd @ ../../../vm/vm_kern.c:172
Seigo> (snip)
Seigo> Poul-Henning> _vm_map_lock(c038afb4,c02f8440,ac,c034a840,1) at _vm_map_lock+0x16
Seigo> Poul-Henning> kmem_alloc(c038afb4,3000,c0e41a00,0,c02fa434) at kmem_alloc+0x41
Seigo> Poul-Henning> _zget(c0e41a00,bfc0,0,c0435cd0,c0281769) at _zget+0xfa
Seigo> Poul-Henning> zalloc(c0e41a00,c034a840,1,c02f8630,a7) at zalloc+0x3b
Seigo> Poul-Henning> vmspace_alloc(0,bfc0,c035c940,c02f8630,8f0) at 
vmspace_alloc+0x2d
Seigo> Poul-Henning> vmspace_fork(c035c940,cbb9ad24,c0331f84,cbb9ab00,c0331d60) at 
vmspace_fork+0x4d
Seigo> Poul-Henning> vm_forkproc(c0332080,cbb9ab00,cbb9ac00,20014) at vm_forkproc+0xc6

Seigo> This seems due to naming all of the vm map locks as "thrd_sleep." The
Seigo> locks of vm maps should have their own hierarachy. For instance, lock
Seigo> the map of a process vm space first, then lock the kernel_map.

The patch attached below renames the lock of the kernel_map to
"kernel_map" once witness gets ready to run. This eliminated all of
the lock order reversals on my PC.




vm_map_renamelock.diff
Description: Binary data


-- 
Seigo Tanimura <[EMAIL PROTECTED]> <[EMAIL PROTECTED]>

Re: -current lock warning...

2002-03-16 Thread Hiten Pandya 

> The patch attached below renames the lock of the kernel_map to
> "kernel_map" once witness gets ready to run. This eliminated all of
> the lock order reversals on my PC.

I will check it out.  I will check this out on my dual-processor machine.
Thanks for the patch. :-)

__
Do You Yahoo!?
Yahoo! Sports - live college hoops coverage
http://sports.yahoo.com/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: -current lock warning...

2002-03-16 Thread Robert Watson 


On Sat, 16 Mar 2002, Poul-Henning Kamp wrote:

> I get this one on every single boot.  We're not shipping the snapshot
> with that in place, right ? 

I assume this is from Brian's recent VM locking commit.  Currently, it's
on the list of commits to back out of the DP1 branch, assuming the issues
aren't resolved in the next day or two.  If they get resolved, all the
better, we'll leave it in and pick up the fix.

Robert N M Watson FreeBSD Core Team, TrustedBSD Project
[EMAIL PROTECTED]  NAI Labs, Safeport Network Services



To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: -current lock warning...

2002-03-16 Thread Seigo Tanimura 

On Sat, 16 Mar 2002 10:22:22 +0100,
  Poul-Henning Kamp <[EMAIL PROTECTED]> said:

Poul-Henning> acquiring duplicate lock of same type: "thrd_sleep"
Poul-Henning>  1st @ ../../../vm/vm_map.c:2288
Poul-Henning>  2nd @ ../../../vm/vm_kern.c:172
(snip)
Poul-Henning> _vm_map_lock(c038afb4,c02f8440,ac,c034a840,1) at _vm_map_lock+0x16
Poul-Henning> kmem_alloc(c038afb4,3000,c0e41a00,0,c02fa434) at kmem_alloc+0x41
Poul-Henning> _zget(c0e41a00,bfc0,0,c0435cd0,c0281769) at _zget+0xfa
Poul-Henning> zalloc(c0e41a00,c034a840,1,c02f8630,a7) at zalloc+0x3b
Poul-Henning> vmspace_alloc(0,bfc0,c035c940,c02f8630,8f0) at vmspace_alloc+0x2d
Poul-Henning> vmspace_fork(c035c940,cbb9ad24,c0331f84,cbb9ab00,c0331d60) at 
vmspace_fork+0x4d
Poul-Henning> vm_forkproc(c0332080,cbb9ab00,cbb9ac00,20014) at vm_forkproc+0xc6

This seems due to naming all of the vm map locks as "thrd_sleep." The
locks of vm maps should have their own hierarachy. For instance, lock
the map of a process vm space first, then lock the kernel_map.

-- 
Seigo Tanimura <[EMAIL PROTECTED]> <[EMAIL PROTECTED]>

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: -current lock warning...

2002-03-16 Thread Hiten Pandya 

--- Poul-Henning Kamp <[EMAIL PROTECTED]> wrote:
> >I haven't seen this.  I built a kernel today, and I have a dual processor
> >machine.  Are you using any special kernel options, such as VFS_BIO_DEBUG
> >or something, or am I talking nuts? :)
> 
> Well, I have.  On a single CPU net-booting -current.  No.  Yes :-)

Although I am still getting the following lock problems when I shut
the system down:

lock order reversal
 1st 0xc036afc0 allproc @ ../../../kern/vfs_syscalls.c:452
 2nd 0xc7ecce34 filedesc structure @ ../../../kern/vfs_syscalls.c:457

I think I will have to check out lock problems you are getting today,
once I finish my breakfast. :-)

__
Do You Yahoo!?
Yahoo! Sports - live college hoops coverage
http://sports.yahoo.com/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: -current lock warning...

2002-03-16 Thread Poul-Henning Kamp 

In message <[EMAIL PROTECTED]>, Hiten Pandya w
rites:

>I haven't seen this.  I built a kernel today, and I have a dual processor
>machine.  Are you using any special kernel options, such as VFS_BIO_DEBUG
>or something, or am I talking nuts? :)

Well, I have.  On a single CPU net-booting -current.  No.  Yes :-)

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: -current lock warning...

2002-03-16 Thread Hiten Pandya 

--- Poul-Henning Kamp <[EMAIL PROTECTED]> wrote:
> I get this one on every single boot.  We're not shipping the snapshot
> with that in place, right ?
> 
> real memory  = 268423168 (262132K bytes)
> avail memory = 257003520 (250980K bytes)
> acquiring duplicate lock of same type: "thrd_sleep"
>  1st @ ../../../vm/vm_map.c:2288
>  2nd @ ../../../vm/vm_kern.c:172
> Debugger("witness_lock")
> Stopped at  Debugger+0x40:  xorl%eax,%eax
> db> trace
> Debugger(c02e9ace) at Debugger+0x40
> witness_lock(c038afe4,8,c02f8440,ac,c038afb4) at witness_lock+0x546
> _sx_xlock(c038afe4,c02f8440,ac,bfc0,c0435c5c) at _sx_xlock+0xa1
> _vm_map_lock(c038afb4,c02f8440,ac,c034a840,1) at _vm_map_lock+0x16
> kmem_alloc(c038afb4,3000,c0e41a00,0,c02fa434) at kmem_alloc+0x41
> _zget(c0e41a00,bfc0,0,c0435cd0,c0281769) at _zget+0xfa
> zalloc(c0e41a00,c034a840,1,c02f8630,a7) at zalloc+0x3b
> vmspace_alloc(0,bfc0,c035c940,c02f8630,8f0) at vmspace_alloc+0x2d
> vmspace_fork(c035c940,cbb9ad24,c0331f84,cbb9ab00,c0331d60) at
> vmspace_fork+0x4d
> vm_forkproc(c0332080,cbb9ab00,cbb9ac00,20014) at vm_forkproc+0xc6
> fork1(c0332080,20014,c0332130) at fork1+0xd58
> create_init(0,432c00,432000,0,c012368c) at create_init+0x17
> mi_startup() at mi_startup+0x90
> begin() at begin+0x43
> db> 

I haven't seen this.  I built a kernel today, and I have a dual processor
machine.  Are you using any special kernel options, such as VFS_BIO_DEBUG
or something, or am I talking nuts? :)

Regards,

  -- Hiten

__
Do You Yahoo!?
Yahoo! Sports - live college hoops coverage
http://sports.yahoo.com/

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message