Re: svn commit: r273966 - in head: share/man/man9 sys/kern sys/sys

2014-11-03 Thread Attilio Rao
On Sun, Nov 2, 2014 at 11:49 PM, Konstantin Belousov
 wrote:
> On Sun, Nov 02, 2014 at 11:37:57PM +0100, Attilio Rao wrote:
>> On Sun, Nov 2, 2014 at 10:38 PM, Konstantin Belousov
>>  wrote:
>> > On Sun, Nov 02, 2014 at 10:17:26PM +0100, Attilio Rao wrote:
>> >> I think that your initial patch (what is in head now) is a better 
>> >> approach.
>> >> I would just make it a lockinit() flag to make it less alien to the KPI.
>> >>
>> >
>> > Ok.
>> >
>> > Can you explain what would the proposed lockinit() flag do ? What should
>> > it change comparing with the current code ?
>>
>> You now provide LK_NODDLKTREAT on a call basis.
>> The lockinit() flag would embed this into the lock attribute and make
>> it always real, without the need for the callers to provide
>> LK_NODDLKTREAT on a call basis.
>
> Am I reading your proposal right ? Do you mean, that for all vnodes,
> I should disable exclusive starvation avoidance at all ? I completely
> disagree with this.

No, I'm saying that we should support doing this at the KPI level. Not
that you should enable this for all the vnodes.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r273966 - in head: share/man/man9 sys/kern sys/sys

2014-11-02 Thread Attilio Rao
On Sun, Nov 2, 2014 at 10:38 PM, Konstantin Belousov
 wrote:
> On Sun, Nov 02, 2014 at 10:17:26PM +0100, Attilio Rao wrote:
>> On Sun, Nov 2, 2014 at 8:10 PM, Konstantin Belousov  
>> wrote:
>> > On Sun, Nov 02, 2014 at 06:53:44PM +0100, Attilio Rao wrote:
>> >> > I did not proposed to verify owner chain.  I said that it is easy to
>> >> > record the locks owned by current thread, only for current thread
>> >> > consumption.  Below is the prototype.
>> >>
>> >> I think it is too expensive, think that this must happen for every shared 
>> >> lock.
>> >> I know we may not be using too many shared locks on lockmgr right now,
>> >> but it is not a good reason to make  shared lock bloated and more
>> >> expensive on lockmgr.
>> >
>> > It can be significantly simplified, if the array of lock pointers is
>> > kept dense.  Then the only non-trivial operation is unlock out of order,
>> > when the array have to be compacted.
>> >
>> > The code adds one write and n reads on shared lock, where n is the
>> > number of shared-locked locks already owned by thread. Typical n is 0
>> > or 1. On unlock, if done in order, the code adds one read; unordered
>> > unlock shuffles array elements. Again, for typical lock nesting of 2,
>> > this means one read and one write, and even this is rare. All reads and
>> > writes are for thread-local memory.
>> >
>> > I am not going to spend any more time on this if people do not consider
>> > the lock tracking worth it.  Otherwise, I will benchmark the patch.
>>
>> I think that your initial patch (what is in head now) is a better approach.
>> I would just make it a lockinit() flag to make it less alien to the KPI.
>>
>
> Ok.
>
> Can you explain what would the proposed lockinit() flag do ? What should
> it change comparing with the current code ?

You now provide LK_NODDLKTREAT on a call basis.
The lockinit() flag would embed this into the lock attribute and make
it always real, without the need for the callers to provide
LK_NODDLKTREAT on a call basis.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r273966 - in head: share/man/man9 sys/kern sys/sys

2014-11-02 Thread Attilio Rao
On Sun, Nov 2, 2014 at 8:10 PM, Konstantin Belousov  wrote:
> On Sun, Nov 02, 2014 at 06:53:44PM +0100, Attilio Rao wrote:
>> > I did not proposed to verify owner chain.  I said that it is easy to
>> > record the locks owned by current thread, only for current thread
>> > consumption.  Below is the prototype.
>>
>> I think it is too expensive, think that this must happen for every shared 
>> lock.
>> I know we may not be using too many shared locks on lockmgr right now,
>> but it is not a good reason to make  shared lock bloated and more
>> expensive on lockmgr.
>
> It can be significantly simplified, if the array of lock pointers is
> kept dense.  Then the only non-trivial operation is unlock out of order,
> when the array have to be compacted.
>
> The code adds one write and n reads on shared lock, where n is the
> number of shared-locked locks already owned by thread. Typical n is 0
> or 1. On unlock, if done in order, the code adds one read; unordered
> unlock shuffles array elements. Again, for typical lock nesting of 2,
> this means one read and one write, and even this is rare. All reads and
> writes are for thread-local memory.
>
> I am not going to spend any more time on this if people do not consider
> the lock tracking worth it.  Otherwise, I will benchmark the patch.

I think that your initial patch (what is in head now) is a better approach.
I would just make it a lockinit() flag to make it less alien to the KPI.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r273966 - in head: share/man/man9 sys/kern sys/sys

2014-11-02 Thread Attilio Rao
On Sun, Nov 2, 2014 at 6:53 PM, Attilio Rao  wrote:
> On Sun, Nov 2, 2014 at 6:49 PM, Konstantin Belousov  
> wrote:
>> On Sun, Nov 02, 2014 at 06:07:20PM +0100, Attilio Rao wrote:
>>> On Sun, Nov 2, 2014 at 5:59 PM, Konstantin Belousov  
>>> wrote:
>>> > It is easy and cheap to record the set of the owned lockmgr locks for
>>> > current thread. I do not believe that we have a situation where more
>>> > than 3 locks are held in one time. To give it some slack, we can record
>>> > 8 locks shared-owned and do the check for recursion in LK_CAN_SHARE().
>>> > If the thread-locks table overflows, we could either panic() or fall
>>> > back to indiscriminative deadlock avoidance (i.e. relying on the sole
>>> > td_lk_slocks value).
>>>
>>> I don't think it is going to be cheap (and certainly it is not viable
>>> for things like sx locks and rwlocks).
>> sx and rw do not implement exclusive starvation avoidance.
>
> rw do.
> It is believed for sx it should not be too helpful, on FreeBSD I've
> never seen an sx that is particulary congested.
> It can be added, however and the algorithm they would use is the same than rw.
>
>>> Checking for the owner chain anytime you acquire the lock is certainly
>>> not I would call cheap.
>>
>> I did not proposed to verify owner chain.  I said that it is easy to
>> record the locks owned by current thread, only for current thread
>> consumption.  Below is the prototype.
>
> I think it is too expensive, think that this must happen for every shared 
> lock.
> I know we may not be using too many shared locks on lockmgr right now,
> but it is not a good reason to make  shared lock bloated and more
> expensive on lockmgr.

And to be honest, I find already wrong the fact that we don't have a
faster path for lockmgr() (ie. inlined atomics).
I think this sloppiness (at least on my side) cames on the fact that
locking overhead is not a major factor for the performance of current
lockmgr users.
That doesn't mean, however, that we should have a slower primitive.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r273966 - in head: share/man/man9 sys/kern sys/sys

2014-11-02 Thread Attilio Rao
On Sun, Nov 2, 2014 at 6:49 PM, Konstantin Belousov  wrote:
> On Sun, Nov 02, 2014 at 06:07:20PM +0100, Attilio Rao wrote:
>> On Sun, Nov 2, 2014 at 5:59 PM, Konstantin Belousov  
>> wrote:
>> > It is easy and cheap to record the set of the owned lockmgr locks for
>> > current thread. I do not believe that we have a situation where more
>> > than 3 locks are held in one time. To give it some slack, we can record
>> > 8 locks shared-owned and do the check for recursion in LK_CAN_SHARE().
>> > If the thread-locks table overflows, we could either panic() or fall
>> > back to indiscriminative deadlock avoidance (i.e. relying on the sole
>> > td_lk_slocks value).
>>
>> I don't think it is going to be cheap (and certainly it is not viable
>> for things like sx locks and rwlocks).
> sx and rw do not implement exclusive starvation avoidance.

rw do.
It is believed for sx it should not be too helpful, on FreeBSD I've
never seen an sx that is particulary congested.
It can be added, however and the algorithm they would use is the same than rw.

>> Checking for the owner chain anytime you acquire the lock is certainly
>> not I would call cheap.
>
> I did not proposed to verify owner chain.  I said that it is easy to
> record the locks owned by current thread, only for current thread
> consumption.  Below is the prototype.

I think it is too expensive, think that this must happen for every shared lock.
I know we may not be using too many shared locks on lockmgr right now,
but it is not a good reason to make  shared lock bloated and more
expensive on lockmgr.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r273966 - in head: share/man/man9 sys/kern sys/sys

2014-11-02 Thread Attilio Rao
On Sun, Nov 2, 2014 at 5:59 PM, Konstantin Belousov  wrote:
> On Sun, Nov 02, 2014 at 05:42:55PM +0100, Attilio Rao wrote:
>> On Sun, Nov 2, 2014 at 5:37 PM, Konstantin Belousov  
>> wrote:
>> > On Sun, Nov 02, 2014 at 04:54:46PM +0100, Attilio Rao wrote:
>> >> On Sun, Nov 2, 2014 at 2:10 PM, Konstantin Belousov  
>> >> wrote:
>> >> > Author: kib
>> >> > Date: Sun Nov  2 13:10:31 2014
>> >> > New Revision: 273966
>> >> > URL: https://svnweb.freebsd.org/changeset/base/273966
>> >> >
>> >> > Log:
>> >> >   Fix two issues with lockmgr(9) LK_CAN_SHARE() test, which determines
>> >> >   whether the shared request for already shared-locked lock could be
>> >> >   granted.  Both problems result in the exclusive locker starvation.
>> >> >
>> >> >   The concurrent exclusive request is indicated by either
>> >> >   LK_EXCLUSIVE_WAITERS or LK_EXCLUSIVE_SPINNERS flags.  The reverse
>> >> >   condition, i.e. no exclusive waiters, must check that both flags are
>> >> >   cleared.
>> >> >
>> >> >   Add a flag LK_NODDLKTREAT for shared lock request to indicate that
>> >> >   current thread guarantees that it does not own the lock in shared
>> >> >   mode.  This turns back the exclusive lock starvation avoidance code;
>> >> >   see man page update for detailed description.
>> >> >
>> >> >   Use LK_NODDLKTREAT when doing lookup(9).
>> >>
>> >> The right way to implement this (selectively disabling writer
>> >> starvation avoidance) must be on a lock-basis.
>> >> So you need a new flag to pass to lockinit(). This is to support it 
>> >> "globaly".
>> > Any vnode (except some very special) does participate in lookup.
>> > So the proposed new flag has to be passed to every lockmgr init call.
>> > Disabling exclusive starvation support for all vnode lock calls is
>> > also wrong.
>> >
>> >> Then, you can pass it on a lockmgr() instance basis (like FreeBSD also
>> >> passes, for example, LK_NOWITNESS).
>> >> You can use it during lookup() calls. Maybe you will need to propagate
>> >> it via the vn_lock() interface.
>> > Well, if you indeed looked at the patch, you would note that this is
>> > exactly what is done.  The flag is passed only to vn_lock() calls
>> > which are coming from lookup (VOP_LOOKUP as well).
>> >
>> > The flag use is safe since deadlock prevention by td_lk_slocks only
>> > matters when the same lock was previously taken by the same thread
>> > which does recursive shared request [*].  td_lk_slocks is the poor and
>> > very indiscriminative way to express this, but might be unavoidable
>> > since shared owners are not recorded.
>>
>> If you have a better way to fix this into a "rich and discriminative"
>> way I'm all ears.
> It is easy and cheap to record the set of the owned lockmgr locks for
> current thread. I do not believe that we have a situation where more
> than 3 locks are held in one time. To give it some slack, we can record
> 8 locks shared-owned and do the check for recursion in LK_CAN_SHARE().
> If the thread-locks table overflows, we could either panic() or fall
> back to indiscriminative deadlock avoidance (i.e. relying on the sole
> td_lk_slocks value).

I don't think it is going to be cheap (and certainly it is not viable
for things like sx locks and rwlocks).
Checking for the owner chain anytime you acquire the lock is certainly
not I would call cheap.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r273966 - in head: share/man/man9 sys/kern sys/sys

2014-11-02 Thread Attilio Rao
On Sun, Nov 2, 2014 at 5:37 PM, Konstantin Belousov  wrote:
> On Sun, Nov 02, 2014 at 04:54:46PM +0100, Attilio Rao wrote:
>> On Sun, Nov 2, 2014 at 2:10 PM, Konstantin Belousov  wrote:
>> > Author: kib
>> > Date: Sun Nov  2 13:10:31 2014
>> > New Revision: 273966
>> > URL: https://svnweb.freebsd.org/changeset/base/273966
>> >
>> > Log:
>> >   Fix two issues with lockmgr(9) LK_CAN_SHARE() test, which determines
>> >   whether the shared request for already shared-locked lock could be
>> >   granted.  Both problems result in the exclusive locker starvation.
>> >
>> >   The concurrent exclusive request is indicated by either
>> >   LK_EXCLUSIVE_WAITERS or LK_EXCLUSIVE_SPINNERS flags.  The reverse
>> >   condition, i.e. no exclusive waiters, must check that both flags are
>> >   cleared.
>> >
>> >   Add a flag LK_NODDLKTREAT for shared lock request to indicate that
>> >   current thread guarantees that it does not own the lock in shared
>> >   mode.  This turns back the exclusive lock starvation avoidance code;
>> >   see man page update for detailed description.
>> >
>> >   Use LK_NODDLKTREAT when doing lookup(9).
>>
>> The right way to implement this (selectively disabling writer
>> starvation avoidance) must be on a lock-basis.
>> So you need a new flag to pass to lockinit(). This is to support it 
>> "globaly".
> Any vnode (except some very special) does participate in lookup.
> So the proposed new flag has to be passed to every lockmgr init call.
> Disabling exclusive starvation support for all vnode lock calls is
> also wrong.
>
>> Then, you can pass it on a lockmgr() instance basis (like FreeBSD also
>> passes, for example, LK_NOWITNESS).
>> You can use it during lookup() calls. Maybe you will need to propagate
>> it via the vn_lock() interface.
> Well, if you indeed looked at the patch, you would note that this is
> exactly what is done.  The flag is passed only to vn_lock() calls
> which are coming from lookup (VOP_LOOKUP as well).
>
> The flag use is safe since deadlock prevention by td_lk_slocks only
> matters when the same lock was previously taken by the same thread
> which does recursive shared request [*].  td_lk_slocks is the poor and
> very indiscriminative way to express this, but might be unavoidable
> since shared owners are not recorded.

If you have a better way to fix this into a "rich and discriminative"
way I'm all ears.

> * I believe there are also situation (like vnode locking in page fault)
> where it matters in way similar to TDP_DEADLKTREAT without flag being
> set, but lookup() cannot expose them.
>
> Now, thread calling namei() cannot own shared lock on the vnode it
> tries to lock, since such code will cause deadlock for many reason. As
> such, td_lk_slocks counter is not relevant for the vn_lock() calls from
> lookup. More, when vn_lock() call is performed, the counter is always >=
> 1, due to the lookup already owning the lock for the different (parent)
> vnode. The last fact means that the exclusive starvation happen.
>
>>
>> The patch will be bigger but much cleaner and more correct than what
>> is in head now.
> I do not understand what would change except adding useless flag
> to init.

I didn't look into details of the original problem, but this would
cleanup semantics.
As it is now it is just confusing for people.
I think that we should invest some time in cleaning up (or keep clean)
interface that will be used into drivers and thirdy part code.
Right now the new flag is just a shot in the dark.

As always, feel free to skip my suggestions.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r273966 - in head: share/man/man9 sys/kern sys/sys

2014-11-02 Thread Attilio Rao
On Sun, Nov 2, 2014 at 2:10 PM, Konstantin Belousov  wrote:
> Author: kib
> Date: Sun Nov  2 13:10:31 2014
> New Revision: 273966
> URL: https://svnweb.freebsd.org/changeset/base/273966
>
> Log:
>   Fix two issues with lockmgr(9) LK_CAN_SHARE() test, which determines
>   whether the shared request for already shared-locked lock could be
>   granted.  Both problems result in the exclusive locker starvation.
>
>   The concurrent exclusive request is indicated by either
>   LK_EXCLUSIVE_WAITERS or LK_EXCLUSIVE_SPINNERS flags.  The reverse
>   condition, i.e. no exclusive waiters, must check that both flags are
>   cleared.
>
>   Add a flag LK_NODDLKTREAT for shared lock request to indicate that
>   current thread guarantees that it does not own the lock in shared
>   mode.  This turns back the exclusive lock starvation avoidance code;
>   see man page update for detailed description.
>
>   Use LK_NODDLKTREAT when doing lookup(9).

The right way to implement this (selectively disabling writer
starvation avoidance) must be on a lock-basis.
So you need a new flag to pass to lockinit(). This is to support it "globaly".
Then, you can pass it on a lockmgr() instance basis (like FreeBSD also
passes, for example, LK_NOWITNESS).
You can use it during lookup() calls. Maybe you will need to propagate
it via the vn_lock() interface.

The patch will be bigger but much cleaner and more correct than what
is in head now.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r273966 - in head: share/man/man9 sys/kern sys/sys

2014-11-02 Thread Attilio Rao
On Sun, Nov 2, 2014 at 2:10 PM, Konstantin Belousov  wrote:
> Author: kib
> Date: Sun Nov  2 13:10:31 2014
> New Revision: 273966
> URL: https://svnweb.freebsd.org/changeset/base/273966
>
> Log:
>   Fix two issues with lockmgr(9) LK_CAN_SHARE() test, which determines
>   whether the shared request for already shared-locked lock could be
>   granted.  Both problems result in the exclusive locker starvation.
>
>   The concurrent exclusive request is indicated by either
>   LK_EXCLUSIVE_WAITERS or LK_EXCLUSIVE_SPINNERS flags.  The reverse
>   condition, i.e. no exclusive waiters, must check that both flags are
>   cleared.
>
>   Add a flag LK_NODDLKTREAT for shared lock request to indicate that
>   current thread guarantees that it does not own the lock in shared
>   mode.  This turns back the exclusive lock starvation avoidance code;
>   see man page update for detailed description.
>
>   Use LK_NODDLKTREAT when doing lookup(9).
>
>   Reported and tested by:   pho
>   No objections from:   attilio

Not true, I objected to the part of the patch that modifies the
semantic for changing the writer starvation avoidance technique.
As I said the td_lk_slocks must work for every consumer of lockmgr, it
is not specific to any usage in the kernel, including namei() or
lookups. This is an hinerent property of lockmgr, it has not strict
relation with its callers.
Now you can deadlock lockmgr if you use recursive shared acquisition
in a path that has not been identified.

This part of the patch should be reverted.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r267858 - in head/sys/dev: virtio/balloon xen/balloon

2014-06-25 Thread Attilio Rao
On Wed, Jun 25, 2014 at 5:16 PM, Alfred Perlstein  wrote:
> On 6/25/14 5:41 AM, Attilio Rao wrote:
>>
>> On Wed, Jun 25, 2014 at 2:09 PM, Gleb Smirnoff 
>> wrote:
>>>
>>> On Wed, Jun 25, 2014 at 01:58:29PM +0200, Attilio Rao wrote:
>>> A> > Log:
>>> A> >   xen/virtio: fix balloon drivers to not mark pages as WIRED
>>> A> >
>>> A> >   Prevent the Xen and VirtIO balloon drivers from marking pages as
>>> A> >   wired. This prevents them from increasing the system wired page
>>> count,
>>> A> >   which can lead to mlock failing because of hitting the limit in
>>> A> >   vm.max_wired.
>>> A>
>>> A> This change is conceptually wrong.
>>> A> The pages balloon is allocating are unmanaged and they should be wired
>>> A> by definition. Alan and I are considering enforcing this (mandatory
>>> A> wired pages for unmanaged pages allocation) directly in the KPI.
>>> A> This in practice just seem an artifact to deal with scarce  wired
>>> A> memory limit. I suggest that for the XEN case this limit gets bumped
>>> A> rather relying on similar type of hacks.
>>>
>>> Proper limit would be to count pages wired by userland via mlock(2)
>>> and enforce limit only on those pages. Pages wired by kernel should
>>> be either unlimited or controled by a separate limit.
>>
>> FWIW, I mostly agree with this. I think that the kernel and userland
>> limits should be split apart. But for the time being, rising the limit
>> is better.
>>
>> Attilio
>>
>>
> Can you explain?  I would think that if you were designing some kind of
> embedded device you would want to know exactly how much locked pages there
> are overall, not just in userland.
>
> Meaning you would not want to overcommit and have too many locked pages due
> to kernel+user.

Well, assuming you trace them indipendently I don't think this is
going to be problematic to aggregate them, is it?

As far as I understand it, right now we have RMEM_LIMIT to someway
limit per-process amount of wired memory and finally max_wired as a
global accounted wired memory limit.

I think that the idea now is that RMEM_LIMIT is enough to correctly
control all the front-end check, coming from untrusted sources
(userland, non-privileged syscalls like mlock(), mmap(), etc.).
Possibly that's not always the case and I think that the hypervisor
can be a fair example of this.

Having "more granular" accountability, means that rather than having a
global limit (or, rather, along with it) we can grow a per-process
limit to control kernel-allocated wired memory.

> Perhaps that needs an API as well?

I don't have anything in my mind yet. My initial point was more trying
to get a better semantic on a paradigm that is at least dangerous.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r267858 - in head/sys/dev: virtio/balloon xen/balloon

2014-06-25 Thread Attilio Rao
On Wed, Jun 25, 2014 at 4:06 PM, Roger Pau Monné  wrote:
> On 25/06/14 15:44, Attilio Rao wrote:
>> On Wed, Jun 25, 2014 at 3:29 PM, Roger Pau Monné  wrote:
>>> On 25/06/14 13:58, Attilio Rao wrote:
>>>> On Wed, Jun 25, 2014 at 11:51 AM, Roger Pau Monné  
>>>> wrote:
>>>>> Author: royger
>>>>> Date: Wed Jun 25 09:51:08 2014
>>>>> New Revision: 267858
>>>>> URL: http://svnweb.freebsd.org/changeset/base/267858
>>>>>
>>>>> Log:
>>>>>   xen/virtio: fix balloon drivers to not mark pages as WIRED
>>>>>
>>>>>   Prevent the Xen and VirtIO balloon drivers from marking pages as
>>>>>   wired. This prevents them from increasing the system wired page count,
>>>>>   which can lead to mlock failing because of hitting the limit in
>>>>>   vm.max_wired.
>>>>
>>>> This change is conceptually wrong.
>>>> The pages balloon is allocating are unmanaged and they should be wired
>>>> by definition. Alan and I are considering enforcing this (mandatory
>>>> wired pages for unmanaged pages allocation) directly in the KPI.
>>>> This in practice just seem an artifact to deal with scarce  wired
>>>> memory limit. I suggest that for the XEN case this limit gets bumped
>>>> rather relying on similar type of hacks.
>>>
>>> IMHO, marking them as wired seems wrong too, those pages are not wired,
>>> they are simply not there any more. This was discussed in:
>>
>> I'm not entirely sure what do you mean with "not there anymore", so
>> I'm just guessing and I assume that you mean "pages are not collected
>> in any vm object and then they are not referenced in any pagequeue".
>> By extension this also matches what "unmanaged page" means.
>>
>> If the page is unmanaged it means that the pagedaemon won't see it, so
>> they won't be swapped out anyway. Wiring them it will enforce more
>> sanity checking on the page. The max_wired concern may be real,
>> however, please see below.
>
> What I meant by "not there" is that the physical addresses associated
> with the pages are no longer valid (it's a hole in the physical memory
> map). To me it seems conceptually wrong to account those pages as wired.

They are not. They are at all effect wired (even if with a slightly
different semantic), meaning wired as also not-pageable. The fact that
you won't mark them wired will just weaken the accounting,
debuggability and tracing side.

> Thinking from a user point of view for example, if I'm inside of a VM
> that has ballooned down, I will see a huge amount of memory marked as
> wired on the top utility, which seems wrong, those pages are simply not
> there anymore, and as so, they shouldn't show up in stats.
>
> It also makes it really hard to know which memory is really wired and in
> use by the system, as opposed to the wired memory used by the balloon
> driver.

I think this is really the problem with your approach. Now memory will
not be reachable by the pagedaemon and yet you won't see it showing up
anywhere. Memory leaks could go unnoticed, to some extent. I don't
think that's really what you want.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r267858 - in head/sys/dev: virtio/balloon xen/balloon

2014-06-25 Thread Attilio Rao
On Wed, Jun 25, 2014 at 3:29 PM, Roger Pau Monné  wrote:
> On 25/06/14 13:58, Attilio Rao wrote:
>> On Wed, Jun 25, 2014 at 11:51 AM, Roger Pau Monné  wrote:
>>> Author: royger
>>> Date: Wed Jun 25 09:51:08 2014
>>> New Revision: 267858
>>> URL: http://svnweb.freebsd.org/changeset/base/267858
>>>
>>> Log:
>>>   xen/virtio: fix balloon drivers to not mark pages as WIRED
>>>
>>>   Prevent the Xen and VirtIO balloon drivers from marking pages as
>>>   wired. This prevents them from increasing the system wired page count,
>>>   which can lead to mlock failing because of hitting the limit in
>>>   vm.max_wired.
>>
>> This change is conceptually wrong.
>> The pages balloon is allocating are unmanaged and they should be wired
>> by definition. Alan and I are considering enforcing this (mandatory
>> wired pages for unmanaged pages allocation) directly in the KPI.
>> This in practice just seem an artifact to deal with scarce  wired
>> memory limit. I suggest that for the XEN case this limit gets bumped
>> rather relying on similar type of hacks.
>
> IMHO, marking them as wired seems wrong too, those pages are not wired,
> they are simply not there any more. This was discussed in:

I'm not entirely sure what do you mean with "not there anymore", so
I'm just guessing and I assume that you mean "pages are not collected
in any vm object and then they are not referenced in any pagequeue".
By extension this also matches what "unmanaged page" means.

If the page is unmanaged it means that the pagedaemon won't see it, so
they won't be swapped out anyway. Wiring them it will enforce more
sanity checking on the page. The max_wired concern may be real,
however, please see below.

> http://lists.freebsd.org/pipermail/freebsd-virtualization/2014-June/002643.html
>
> If there's consensus I will revert the change, but I would say that
> increasing vm.max_wired for VMs is also a gross hack.

Why? If VM needs more wired memory I assume that we can tune up the
default value of max_wired?

I think that however your case makes an interesting point: if we want
to make unmanaged pages as inherently wired, we likely need a little
bit higher max_wired value. When I completed a patch for this, pho@
couldn't reproduce any similar issue even with stress-testing (and
also, the places to allocate unmanaged pages and not requesting
VM_ALLOC_WIRED were very little, almost 0, with the exception of
vm_page_alloc_contig() calls) but I think it is a valid proposition.

However I would still like to have more control on kernel-specific
wired memory for processes. I'm for example thinking to ARC vs. buffer
cache, where I expect the wired memory consumption to be much bigger
for the former case.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

Re: svn commit: r267858 - in head/sys/dev: virtio/balloon xen/balloon

2014-06-25 Thread Attilio Rao
On Wed, Jun 25, 2014 at 2:09 PM, Gleb Smirnoff  wrote:
> On Wed, Jun 25, 2014 at 01:58:29PM +0200, Attilio Rao wrote:
> A> > Log:
> A> >   xen/virtio: fix balloon drivers to not mark pages as WIRED
> A> >
> A> >   Prevent the Xen and VirtIO balloon drivers from marking pages as
> A> >   wired. This prevents them from increasing the system wired page count,
> A> >   which can lead to mlock failing because of hitting the limit in
> A> >   vm.max_wired.
> A>
> A> This change is conceptually wrong.
> A> The pages balloon is allocating are unmanaged and they should be wired
> A> by definition. Alan and I are considering enforcing this (mandatory
> A> wired pages for unmanaged pages allocation) directly in the KPI.
> A> This in practice just seem an artifact to deal with scarce  wired
> A> memory limit. I suggest that for the XEN case this limit gets bumped
> A> rather relying on similar type of hacks.
>
> Proper limit would be to count pages wired by userland via mlock(2)
> and enforce limit only on those pages. Pages wired by kernel should
> be either unlimited or controled by a separate limit.

FWIW, I mostly agree with this. I think that the kernel and userland
limits should be split apart. But for the time being, rising the limit
is better.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r267858 - in head/sys/dev: virtio/balloon xen/balloon

2014-06-25 Thread Attilio Rao
On Wed, Jun 25, 2014 at 11:51 AM, Roger Pau Monné  wrote:
> Author: royger
> Date: Wed Jun 25 09:51:08 2014
> New Revision: 267858
> URL: http://svnweb.freebsd.org/changeset/base/267858
>
> Log:
>   xen/virtio: fix balloon drivers to not mark pages as WIRED
>
>   Prevent the Xen and VirtIO balloon drivers from marking pages as
>   wired. This prevents them from increasing the system wired page count,
>   which can lead to mlock failing because of hitting the limit in
>   vm.max_wired.

This change is conceptually wrong.
The pages balloon is allocating are unmanaged and they should be wired
by definition. Alan and I are considering enforcing this (mandatory
wired pages for unmanaged pages allocation) directly in the KPI.
This in practice just seem an artifact to deal with scarce  wired
memory limit. I suggest that for the XEN case this limit gets bumped
rather relying on similar type of hacks.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r267820 - head/sys/kern

2014-06-24 Thread Attilio Rao
Author: attilio
Date: Tue Jun 24 15:16:55 2014
New Revision: 267820
URL: http://svnweb.freebsd.org/changeset/base/267820

Log:
  sysctl subsystem uses sxlocks so avoid to setup dynamic sysctl nodes
  before sleepinit() has been fully executed in the SLEEPQUEUE_PROFILING
  case.
  
  Sponsored by: EMC / Isilon storage division

Modified:
  head/sys/kern/subr_sleepqueue.c

Modified: head/sys/kern/subr_sleepqueue.c
==
--- head/sys/kern/subr_sleepqueue.c Tue Jun 24 15:03:00 2014
(r267819)
+++ head/sys/kern/subr_sleepqueue.c Tue Jun 24 15:16:55 2014
(r267820)
@@ -166,24 +166,20 @@ SDT_PROBE_DECLARE(sched, , , sleep);
 SDT_PROBE_DECLARE(sched, , , wakeup);
 
 /*
- * Early initialization of sleep queues that is called from the sleepinit()
- * SYSINIT.
+ * Initialize SLEEPQUEUE_PROFILING specific sysctl nodes.
+ * Note that it must happen after sleepinit() has been fully executed, so
+ * it must happen after SI_SUB_KMEM SYSINIT() subsystem setup.
  */
-void
-init_sleepqueues(void)
-{
 #ifdef SLEEPQUEUE_PROFILING
-   struct sysctl_oid *chain_oid;
+static void
+init_sleepqueue_profiling(void)
+{
char chain_name[10];
-#endif
-   int i;
+   struct sysctl_oid *chain_oid;
+   u_int i;
 
for (i = 0; i < SC_TABLESIZE; i++) {
-   LIST_INIT(&sleepq_chains[i].sc_queues);
-   mtx_init(&sleepq_chains[i].sc_lock, "sleepq chain", NULL,
-   MTX_SPIN | MTX_RECURSE);
-#ifdef SLEEPQUEUE_PROFILING
-   snprintf(chain_name, sizeof(chain_name), "%d", i);
+   snprintf(chain_name, sizeof(chain_name), "%u", i);
chain_oid = SYSCTL_ADD_NODE(NULL, 
SYSCTL_STATIC_CHILDREN(_debug_sleepq_chains), OID_AUTO,
chain_name, CTLFLAG_RD, NULL, "sleepq chain stats");
@@ -192,7 +188,26 @@ init_sleepqueues(void)
SYSCTL_ADD_UINT(NULL, SYSCTL_CHILDREN(chain_oid), OID_AUTO,
"max_depth", CTLFLAG_RD, &sleepq_chains[i].sc_max_depth, 0,
NULL);
+   }
+}
+
+SYSINIT(sleepqueue_profiling, SI_SUB_LOCK, SI_ORDER_ANY,
+init_sleepqueue_profiling, NULL);
 #endif
+
+/*
+ * Early initialization of sleep queues that is called from the sleepinit()
+ * SYSINIT.
+ */
+void
+init_sleepqueues(void)
+{
+   int i;
+
+   for (i = 0; i < SC_TABLESIZE; i++) {
+   LIST_INIT(&sleepq_chains[i].sc_queues);
+   mtx_init(&sleepq_chains[i].sc_lock, "sleepq chain", NULL,
+   MTX_SPIN | MTX_RECURSE);
}
sleepq_zone = uma_zcreate("SLEEPQUEUE", sizeof(struct sleepqueue),
 #ifdef INVARIANTS
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r267651 - in head: share/man/man4 sys/dev/cpuctl sys/sys usr.sbin/cpucontrol

2014-06-20 Thread Attilio Rao
On Jun 20, 2014 9:49 AM, "Konstantin Belousov"  wrote:
>
> On Fri, Jun 20, 2014 at 08:12:59AM +0200, Attilio Rao wrote:
> > On Fri, Jun 20, 2014 at 6:08 AM, Konstantin Belousov
> >  wrote:
> > > On Thu, Jun 19, 2014 at 09:54:41PM +, Attilio Rao wrote:
> > >> Author: attilio
> > >> Date: Thu Jun 19 21:54:41 2014
> > >> New Revision: 267651
> > >> URL: http://svnweb.freebsd.org/changeset/base/267651
> > >>
> > >> Log:
> > >>   Following comments in r242565 add the possibility to specify ecx
when
> > >>   performing cpuid calls.
> > >>   Add also a new way to specify the level type to cpucontrol(8) as
> > >>   reported in the manpage.
> > >>
> > >>   Sponsored by:   EMC / Isilon storage division
> > >>   Reviewed by:bdrewery, gcooper
> > >>   Testerd by: bdrewery
> > >> Modified: head/sys/sys/cpuctl.h
> > >>
==
> > >> --- head/sys/sys/cpuctl.h Thu Jun 19 21:05:07 2014
 (r267650)
> > >> +++ head/sys/sys/cpuctl.h Thu Jun 19 21:54:41 2014
 (r267651)
> > >> @@ -35,7 +35,8 @@ typedef struct {
> > >>  } cpuctl_msr_args_t;
> > >>
> > >>  typedef struct {
> > >> - int level;  /* CPUID level */
> > >> + int level;  /* CPUID level */
> > >> + int level_type; /* CPUID level type */
> > >>   uint32_tdata[4];
> > >>  } cpuctl_cpuid_args_t;
> > >>
> > >> @@ -50,5 +51,6 @@ typedef struct {
> > >>  #define  CPUCTL_UPDATE   _IOWR('c', 4, cpuctl_update_args_t)
> > >>  #define  CPUCTL_MSRSBIT  _IOWR('c', 5, cpuctl_msr_args_t)
> > >>  #define  CPUCTL_MSRCBIT  _IOWR('c', 6, cpuctl_msr_args_t)
> > >> +#define  CPUCTL_CPUID_COUNT _IOWR('c', 7, cpuctl_cpuid_args_t)
> > >>
> > >>  #endif /* _CPUCTL_H_ */
> > >
> > > The cpuctl(4) is used by third-party code, and this change breaks its
> > > ABI. The numeric value for CPUCTL_CPUID is changed, which means that
> > > old binaries call non-existing ioctl now. This is at least a visible
> > > breakage, since the argument for the ioctl changed the layout as well.
> > >
> > > The following patch restored the CPUCTL_CPUID for me.  I considered
> > > naming its argument differently, instead of renaming the argument
> > > of CPUCTL_CPUID_COUNT (which you tried to do ?), but decided not,
> > > to preserve the API as well.
> >
> > No, breaking the ABI is fine for -CURRENT so I don't see why we need
the bloat.
>
> No, breaking ABI is not fine at all, be it CURRENT or not. We try to
> keep the ABI stable, doing costly measures like symbol versioning, where
> applicable. Since this is a change to ABI for kernel interface, it
> cannot be covered by symver tricks and must be done in kernel.
>
> > I don't plan on MFC this patch. If I need to (or any user requests
> > that) I will do with the appropriate ABI-compliant way (ie. adding a
> > new argument like this one).
>
> Besides the same-world ABI issue, people run jails with lower world
> version on higher-versioned kernels.

I think you have a point,I forgot we have different requirements for ABI
and KBI/KPI in -CURRENT per jail. I will make a patch for it when I get
back home in few hours.

Attilio
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r267651 - in head: share/man/man4 sys/dev/cpuctl sys/sys usr.sbin/cpucontrol

2014-06-19 Thread Attilio Rao
On Fri, Jun 20, 2014 at 6:08 AM, Konstantin Belousov
 wrote:
> On Thu, Jun 19, 2014 at 09:54:41PM +0000, Attilio Rao wrote:
>> Author: attilio
>> Date: Thu Jun 19 21:54:41 2014
>> New Revision: 267651
>> URL: http://svnweb.freebsd.org/changeset/base/267651
>>
>> Log:
>>   Following comments in r242565 add the possibility to specify ecx when
>>   performing cpuid calls.
>>   Add also a new way to specify the level type to cpucontrol(8) as
>>   reported in the manpage.
>>
>>   Sponsored by:   EMC / Isilon storage division
>>   Reviewed by:bdrewery, gcooper
>>   Testerd by: bdrewery
>> Modified: head/sys/sys/cpuctl.h
>> ==
>> --- head/sys/sys/cpuctl.h Thu Jun 19 21:05:07 2014(r267650)
>> +++ head/sys/sys/cpuctl.h Thu Jun 19 21:54:41 2014(r267651)
>> @@ -35,7 +35,8 @@ typedef struct {
>>  } cpuctl_msr_args_t;
>>
>>  typedef struct {
>> - int level;  /* CPUID level */
>> + int level;  /* CPUID level */
>> + int level_type; /* CPUID level type */
>>   uint32_tdata[4];
>>  } cpuctl_cpuid_args_t;
>>
>> @@ -50,5 +51,6 @@ typedef struct {
>>  #define  CPUCTL_UPDATE   _IOWR('c', 4, cpuctl_update_args_t)
>>  #define  CPUCTL_MSRSBIT  _IOWR('c', 5, cpuctl_msr_args_t)
>>  #define  CPUCTL_MSRCBIT  _IOWR('c', 6, cpuctl_msr_args_t)
>> +#define  CPUCTL_CPUID_COUNT _IOWR('c', 7, cpuctl_cpuid_args_t)
>>
>>  #endif /* _CPUCTL_H_ */
>
> The cpuctl(4) is used by third-party code, and this change breaks its
> ABI. The numeric value for CPUCTL_CPUID is changed, which means that
> old binaries call non-existing ioctl now. This is at least a visible
> breakage, since the argument for the ioctl changed the layout as well.
>
> The following patch restored the CPUCTL_CPUID for me.  I considered
> naming its argument differently, instead of renaming the argument
> of CPUCTL_CPUID_COUNT (which you tried to do ?), but decided not,
> to preserve the API as well.

No, breaking the ABI is fine for -CURRENT so I don't see why we need the bloat.
I don't plan on MFC this patch. If I need to (or any user requests
that) I will do with the appropriate ABI-compliant way (ie. adding a
new argument like this one).

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r267651 - in head: share/man/man4 sys/dev/cpuctl sys/sys usr.sbin/cpucontrol

2014-06-19 Thread Attilio Rao
Author: attilio
Date: Thu Jun 19 21:54:41 2014
New Revision: 267651
URL: http://svnweb.freebsd.org/changeset/base/267651

Log:
  Following comments in r242565 add the possibility to specify ecx when
  performing cpuid calls.
  Add also a new way to specify the level type to cpucontrol(8) as
  reported in the manpage.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  bdrewery, gcooper
  Testerd by:   bdrewery

Modified:
  head/share/man/man4/cpuctl.4
  head/sys/dev/cpuctl/cpuctl.c
  head/sys/sys/cpuctl.h
  head/usr.sbin/cpucontrol/cpucontrol.8
  head/usr.sbin/cpucontrol/cpucontrol.c

Modified: head/share/man/man4/cpuctl.4
==
--- head/share/man/man4/cpuctl.4Thu Jun 19 21:05:07 2014
(r267650)
+++ head/share/man/man4/cpuctl.4Thu Jun 19 21:54:41 2014
(r267651)
@@ -86,19 +86,27 @@ Set/clear MSR bits according to the mask
 .Va data
 field.
 .It Dv CPUCTL_CPUID Fa cpuctl_cpuid_args_t *args
+.It Dv CPUCTL_CPUID_COUNT Fa cpuctl_cpuid_args_t *args
 Retrieve CPUID information.
 Arguments are supplied in
 the following struct:
 .Bd -literal
 typedef struct {
-   int level;  /* CPUID level */
+   int level;  /* CPUID level */
+   int level_type; /* CPUID level type */
uint32_tdata[4];
 } cpuctl_cpuid_args_t;
 .Ed
 .Pp
 The
 .Va level
-field indicates the CPUID level to retrieve information for, while the
+field indicates the CPUID level to retrieve.
+The
+.Va level_type
+field indicates the CPUID level type to retrieve.
+It is overriden to 0 for
+.Va CPUCTL_CPUID .
+Finally, the
 .Va data
 field is used to store the received CPUID data.
 .It Dv CPUCTL_UPDATE cpuctl_update_args_t *args

Modified: head/sys/dev/cpuctl/cpuctl.c
==
--- head/sys/dev/cpuctl/cpuctl.cThu Jun 19 21:05:07 2014
(r267650)
+++ head/sys/dev/cpuctl/cpuctl.cThu Jun 19 21:54:41 2014
(r267651)
@@ -69,6 +69,8 @@ static int cpuctl_do_msr(int cpu, cpuctl
 struct thread *td);
 static int cpuctl_do_cpuid(int cpu, cpuctl_cpuid_args_t *data,
 struct thread *td);
+static int cpuctl_do_cpuid_count(int cpu, cpuctl_cpuid_args_t *data,
+struct thread *td);
 static int cpuctl_do_update(int cpu, cpuctl_update_args_t *data,
 struct thread *td);
 static int update_intel(int cpu, cpuctl_update_args_t *args,
@@ -177,6 +179,10 @@ cpuctl_ioctl(struct cdev *dev, u_long cm
goto fail;
ret = cpuctl_do_update(cpu, (cpuctl_update_args_t *)data, td);
break;
+   case CPUCTL_CPUID_COUNT:
+   ret = cpuctl_do_cpuid_count(cpu, (cpuctl_cpuid_args_t *)data,
+   td);
+   break;
default:
ret = EINVAL;
break;
@@ -189,7 +195,7 @@ fail:
  * Actually perform cpuid operation.
  */
 static int
-cpuctl_do_cpuid(int cpu, cpuctl_cpuid_args_t *data, struct thread *td)
+cpuctl_do_cpuid_count(int cpu, cpuctl_cpuid_args_t *data, struct thread *td)
 {
int is_bound = 0;
int oldcpu;
@@ -199,16 +205,25 @@ cpuctl_do_cpuid(int cpu, cpuctl_cpuid_ar
 
/* Explicitly clear cpuid data to avoid returning stale info. */
bzero(data->data, sizeof(data->data));
-   DPRINTF("[cpuctl,%d]: retriving cpuid level %#0x for %d cpu\n",
-   __LINE__, data->level, cpu);
+   DPRINTF("[cpuctl,%d]: retrieving cpuid lev %#0x type %#0x for %d cpu\n",
+   __LINE__, data->level, data->level_type, cpu);
oldcpu = td->td_oncpu;
is_bound = cpu_sched_is_bound(td);
set_cpu(cpu, td);
-   cpuid_count(data->level, 0, data->data);
+   cpuid_count(data->level, data->level_type, data->data);
restore_cpu(oldcpu, is_bound, td);
return (0);
 }
 
+static int
+cpuctl_do_cpuid(int cpu, cpuctl_cpuid_args_t *data, struct thread *td)
+{
+
+   /* Override the level type. */
+   data->level_type = 0;
+   return (cpuctl_do_cpuid_count(cpu, data, td));
+}
+
 /*
  * Actually perform MSR operations.
  */

Modified: head/sys/sys/cpuctl.h
==
--- head/sys/sys/cpuctl.h   Thu Jun 19 21:05:07 2014(r267650)
+++ head/sys/sys/cpuctl.h   Thu Jun 19 21:54:41 2014(r267651)
@@ -35,7 +35,8 @@ typedef struct {
 } cpuctl_msr_args_t;
 
 typedef struct {
-   int level;  /* CPUID level */
+   int level;  /* CPUID level */
+   int level_type; /* CPUID level type */
uint32_tdata[4];
 } cpuctl_cpuid_args_t;
 
@@ -50,5 +51,6 @@ typedef struct {
 #defineCPUCTL_UPDATE   _IOWR('c', 4, cpuctl_update_args_t)
 #defineCPUCTL_MSRSBIT  _IOWR('c', 5, cpuctl_msr_args_t)
 #defineCPUCTL_MSRCBIT  _IOWR('c', 6, cpuctl_msr_args_t)
+#def

svn commit: r267548 - in head/sys: amd64/amd64 arm/arm dev/agp dev/cxgbe/tom dev/drm dev/drm2/i915 dev/drm2/ttm dev/ti dev/virtio/balloon dev/xen/balloon i386/i386 i386/xen ia64/ia64 kern mips/mips...

2014-06-16 Thread Attilio Rao
Author: attilio
Date: Mon Jun 16 18:15:27 2014
New Revision: 267548
URL: http://svnweb.freebsd.org/changeset/base/267548

Log:
  - Modify vm_page_unwire() and vm_page_enqueue() to directly accept
the queue where to enqueue pages that are going to be unwired.
  - Add stronger checks to the enqueue/dequeue for the pagequeues when
adding and removing pages to them.
  
  Of course, for unmanaged pages the queue parameter of vm_page_unwire() will
  be ignored, just as the active parameter today.
  This makes adding new pagequeues quicker.
  
  This change effectively modifies the KPI.  __FreeBSD_version will be,
  however, bumped just when the full cache of free pages will be
  evicted.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  alc
  Tested by:pho

Modified:
  head/sys/amd64/amd64/pmap.c
  head/sys/arm/arm/pmap-v6.c
  head/sys/dev/agp/agp.c
  head/sys/dev/agp/agp_i810.c
  head/sys/dev/cxgbe/tom/t4_ddp.c
  head/sys/dev/drm/via_dmablit.c
  head/sys/dev/drm2/i915/i915_gem.c
  head/sys/dev/drm2/i915/i915_gem_gtt.c
  head/sys/dev/drm2/ttm/ttm_page_alloc.c
  head/sys/dev/ti/if_ti.c
  head/sys/dev/virtio/balloon/virtio_balloon.c
  head/sys/dev/xen/balloon/balloon.c
  head/sys/i386/i386/pmap.c
  head/sys/i386/xen/pmap.c
  head/sys/ia64/ia64/pmap.c
  head/sys/kern/uipc_syscalls.c
  head/sys/kern/vfs_bio.c
  head/sys/mips/mips/pmap.c
  head/sys/net/bpf_zerocopy.c
  head/sys/vm/uma_core.c
  head/sys/vm/vm_fault.c
  head/sys/vm/vm_glue.c
  head/sys/vm/vm_kern.c
  head/sys/vm/vm_page.c
  head/sys/vm/vm_page.h

Modified: head/sys/amd64/amd64/pmap.c
==
--- head/sys/amd64/amd64/pmap.c Mon Jun 16 18:14:05 2014(r267547)
+++ head/sys/amd64/amd64/pmap.c Mon Jun 16 18:15:27 2014(r267548)
@@ -2868,7 +2868,7 @@ free_pv_chunk(struct pv_chunk *pc)
/* entire chunk is free, return it */
m = PHYS_TO_VM_PAGE(DMAP_TO_PHYS((vm_offset_t)pc));
dump_drop_page(m->phys_addr);
-   vm_page_unwire(m, 0);
+   vm_page_unwire(m, PQ_INACTIVE);
vm_page_free(m);
 }
 

Modified: head/sys/arm/arm/pmap-v6.c
==
--- head/sys/arm/arm/pmap-v6.c  Mon Jun 16 18:14:05 2014(r267547)
+++ head/sys/arm/arm/pmap-v6.c  Mon Jun 16 18:15:27 2014(r267548)
@@ -4222,7 +4222,7 @@ pmap_free_pv_chunk(struct pv_chunk *pc)
/* entire chunk is free, return it */
m = PHYS_TO_VM_PAGE(pmap_kextract((vm_offset_t)pc));
pmap_qremove((vm_offset_t)pc, 1);
-   vm_page_unwire(m, 0);
+   vm_page_unwire(m, PQ_INACTIVE);
vm_page_free(m);
pmap_ptelist_free(&pv_vafree, (vm_offset_t)pc);
 

Modified: head/sys/dev/agp/agp.c
==
--- head/sys/dev/agp/agp.c  Mon Jun 16 18:14:05 2014(r267547)
+++ head/sys/dev/agp/agp.c  Mon Jun 16 18:15:27 2014(r267548)
@@ -629,7 +629,7 @@ bad:
if (k >= i)
vm_page_xunbusy(m);
vm_page_lock(m);
-   vm_page_unwire(m, 0);
+   vm_page_unwire(m, PQ_INACTIVE);
vm_page_unlock(m);
}
VM_OBJECT_WUNLOCK(mem->am_obj);
@@ -663,7 +663,7 @@ agp_generic_unbind_memory(device_t dev, 
for (i = 0; i < mem->am_size; i += PAGE_SIZE) {
m = vm_page_lookup(mem->am_obj, atop(i));
vm_page_lock(m);
-   vm_page_unwire(m, 0);
+   vm_page_unwire(m, PQ_INACTIVE);
vm_page_unlock(m);
}
VM_OBJECT_WUNLOCK(mem->am_obj);

Modified: head/sys/dev/agp/agp_i810.c
==
--- head/sys/dev/agp/agp_i810.c Mon Jun 16 18:14:05 2014(r267547)
+++ head/sys/dev/agp/agp_i810.c Mon Jun 16 18:15:27 2014(r267548)
@@ -2009,7 +2009,7 @@ agp_i810_free_memory(device_t dev, struc
VM_OBJECT_WLOCK(mem->am_obj);
m = vm_page_lookup(mem->am_obj, 0);
vm_page_lock(m);
-   vm_page_unwire(m, 0);
+   vm_page_unwire(m, PQ_INACTIVE);
vm_page_unlock(m);
VM_OBJECT_WUNLOCK(mem->am_obj);
} else {

Modified: head/sys/dev/cxgbe/tom/t4_ddp.c
==
--- head/sys/dev/cxgbe/tom/t4_ddp.c Mon Jun 16 18:14:05 2014
(r267547)
+++ head/sys/dev/cxgbe/tom/t4_ddp.c Mon Jun 16 18:15:27 2014
(r267548)
@@ -869,7 +869,7 @@ unwire_ddp_buffer(struct ddp_buffer *db)
for (i = 0; i < db->npages; i++) {
p = db->pages[i];
vm_page_lock(p);
-   vm_page_unwire(p, 0);
+   vm_page_unwire(p, PQ_INACTIVE);
vm_page_unlock(p);
}
 }

Modifi

Re: svn commit: r266775 - head/sys/x86/x86

2014-05-30 Thread Attilio Rao
On Fri, May 30, 2014 at 8:06 PM, John Baldwin  wrote:
> On Friday, May 30, 2014 12:55:06 pm Attilio Rao wrote:
>> On Fri, May 30, 2014 at 6:44 PM, John Baldwin  wrote:
>> > On Friday, May 30, 2014 11:51:38 am Attilio Rao wrote:
>> >> On Fri, May 30, 2014 at 5:47 PM, John Baldwin  wrote:
>> >> > On Friday, May 30, 2014 11:39:24 am Attilio Rao wrote:
>> >> >> On Fri, May 30, 2014 at 5:03 PM, John Baldwin  wrote:
>> >> >> > On Friday, May 30, 2014 10:54:06 am Attilio Rao wrote:
>> >> >> >> On Tue, May 27, 2014 at 11:31 PM, Scott Long 
> wrote:
>> >> >> >> > Author: scottl
>> >> >> >> > Date: Tue May 27 21:31:11 2014
>> >> >> >> > New Revision: 266775
>> >> >> >> > URL: http://svnweb.freebsd.org/changeset/base/266775
>> >> >> >> >
>> >> >> >> > Log:
>> >> >> >> >   Eliminate the fake contig_dmamap and replace it with a new
> flag,
>> >> >> >> >   BUS_DMA_KMEM_ALLOC.  They serve the same purpose, but using the
> flag
>> >> >> >> >   means that the map can be NULL again, which in turn enables
> significant
>> >> >> >> >   optimizations for the common case of no bouncing.
>> >> >> >>
>> >> >> >> While I think this is in general a good idea, unfortunately our
>> >> >> >> drivers do so many dumb things when freeing DMA allocated buffers
> that
>> >> >> >> having a NULL map is going to cause some "turbolence" and make such
>> >> >> >> bugs more visible.
>> >> >> >> An example is with ATA, where I think this fix is needed:
>> >> >> >> http://www.freebsd.org/~attilio/dmamem_free-ata.patch
>> >> >> >>
>> >> >> >> Otherwise, what can happen with bounce buffers, is that the
> allocated
>> >> >> >> memory via contig malloc was not going to be freed anytime.
>> >> >> >>
>> >> >> >> I tried to look around and I found questionable (read broken) code
> in
>> >> >> >> basically every driver which allocates DMA buffers, so I really
> don't
>> >> >> >> feel I want to fix the majority of our drivers. I just think such
>> >> >> >> paths are not excercised enough to be seen in practice often or the
>> >> >> >> bugs just get unnoticed.
>> >> >> >
>> >> >> > Eh, many maps for static allocations were already NULL and have been
> for a
>> >> >> > long time.  This is nothign new.  Plus, the diff you posted has a
> bug
>> >> >> > regardless of explicitly destroying a map created by
> bus_dmamem_alloc().
>> >> >>
>> >> >> Did you notice that I *removed* the destroy not *added*?
>> >> >
>> >> > Yes, my point was that that bug in the original code you are fixing was
> there
>> >> > regardless of Scott's change.
>> >>
>> >> And when I did say something different?
>> >> I don't understand what's the point of your messages, besides showing
>> >> that you didn't read correctly my patch.
>> >
>> > I read yours correctly but worded mine poorly.  My point is that Scott's
>> > change does not introduce anything new.  We've had NULL maps for static
>> > allocations for many, many years.  It's only been recently that we've
>> > had more maps not be NULL for this.  However, even if you discounted
>> > the whole NULL vs non-NULL maps thing, the driver in question that you
>> > are fixing was broken regardless.  That is, due to the extra
>> > bus_dmamap_destroy() the driver was broken regardless of whether the map
>> > was NULL or non-NULL.
>>
>> To be honest, pre-266775 the kernel would actually panic for this
>> specific driver, because we were going to free memory that was never
>> allocated (by having a valid mapping but an invalid dma memory
>> pointer).
>
> pre-239354 bus_dma would have used a NULL map just as it does now.  And
> even some allocations during that window could still use a NULL map.  The
> idea of a NULL map is not a new concept.  Most maps from bus_dmamem_alloc()
> have been NULL for most of bus_dma's existence.
>
>> That was prompted to look at the dma_alloc_*() bits of drivers.
>> We need to make a real sweep at drivers on these bits.
>
> I did a start: http://p4web.freebsd.org/@@1194266?ac=10

I had converted if_alc.c, if_ale.c and e1000/ stuff before to give up.
The first 2 looks good to me.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r266775 - head/sys/x86/x86

2014-05-30 Thread Attilio Rao
On Fri, May 30, 2014 at 6:44 PM, John Baldwin  wrote:
> On Friday, May 30, 2014 11:51:38 am Attilio Rao wrote:
>> On Fri, May 30, 2014 at 5:47 PM, John Baldwin  wrote:
>> > On Friday, May 30, 2014 11:39:24 am Attilio Rao wrote:
>> >> On Fri, May 30, 2014 at 5:03 PM, John Baldwin  wrote:
>> >> > On Friday, May 30, 2014 10:54:06 am Attilio Rao wrote:
>> >> >> On Tue, May 27, 2014 at 11:31 PM, Scott Long  
>> >> >> wrote:
>> >> >> > Author: scottl
>> >> >> > Date: Tue May 27 21:31:11 2014
>> >> >> > New Revision: 266775
>> >> >> > URL: http://svnweb.freebsd.org/changeset/base/266775
>> >> >> >
>> >> >> > Log:
>> >> >> >   Eliminate the fake contig_dmamap and replace it with a new flag,
>> >> >> >   BUS_DMA_KMEM_ALLOC.  They serve the same purpose, but using the 
>> >> >> > flag
>> >> >> >   means that the map can be NULL again, which in turn enables 
>> >> >> > significant
>> >> >> >   optimizations for the common case of no bouncing.
>> >> >>
>> >> >> While I think this is in general a good idea, unfortunately our
>> >> >> drivers do so many dumb things when freeing DMA allocated buffers that
>> >> >> having a NULL map is going to cause some "turbolence" and make such
>> >> >> bugs more visible.
>> >> >> An example is with ATA, where I think this fix is needed:
>> >> >> http://www.freebsd.org/~attilio/dmamem_free-ata.patch
>> >> >>
>> >> >> Otherwise, what can happen with bounce buffers, is that the allocated
>> >> >> memory via contig malloc was not going to be freed anytime.
>> >> >>
>> >> >> I tried to look around and I found questionable (read broken) code in
>> >> >> basically every driver which allocates DMA buffers, so I really don't
>> >> >> feel I want to fix the majority of our drivers. I just think such
>> >> >> paths are not excercised enough to be seen in practice often or the
>> >> >> bugs just get unnoticed.
>> >> >
>> >> > Eh, many maps for static allocations were already NULL and have been 
>> >> > for a
>> >> > long time.  This is nothign new.  Plus, the diff you posted has a bug
>> >> > regardless of explicitly destroying a map created by bus_dmamem_alloc().
>> >>
>> >> Did you notice that I *removed* the destroy not *added*?
>> >
>> > Yes, my point was that that bug in the original code you are fixing was 
>> > there
>> > regardless of Scott's change.
>>
>> And when I did say something different?
>> I don't understand what's the point of your messages, besides showing
>> that you didn't read correctly my patch.
>
> I read yours correctly but worded mine poorly.  My point is that Scott's
> change does not introduce anything new.  We've had NULL maps for static
> allocations for many, many years.  It's only been recently that we've
> had more maps not be NULL for this.  However, even if you discounted
> the whole NULL vs non-NULL maps thing, the driver in question that you
> are fixing was broken regardless.  That is, due to the extra
> bus_dmamap_destroy() the driver was broken regardless of whether the map
> was NULL or non-NULL.

To be honest, pre-266775 the kernel would actually panic for this
specific driver, because we were going to free memory that was never
allocated (by having a valid mapping but an invalid dma memory
pointer).
That was prompted to look at the dma_alloc_*() bits of drivers.
We need to make a real sweep at drivers on these bits.

>
> TL;DR:
>
> - Scott's change did not introduce any new behavior so we don't need to
>   worry about a spate of new bugs uncovered in drivers because of it.

Not entirely true. For ATA it was before a panic and now it is a
silent memory leak. For other drivers it may be the opposite.
I could just find this one becaue I got bitten by it.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r266775 - head/sys/x86/x86

2014-05-30 Thread Attilio Rao
On Fri, May 30, 2014 at 5:47 PM, John Baldwin  wrote:
> On Friday, May 30, 2014 11:39:24 am Attilio Rao wrote:
>> On Fri, May 30, 2014 at 5:03 PM, John Baldwin  wrote:
>> > On Friday, May 30, 2014 10:54:06 am Attilio Rao wrote:
>> >> On Tue, May 27, 2014 at 11:31 PM, Scott Long  wrote:
>> >> > Author: scottl
>> >> > Date: Tue May 27 21:31:11 2014
>> >> > New Revision: 266775
>> >> > URL: http://svnweb.freebsd.org/changeset/base/266775
>> >> >
>> >> > Log:
>> >> >   Eliminate the fake contig_dmamap and replace it with a new flag,
>> >> >   BUS_DMA_KMEM_ALLOC.  They serve the same purpose, but using the flag
>> >> >   means that the map can be NULL again, which in turn enables 
>> >> > significant
>> >> >   optimizations for the common case of no bouncing.
>> >>
>> >> While I think this is in general a good idea, unfortunately our
>> >> drivers do so many dumb things when freeing DMA allocated buffers that
>> >> having a NULL map is going to cause some "turbolence" and make such
>> >> bugs more visible.
>> >> An example is with ATA, where I think this fix is needed:
>> >> http://www.freebsd.org/~attilio/dmamem_free-ata.patch
>> >>
>> >> Otherwise, what can happen with bounce buffers, is that the allocated
>> >> memory via contig malloc was not going to be freed anytime.
>> >>
>> >> I tried to look around and I found questionable (read broken) code in
>> >> basically every driver which allocates DMA buffers, so I really don't
>> >> feel I want to fix the majority of our drivers. I just think such
>> >> paths are not excercised enough to be seen in practice often or the
>> >> bugs just get unnoticed.
>> >
>> > Eh, many maps for static allocations were already NULL and have been for a
>> > long time.  This is nothign new.  Plus, the diff you posted has a bug
>> > regardless of explicitly destroying a map created by bus_dmamem_alloc().
>>
>> Did you notice that I *removed* the destroy not *added*?
>
> Yes, my point was that that bug in the original code you are fixing was there
> regardless of Scott's change.

And when I did say something different?
I don't understand what's the point of your messages, besides showing
that you didn't read correctly my patch.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r266775 - head/sys/x86/x86

2014-05-30 Thread Attilio Rao
On Fri, May 30, 2014 at 5:03 PM, John Baldwin  wrote:
> On Friday, May 30, 2014 10:54:06 am Attilio Rao wrote:
>> On Tue, May 27, 2014 at 11:31 PM, Scott Long  wrote:
>> > Author: scottl
>> > Date: Tue May 27 21:31:11 2014
>> > New Revision: 266775
>> > URL: http://svnweb.freebsd.org/changeset/base/266775
>> >
>> > Log:
>> >   Eliminate the fake contig_dmamap and replace it with a new flag,
>> >   BUS_DMA_KMEM_ALLOC.  They serve the same purpose, but using the flag
>> >   means that the map can be NULL again, which in turn enables significant
>> >   optimizations for the common case of no bouncing.
>>
>> While I think this is in general a good idea, unfortunately our
>> drivers do so many dumb things when freeing DMA allocated buffers that
>> having a NULL map is going to cause some "turbolence" and make such
>> bugs more visible.
>> An example is with ATA, where I think this fix is needed:
>> http://www.freebsd.org/~attilio/dmamem_free-ata.patch
>>
>> Otherwise, what can happen with bounce buffers, is that the allocated
>> memory via contig malloc was not going to be freed anytime.
>>
>> I tried to look around and I found questionable (read broken) code in
>> basically every driver which allocates DMA buffers, so I really don't
>> feel I want to fix the majority of our drivers. I just think such
>> paths are not excercised enough to be seen in practice often or the
>> bugs just get unnoticed.
>
> Eh, many maps for static allocations were already NULL and have been for a
> long time.  This is nothign new.  Plus, the diff you posted has a bug
> regardless of explicitly destroying a map created by bus_dmamem_alloc().

Did you notice that I *removed* the destroy not *added*?

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r266775 - head/sys/x86/x86

2014-05-30 Thread Attilio Rao
On Tue, May 27, 2014 at 11:31 PM, Scott Long  wrote:
> Author: scottl
> Date: Tue May 27 21:31:11 2014
> New Revision: 266775
> URL: http://svnweb.freebsd.org/changeset/base/266775
>
> Log:
>   Eliminate the fake contig_dmamap and replace it with a new flag,
>   BUS_DMA_KMEM_ALLOC.  They serve the same purpose, but using the flag
>   means that the map can be NULL again, which in turn enables significant
>   optimizations for the common case of no bouncing.

While I think this is in general a good idea, unfortunately our
drivers do so many dumb things when freeing DMA allocated buffers that
having a NULL map is going to cause some "turbolence" and make such
bugs more visible.
An example is with ATA, where I think this fix is needed:
http://www.freebsd.org/~attilio/dmamem_free-ata.patch

Otherwise, what can happen with bounce buffers, is that the allocated
memory via contig malloc was not going to be freed anytime.

I tried to look around and I found questionable (read broken) code in
basically every driver which allocates DMA buffers, so I really don't
feel I want to fix the majority of our drivers. I just think such
paths are not excercised enough to be seen in practice often or the
bugs just get unnoticed.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r263318 - in head: sys/compat/freebsd32 sys/kern sys/sys usr.bin/truss

2014-03-19 Thread Attilio Rao
On Wed, Mar 19, 2014 at 7:10 PM, Jilles Tjoelker  wrote:
> On Tue, Mar 18, 2014 at 09:32:03PM +0000, Attilio Rao wrote:
>> Author: attilio
>> Date: Tue Mar 18 21:32:03 2014
>> New Revision: 263318
>> URL: http://svnweb.freebsd.org/changeset/base/263318
>
>> Log:
>>   Remove dead code from umtx support:
>>   - Retire long time unused (basically always unused) sys__umtx_lock()
>> and sys__umtx_unlock() syscalls
>>   - struct umtx and their supporting definitions
>>   - UMUTEX_ERROR_CHECK flag
>>   - Retire UMTX_OP_LOCK/UMTX_OP_UNLOCK from _umtx_op() syscall
>
>>   __FreeBSD_version is not bumped yet because it is expected that further
>>   breakages to the umtx interface will follow up in the next days.
>>   However there will be a final bump when necessary.
>
>>   Sponsored by:   EMC / Isilon storage division
>>   Reviewed by:jhb
>
>> [snip]
>> Modified: head/sys/sys/umtx.h
>> ==
>> --- head/sys/sys/umtx.h   Tue Mar 18 20:14:13 2014(r263317)
>> +++ head/sys/sys/umtx.h   Tue Mar 18 21:32:03 2014(r263318)
>> [snip]
>> -static __inline int
>> -umtx_wait(u_long *p, long val, const struct timespec *timeout)
>> -{
>> - if (_umtx_op(p, UMTX_OP_WAIT, val, 0,
>> - __DECONST(void *, timeout)) == -1)
>> - return (errno);
>> - return (0);
>> -}
>> -
>> -/* Wake threads waiting on a user address. */
>> -static __inline int
>> -umtx_wake(u_long *p, int nr_wakeup)
>> -{
>> - if (_umtx_op(p, UMTX_OP_WAKE, nr_wakeup, 0, 0) == -1)
>> - return (errno);
>> - return (0);
>> -}
>> -
>
> These two do not use struct umtx, are not obsolete and are used by some
> ports, such as lang/sbcl with certain options. They are similar to the
> Linux futex system call. The relatively recent libxshmfence uses the
> underlying umtx_op directly, mainly because there is no static inline
> function for UMTX_OP_WAIT_UINT.

They can just use _umtx_op() straight off.
I already mentioned in the commit message that I will bump
__FreeBSD_version appropriately when more API breakage will be
introduced and we will have a relatively "stable" API situation of
umtx. At that point ports maintainer can fix the ports.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r263350 - head/sys/kern

2014-03-19 Thread Attilio Rao
Author: attilio
Date: Wed Mar 19 12:45:40 2014
New Revision: 263350
URL: http://svnweb.freebsd.org/changeset/base/263350

Log:
  Fix comments.
  
  Sponsored by: EMC / Isilon Storage Division

Modified:
  head/sys/kern/kern_umtx.c

Modified: head/sys/kern/kern_umtx.c
==
--- head/sys/kern/kern_umtx.c   Wed Mar 19 12:35:04 2014(r263349)
+++ head/sys/kern/kern_umtx.c   Wed Mar 19 12:45:40 2014(r263350)
@@ -3115,12 +3115,12 @@ static _umtx_op_func op_table[] = {
__umtx_op_rw_unlock,/* UMTX_OP_RW_UNLOCK */
__umtx_op_wait_uint_private,/* UMTX_OP_WAIT_UINT_PRIVATE */
__umtx_op_wake_private, /* UMTX_OP_WAKE_PRIVATE */
-   __umtx_op_wait_umutex,  /* UMTX_OP_UMUTEX_WAIT */
-   __umtx_op_wake_umutex,  /* UMTX_OP_UMUTEX_WAKE */
+   __umtx_op_wait_umutex,  /* UMTX_OP_MUTEX_WAIT */
+   __umtx_op_wake_umutex,  /* UMTX_OP_MUTEX_WAKE */
__umtx_op_sem_wait, /* UMTX_OP_SEM_WAIT */
__umtx_op_sem_wake, /* UMTX_OP_SEM_WAKE */
__umtx_op_nwake_private,/* UMTX_OP_NWAKE_PRIVATE */
-   __umtx_op_wake2_umutex  /* UMTX_OP_UMUTEX_WAKE2 */
+   __umtx_op_wake2_umutex  /* UMTX_OP_MUTEX_WAKE2 */
 };
 
 int
@@ -3381,12 +3381,12 @@ static _umtx_op_func op_table_compat32[]
__umtx_op_rw_unlock,/* UMTX_OP_RW_UNLOCK */
__umtx_op_wait_uint_private_compat32,   /* UMTX_OP_WAIT_UINT_PRIVATE */
__umtx_op_wake_private, /* UMTX_OP_WAKE_PRIVATE */
-   __umtx_op_wait_umutex_compat32, /* UMTX_OP_UMUTEX_WAIT */
-   __umtx_op_wake_umutex,  /* UMTX_OP_UMUTEX_WAKE */
+   __umtx_op_wait_umutex_compat32, /* UMTX_OP_MUTEX_WAIT */
+   __umtx_op_wake_umutex,  /* UMTX_OP_MUTEX_WAKE */
__umtx_op_sem_wait_compat32,/* UMTX_OP_SEM_WAIT */
__umtx_op_sem_wake, /* UMTX_OP_SEM_WAKE */
__umtx_op_nwake_private32,  /* UMTX_OP_NWAKE_PRIVATE */
-   __umtx_op_wake2_umutex  /* UMTX_OP_UMUTEX_WAKE2 */
+   __umtx_op_wake2_umutex  /* UMTX_OP_MUTEX_WAKE2 */
 };
 
 int
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r263328 - in head/sys: dev/md vm

2014-03-18 Thread Attilio Rao
Author: attilio
Date: Wed Mar 19 01:13:42 2014
New Revision: 263328
URL: http://svnweb.freebsd.org/changeset/base/263328

Log:
  vm_page_grab() and vm_pager_get_pages() can drop the vm_object lock,
  then threads can sleep on the pip condition.
  Avoid to deadlock such threads by correctly awakening the sleeping ones
  after the pip is finished.
  swapoff side of the bug can likely result in shutdown deadlocks.
  
  Sponsored by: EMC / Isilon Storage Division
  Reported by:  pho, pluknet
  Tested by:pho

Modified:
  head/sys/dev/md/md.c
  head/sys/vm/swap_pager.c

Modified: head/sys/dev/md/md.c
==
--- head/sys/dev/md/md.cWed Mar 19 00:55:12 2014(r263327)
+++ head/sys/dev/md/md.cWed Mar 19 01:13:42 2014(r263328)
@@ -903,7 +903,7 @@ mdstart_swap(struct md_s *sc, struct bio
offs = 0;
ma_offs += len;
}
-   vm_object_pip_subtract(sc->object, 1);
+   vm_object_pip_wakeup(sc->object);
VM_OBJECT_WUNLOCK(sc->object);
return (rv != VM_PAGER_ERROR ? 0 : ENOSPC);
 }

Modified: head/sys/vm/swap_pager.c
==
--- head/sys/vm/swap_pager.cWed Mar 19 00:55:12 2014(r263327)
+++ head/sys/vm/swap_pager.cWed Mar 19 01:13:42 2014(r263328)
@@ -1713,7 +1713,7 @@ swp_pager_force_pagein(vm_object_t objec
vm_object_pip_add(object, 1);
m = vm_page_grab(object, pindex, VM_ALLOC_NORMAL);
if (m->valid == VM_PAGE_BITS_ALL) {
-   vm_object_pip_subtract(object, 1);
+   vm_object_pip_wakeup(object);
vm_page_dirty(m);
vm_page_lock(m);
vm_page_activate(m);
@@ -1725,7 +1725,7 @@ swp_pager_force_pagein(vm_object_t objec
 
if (swap_pager_getpages(object, &m, 1, 0) != VM_PAGER_OK)
panic("swap_pager_force_pagein: read from swap failed");/*XXX*/
-   vm_object_pip_subtract(object, 1);
+   vm_object_pip_wakeup(object);
vm_page_dirty(m);
vm_page_lock(m);
vm_page_deactivate(m);
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r263324 - head/sys/kern

2014-03-18 Thread Attilio Rao
Author: attilio
Date: Wed Mar 19 00:38:27 2014
New Revision: 263324
URL: http://svnweb.freebsd.org/changeset/base/263324

Log:
  Fix GENERIC build.

Modified:
  head/sys/kern/sched_ule.c

Modified: head/sys/kern/sched_ule.c
==
--- head/sys/kern/sched_ule.c   Tue Mar 18 23:51:34 2014(r263323)
+++ head/sys/kern/sched_ule.c   Wed Mar 19 00:38:27 2014(r263324)
@@ -46,6 +46,7 @@ __FBSDID("$FreeBSD$");
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r263319 - in head/sys: compat/freebsd32 kern sys

2014-03-18 Thread Attilio Rao
Author: attilio
Date: Tue Mar 18 21:34:11 2014
New Revision: 263319
URL: http://svnweb.freebsd.org/changeset/base/263319

Log:
  Regen per r263318.
  
  Sponsored by: EMC / Isilon storage division

Modified:
  head/sys/compat/freebsd32/freebsd32_proto.h
  head/sys/compat/freebsd32/freebsd32_syscall.h
  head/sys/compat/freebsd32/freebsd32_syscalls.c
  head/sys/compat/freebsd32/freebsd32_sysent.c
  head/sys/compat/freebsd32/freebsd32_systrace_args.c
  head/sys/kern/init_sysent.c
  head/sys/kern/syscalls.c
  head/sys/kern/systrace_args.c
  head/sys/sys/syscall.h
  head/sys/sys/syscall.mk
  head/sys/sys/sysproto.h

Modified: head/sys/compat/freebsd32/freebsd32_proto.h
==
--- head/sys/compat/freebsd32/freebsd32_proto.h Tue Mar 18 21:32:03 2014
(r263318)
+++ head/sys/compat/freebsd32/freebsd32_proto.h Tue Mar 18 21:34:11 2014
(r263319)
@@ -3,7 +3,7 @@
  *
  * DO NOT EDIT-- this file is automatically generated.
  * $FreeBSD$
- * created from FreeBSD: head/sys/compat/freebsd32/syscalls.master 255708 
2013-09-19 18:53:42Z jhb 
+ * created from FreeBSD: head/sys/compat/freebsd32/syscalls.master 263318 
2014-03-18 21:32:03Z attilio 
  */
 
 #ifndef _FREEBSD32_SYSPROTO_H_
@@ -367,12 +367,6 @@ struct freebsd32_swapcontext_args {
char oucp_l_[PADL_(struct freebsd32_ucontext *)]; struct 
freebsd32_ucontext * oucp; char oucp_r_[PADR_(struct freebsd32_ucontext *)];
char ucp_l_[PADL_(const struct freebsd32_ucontext *)]; const struct 
freebsd32_ucontext * ucp; char ucp_r_[PADR_(const struct freebsd32_ucontext *)];
 };
-struct freebsd32_umtx_lock_args {
-   char umtx_l_[PADL_(struct umtx *)]; struct umtx * umtx; char 
umtx_r_[PADR_(struct umtx *)];
-};
-struct freebsd32_umtx_unlock_args {
-   char umtx_l_[PADL_(struct umtx *)]; struct umtx * umtx; char 
umtx_r_[PADR_(struct umtx *)];
-};
 struct freebsd32_ksem_timedwait_args {
char id_l_[PADL_(semid_t)]; semid_t id; char id_r_[PADR_(semid_t)];
char abstime_l_[PADL_(const struct timespec32 *)]; const struct 
timespec32 * abstime; char abstime_r_[PADR_(const struct timespec32 *)];
@@ -758,8 +752,6 @@ int freebsd32_sigreturn(struct thread *,
 intfreebsd32_getcontext(struct thread *, struct freebsd32_getcontext_args 
*);
 intfreebsd32_setcontext(struct thread *, struct freebsd32_setcontext_args 
*);
 intfreebsd32_swapcontext(struct thread *, struct 
freebsd32_swapcontext_args *);
-intfreebsd32_umtx_lock(struct thread *, struct freebsd32_umtx_lock_args *);
-intfreebsd32_umtx_unlock(struct thread *, struct 
freebsd32_umtx_unlock_args *);
 intfreebsd32_ksem_timedwait(struct thread *, struct 
freebsd32_ksem_timedwait_args *);
 intfreebsd32_thr_suspend(struct thread *, struct 
freebsd32_thr_suspend_args *);
 intfreebsd32_umtx_op(struct thread *, struct freebsd32_umtx_op_args *);
@@ -1185,8 +1177,6 @@ int   freebsd7_freebsd32_shmctl(struct thr
 #defineFREEBSD32_SYS_AUE_freebsd32_getcontext  AUE_NULL
 #defineFREEBSD32_SYS_AUE_freebsd32_setcontext  AUE_NULL
 #defineFREEBSD32_SYS_AUE_freebsd32_swapcontext AUE_NULL
-#defineFREEBSD32_SYS_AUE_freebsd32_umtx_lock   AUE_NULL
-#defineFREEBSD32_SYS_AUE_freebsd32_umtx_unlock AUE_NULL
 #defineFREEBSD32_SYS_AUE_freebsd32_ksem_timedwait  AUE_NULL
 #defineFREEBSD32_SYS_AUE_freebsd32_thr_suspend AUE_NULL
 #defineFREEBSD32_SYS_AUE_freebsd32_umtx_op AUE_NULL

Modified: head/sys/compat/freebsd32/freebsd32_syscall.h
==
--- head/sys/compat/freebsd32/freebsd32_syscall.h   Tue Mar 18 21:32:03 
2014(r263318)
+++ head/sys/compat/freebsd32/freebsd32_syscall.h   Tue Mar 18 21:34:11 
2014(r263319)
@@ -3,7 +3,7 @@
  *
  * DO NOT EDIT-- this file is automatically generated.
  * $FreeBSD$
- * created from FreeBSD: head/sys/compat/freebsd32/syscalls.master 255708 
2013-09-19 18:53:42Z jhb 
+ * created from FreeBSD: head/sys/compat/freebsd32/syscalls.master 263318 
2014-03-18 21:32:03Z attilio 
  */
 
 #defineFREEBSD32_SYS_syscall   0
@@ -339,8 +339,6 @@
 #defineFREEBSD32_SYS_thr_exit  431
 #defineFREEBSD32_SYS_thr_self  432
 #defineFREEBSD32_SYS_thr_kill  433
-#defineFREEBSD32_SYS_freebsd32_umtx_lock   434
-#defineFREEBSD32_SYS_freebsd32_umtx_unlock 435
 #defineFREEBSD32_SYS_jail_attach   436
 #defineFREEBSD32_SYS_extattr_list_fd   437
 #defineFREEBSD32_SYS_extattr_list_file 438

Modified: head/sys/compat/freebsd32/freebsd32_syscalls.c
==
--- head/sys/compat/freebsd32/freebsd32_syscalls.c  Tue Mar 18 21:32:03 
2014(r263318)
+++ head/sys/compat/freebsd32/freebsd32_syscalls.c  Tue Mar 18 21:34:11 
2014(r263319)
@@ -3,7 +3,7 @@
  *
  * DO NOT EDIT-- this f

svn commit: r263318 - in head: sys/compat/freebsd32 sys/kern sys/sys usr.bin/truss

2014-03-18 Thread Attilio Rao
Author: attilio
Date: Tue Mar 18 21:32:03 2014
New Revision: 263318
URL: http://svnweb.freebsd.org/changeset/base/263318

Log:
  Remove dead code from umtx support:
  - Retire long time unused (basically always unused) sys__umtx_lock()
and sys__umtx_unlock() syscalls
  - struct umtx and their supporting definitions
  - UMUTEX_ERROR_CHECK flag
  - Retire UMTX_OP_LOCK/UMTX_OP_UNLOCK from _umtx_op() syscall
  
  __FreeBSD_version is not bumped yet because it is expected that further
  breakages to the umtx interface will follow up in the next days.
  However there will be a final bump when necessary.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  jhb

Modified:
  head/sys/compat/freebsd32/syscalls.master
  head/sys/kern/kern_umtx.c
  head/sys/kern/syscalls.master
  head/sys/sys/_umtx.h
  head/sys/sys/umtx.h
  head/usr.bin/truss/syscall.h
  head/usr.bin/truss/syscalls.c

Modified: head/sys/compat/freebsd32/syscalls.master
==
--- head/sys/compat/freebsd32/syscalls.master   Tue Mar 18 20:14:13 2014
(r263317)
+++ head/sys/compat/freebsd32/syscalls.master   Tue Mar 18 21:32:03 2014
(r263318)
@@ -775,8 +775,8 @@
 431AUE_NULLNOPROTO { void thr_exit(long *state); }
 432AUE_NULLNOPROTO { int thr_self(long *id); }
 433AUE_NULLNOPROTO { int thr_kill(long id, int sig); }
-434AUE_NULLSTD { int freebsd32_umtx_lock(struct umtx *umtx); }
-435AUE_NULLSTD { int freebsd32_umtx_unlock(struct umtx *umtx); 
}
+434AUE_NULLUNIMPL  nosys
+435AUE_NULLUNIMPL  nosys
 436AUE_NULLNOPROTO { int jail_attach(int jid); }
 437AUE_EXTATTR_LIST_FD NOPROTO { ssize_t extattr_list_fd(int fd, \
int attrnamespace, void *data, \

Modified: head/sys/kern/kern_umtx.c
==
--- head/sys/kern/kern_umtx.c   Tue Mar 18 20:14:13 2014(r263317)
+++ head/sys/kern/kern_umtx.c   Tue Mar 18 21:32:03 2014(r263318)
@@ -838,367 +838,6 @@ umtx_key_release(struct umtx_key *key)
 }
 
 /*
- * Lock a umtx object.
- */
-static int
-do_lock_umtx(struct thread *td, struct umtx *umtx, u_long id,
-   const struct timespec *timeout)
-{
-   struct abs_timeout timo;
-   struct umtx_q *uq;
-   u_long owner;
-   u_long old;
-   int error = 0;
-
-   uq = td->td_umtxq;
-   if (timeout != NULL)
-   abs_timeout_init(&timo, CLOCK_REALTIME, 0, timeout);
-
-   /*
-* Care must be exercised when dealing with umtx structure. It
-* can fault on any access.
-*/
-   for (;;) {
-   /*
-* Try the uncontested case.  This should be done in userland.
-*/
-   owner = casuword(&umtx->u_owner, UMTX_UNOWNED, id);
-
-   /* The acquire succeeded. */
-   if (owner == UMTX_UNOWNED)
-   return (0);
-
-   /* The address was invalid. */
-   if (owner == -1)
-   return (EFAULT);
-
-   /* If no one owns it but it is contested try to acquire it. */
-   if (owner == UMTX_CONTESTED) {
-   owner = casuword(&umtx->u_owner,
-   UMTX_CONTESTED, id | UMTX_CONTESTED);
-
-   if (owner == UMTX_CONTESTED)
-   return (0);
-
-   /* The address was invalid. */
-   if (owner == -1)
-   return (EFAULT);
-
-   error = umtxq_check_susp(td);
-   if (error != 0)
-   break;
-
-   /* If this failed the lock has changed, restart. */
-   continue;
-   }
-
-   /*
-* If we caught a signal, we have retried and now
-* exit immediately.
-*/
-   if (error != 0)
-   break;
-
-   if ((error = umtx_key_get(umtx, TYPE_SIMPLE_LOCK,
-   AUTO_SHARE, &uq->uq_key)) != 0)
-   return (error);
-
-   umtxq_lock(&uq->uq_key);
-   umtxq_busy(&uq->uq_key);
-   umtxq_insert(uq);
-   umtxq_unbusy(&uq->uq_key);
-   umtxq_unlock(&uq->uq_key);
-
-   /*
-* Set the contested bit so that a release in user space
-* knows to use the system call for unlock.  If this fails
-* either some one else has acquired the lock or it has been
-* released.
-*/
-   old = casuword(&umtx->u_owner, owner, owner | UMTX_CONTESTED);
-
-   /* The address was invalid. */
-   if (old == -1) {
- 

svn commit: r262291 - stable/10/sys/vm

2014-02-21 Thread Attilio Rao
Author: attilio
Date: Fri Feb 21 09:43:34 2014
New Revision: 262291
URL: http://svnweb.freebsd.org/changeset/base/262291

Log:
  MFC r261867:
  Use the right index to free swapspace after vm_page_rename().

Modified:
  stable/10/sys/vm/vm_object.c
Directory Properties:
  stable/10/   (props changed)

Modified: stable/10/sys/vm/vm_object.c
==
--- stable/10/sys/vm/vm_object.cFri Feb 21 09:42:50 2014
(r262290)
+++ stable/10/sys/vm/vm_object.cFri Feb 21 09:43:34 2014
(r262291)
@@ -1628,9 +1628,11 @@ vm_object_backing_scan(vm_object_t objec
p = TAILQ_FIRST(&backing_object->memq);
continue;
}
+
+   /* Use the old pindex to free the right page. */
if (backing_object->type == OBJT_SWAP)
-   swap_pager_freespace(backing_object, p->pindex,
-   1);
+   swap_pager_freespace(backing_object,
+   new_pindex + backing_offset_index, 1);
 
 #if VM_NRESERVLEVEL > 0
/*
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r261867 - head/sys/vm

2014-02-13 Thread Attilio Rao
Author: attilio
Date: Fri Feb 14 03:34:12 2014
New Revision: 261867
URL: http://svnweb.freebsd.org/changeset/base/261867

Log:
  Fix-up r254141: in the process of making a failing vm_page_rename()
  a call of pager_swap_freespace() was moved around, now leading to freeing
  the incorrect page because of the pindex changes after vm_page_rename().
  
  Get back to use the correct pindex when destroying the swap space.
  
  Sponsored by: EMC / Isilon storage division
  Reported by:  avg
  Tested by:pho
  MFC after:7 days

Modified:
  head/sys/vm/vm_object.c

Modified: head/sys/vm/vm_object.c
==
--- head/sys/vm/vm_object.c Fri Feb 14 03:30:55 2014(r261866)
+++ head/sys/vm/vm_object.c Fri Feb 14 03:34:12 2014(r261867)
@@ -1627,9 +1627,11 @@ vm_object_backing_scan(vm_object_t objec
p = TAILQ_FIRST(&backing_object->memq);
continue;
}
+
+   /* Use the old pindex to free the right page. */
if (backing_object->type == OBJT_SWAP)
-   swap_pager_freespace(backing_object, p->pindex,
-   1);
+   swap_pager_freespace(backing_object,
+   new_pindex + backing_offset_index, 1);
 
 #if VM_NRESERVLEVEL > 0
/*
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r260393 - head/sys/sys

2014-01-07 Thread Attilio Rao
Author: attilio
Date: Tue Jan  7 14:03:42 2014
New Revision: 260393
URL: http://svnweb.freebsd.org/changeset/base/260393

Log:
  Use __predict_false() on sensitive lock paths as most of the times,
  when PMC-soft feature is not used the check will be false.
  
  Sponsored by: EMC / Isilon storage division
  Submitted by: Anton Rang 

Modified:
  head/sys/sys/pmckern.h

Modified: head/sys/sys/pmckern.h
==
--- head/sys/sys/pmckern.h  Tue Jan  7 13:09:35 2014(r260392)
+++ head/sys/sys/pmckern.h  Tue Jan  7 14:03:42 2014(r260393)
@@ -110,7 +110,7 @@ struct pmckern_soft {
 #ifdef PMC_FAKE_TRAPFRAME
 #define PMC_SOFT_CALL(pr, mo, fu, na)  
\
 do {   
\
-   if (pmc_##pr##_##mo##_##fu##_##na.ps_running) { 
\
+   if (__predict_false(pmc_##pr##_##mo##_##fu##_##na.ps_running)) {
\
struct pmckern_soft ks; 
\
register_t intr;
\
intr = intr_disable();  
\
@@ -135,7 +135,7 @@ do {
\
  */
 #define PMC_SOFT_CALL_TF(pr, mo, fu, na, tf)   
\
 do {   
\
-   if (pmc_##pr##_##mo##_##fu##_##na.ps_running) { 
\
+   if (__predict_false(pmc_##pr##_##mo##_##fu##_##na.ps_running)) {
\
struct pmckern_soft ks; 
\
register_t intr;
\
intr = intr_disable();  
\
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r259647 - head/sys/dev/hwpmc

2013-12-20 Thread Attilio Rao
Author: attilio
Date: Fri Dec 20 14:03:56 2013
New Revision: 259647
URL: http://svnweb.freebsd.org/changeset/base/259647

Log:
  o Remove assertions on ipa_version as sometimes the version detection
using cpuid can be quirky (this is the case of VMWare without the
vPMC support) but fail to probe hwpmc.
  o Apply the fix for XEON family of processors as established by
315338-020 document (bug AJ85).
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  fabient

Modified:
  head/sys/dev/hwpmc/hwpmc_core.c
  head/sys/dev/hwpmc/hwpmc_core.h
  head/sys/dev/hwpmc/hwpmc_intel.c

Modified: head/sys/dev/hwpmc/hwpmc_core.c
==
--- head/sys/dev/hwpmc/hwpmc_core.c Fri Dec 20 13:18:50 2013
(r259646)
+++ head/sys/dev/hwpmc/hwpmc_core.c Fri Dec 20 14:03:56 2013
(r259647)
@@ -2627,35 +2627,33 @@ core2_intr(int cpu, struct trapframe *tf
 }
 
 int
-pmc_core_initialize(struct pmc_mdep *md, int maxcpu)
+pmc_core_initialize(struct pmc_mdep *md, int maxcpu, int version_override)
 {
int cpuid[CORE_CPUID_REQUEST_SIZE];
int ipa_version, flags, nflags;
 
do_cpuid(CORE_CPUID_REQUEST, cpuid);
 
-   ipa_version = cpuid[CORE_CPUID_EAX] & 0xFF;
+   ipa_version = (version_override > 0) ? version_override :
+   cpuid[CORE_CPUID_EAX] & 0xFF;
+   core_cputype = md->pmd_cputype;
 
PMCDBG(MDP,INI,1,"core-init cputype=%d ncpu=%d ipa-version=%d",
-   md->pmd_cputype, maxcpu, ipa_version);
+   core_cputype, maxcpu, ipa_version);
 
-   if (ipa_version < 1 || ipa_version > 3) {
+   if (ipa_version < 1 || ipa_version > 3 ||
+   (core_cputype != PMC_CPU_INTEL_CORE && ipa_version == 1)) {
/* Unknown PMC architecture. */
printf("hwpc_core: unknown PMC architecture: %d\n",
ipa_version);
return (EPROGMISMATCH);
}
 
-   core_cputype = md->pmd_cputype;
-
core_pmcmask = 0;
 
/*
 * Initialize programmable counters.
 */
-   KASSERT(ipa_version >= 1,
-   ("[core,%d] ipa_version %d too small", __LINE__, ipa_version));
-
core_iap_npmc = (cpuid[CORE_CPUID_EAX] >> 8) & 0xFF;
core_iap_width = (cpuid[CORE_CPUID_EAX] >> 16) & 0xFF;
 
@@ -2670,10 +2668,6 @@ pmc_core_initialize(struct pmc_mdep *md,
 * Initialize fixed function counters, if present.
 */
if (core_cputype != PMC_CPU_INTEL_CORE) {
-   KASSERT(ipa_version >= 2,
-   ("[core,%d] ipa_version %d too small", __LINE__,
-   ipa_version));
-
core_iaf_ri = core_iap_npmc;
core_iaf_npmc = cpuid[CORE_CPUID_EDX] & 0x1F;
core_iaf_width = (cpuid[CORE_CPUID_EDX] >> 5) & 0xFF;

Modified: head/sys/dev/hwpmc/hwpmc_core.h
==
--- head/sys/dev/hwpmc/hwpmc_core.h Fri Dec 20 13:18:50 2013
(r259646)
+++ head/sys/dev/hwpmc/hwpmc_core.h Fri Dec 20 14:03:56 2013
(r259647)
@@ -175,7 +175,8 @@ struct pmc_md_iap_pmc {
  * Prototypes.
  */
 
-intpmc_core_initialize(struct pmc_mdep *_md, int _maxcpu);
+intpmc_core_initialize(struct pmc_mdep *_md, int _maxcpu,
+   int _version_override);
 void   pmc_core_finalize(struct pmc_mdep *_md);
 
 void   pmc_core_mark_started(int _cpu, int _pmc);

Modified: head/sys/dev/hwpmc/hwpmc_intel.c
==
--- head/sys/dev/hwpmc/hwpmc_intel.cFri Dec 20 13:18:50 2013
(r259646)
+++ head/sys/dev/hwpmc/hwpmc_intel.cFri Dec 20 14:03:56 2013
(r259647)
@@ -78,7 +78,7 @@ pmc_intel_initialize(void)
 {
struct pmc_mdep *pmc_mdep;
enum pmc_cputype cputype;
-   int error, model, nclasses, ncpus;
+   int error, model, nclasses, ncpus, stepping, verov;
 
KASSERT(cpu_vendor_id == CPU_VENDOR_INTEL,
("[intel,%d] Initializing non-intel processor", __LINE__));
@@ -88,7 +88,9 @@ pmc_intel_initialize(void)
cputype = -1;
nclasses = 2;
error = 0;
+   verov = 0;
model = ((cpu_id & 0xF) >> 12) | ((cpu_id & 0xF0) >> 4);
+   stepping = cpu_id & 0xF;
 
switch (cpu_id & 0xF00) {
 #ifdefined(__i386__)
@@ -119,8 +121,14 @@ pmc_intel_initialize(void)
cputype = PMC_CPU_INTEL_CORE;
break;
case 0xF:
-   cputype = PMC_CPU_INTEL_CORE2;
-   nclasses = 3;
+   /* Per Intel document 315338-020. */
+   if (stepping == 0x7) {
+   cputype = PMC_CPU_INTEL_CORE;
+   verov = 1;
+   } else {
+   cputype = PMC_CPU_INTEL_CORE2;
+   

svn commit: r259509 - head/sys/kern

2013-12-17 Thread Attilio Rao
Author: attilio
Date: Tue Dec 17 13:37:02 2013
New Revision: 259509
URL: http://svnweb.freebsd.org/changeset/base/259509

Log:
  - Assert for not leaking readers rw locks counter on userland return.
  - Use a correct spin_cnt for KDTRACE_HOOK case in rw read lock.
  
  Sponsored by: EMC / Isilon storage division

Modified:
  head/sys/kern/kern_rwlock.c
  head/sys/kern/subr_trap.c

Modified: head/sys/kern/kern_rwlock.c
==
--- head/sys/kern/kern_rwlock.c Tue Dec 17 13:18:41 2013(r259508)
+++ head/sys/kern/kern_rwlock.c Tue Dec 17 13:37:02 2013(r259509)
@@ -440,6 +440,9 @@ __rw_rlock(volatile uintptr_t *c, const 
break;
cpu_spinwait();
}
+#ifdef KDTRACE_HOOKS
+   spin_cnt += rowner_loops - i;
+#endif
if (i != rowner_loops)
continue;
}

Modified: head/sys/kern/subr_trap.c
==
--- head/sys/kern/subr_trap.c   Tue Dec 17 13:18:41 2013(r259508)
+++ head/sys/kern/subr_trap.c   Tue Dec 17 13:37:02 2013(r259509)
@@ -152,6 +152,9 @@ userret(struct thread *td, struct trapfr
("userret: Returning in a critical section"));
KASSERT(td->td_locks == 0,
("userret: Returning with %d locks held", td->td_locks));
+   KASSERT(td->td_rw_rlocks == 0,
+   ("userret: Returning with %d rwlocks held in read mode",
+   td->td_rw_rlocks));
KASSERT((td->td_pflags & TDP_NOFAULTING) == 0,
("userret: Returning with pagefaults disabled"));
KASSERT(td->td_no_sleeping == 0,
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r258858 - head/sys/sys

2013-12-02 Thread Attilio Rao
Author: attilio
Date: Mon Dec  2 22:34:47 2013
New Revision: 258858
URL: http://svnweb.freebsd.org/changeset/base/258858

Log:
  Right now LOCK_DEBUG is enabled if KTR is on.
  This is to support LOCK_LOG_* functionality effectively in debugging
  environments but it is overkill because really LOCK_DEBUG should be on
  only if (KTR_COMPILE & KTR_LOCK) is true.
  Fix this by applying the correct logic.
  In this process, move the KTR classes to its own header to reduce
  namespace pollution.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  jhb

Added:
  head/sys/sys/ktr_class.h   (contents, props changed)
Modified:
  head/sys/sys/ktr.h
  head/sys/sys/lock.h

Modified: head/sys/sys/ktr.h
==
--- head/sys/sys/ktr.h  Mon Dec  2 22:00:15 2013(r258857)
+++ head/sys/sys/ktr.h  Mon Dec  2 22:34:47 2013(r258858)
@@ -36,56 +36,7 @@
 #ifndef _SYS_KTR_H_
 #define _SYS_KTR_H_
 
-/*
- * Trace classes
- *
- * Two of the trace classes (KTR_DEV and KTR_SUBSYS) are special in that
- * they are really placeholders so that indvidual drivers and subsystems
- * can map their internal tracing to the general class when they wish to
- * have tracing enabled and map it to 0 when they don't.
- */
-#defineKTR_GEN 0x0001  /* General (TR) */
-#defineKTR_NET 0x0002  /* Network */
-#defineKTR_DEV 0x0004  /* Device driver */
-#defineKTR_LOCK0x0008  /* MP locking */
-#defineKTR_SMP 0x0010  /* MP general */
-#defineKTR_SUBSYS  0x0020  /* Subsystem. */
-#defineKTR_PMAP0x0040  /* Pmap tracing */
-#defineKTR_MALLOC  0x0080  /* Malloc tracing */
-#defineKTR_TRAP0x0100  /* Trap processing */
-#defineKTR_INTR0x0200  /* Interrupt tracing */
-#defineKTR_SIG 0x0400  /* Signal processing */
-#defineKTR_SPARE2  0x0800  /* XXX Used by cxgb */
-#defineKTR_PROC0x1000  /* Process scheduling */
-#defineKTR_SYSC0x2000  /* System call */
-#defineKTR_INIT0x4000  /* System 
initialization */
-#defineKTR_SPARE3  0x8000  /* XXX Used by cxgb */
-#defineKTR_SPARE4  0x0001  /* XXX Used by cxgb */
-#defineKTR_EVH 0x0002  /* Eventhandler */
-#defineKTR_VFS 0x0004  /* VFS events */
-#defineKTR_VOP 0x0008  /* Auto-generated vop 
events */
-#defineKTR_VM  0x0010  /* The virtual memory 
system */
-#defineKTR_INET0x0020  /* IPv4 stack */
-#defineKTR_RUNQ0x0040  /* Run queue */
-#defineKTR_CONTENTION  0x0080  /* Lock contention */
-#defineKTR_UMA 0x0100  /* UMA slab allocator */
-#defineKTR_CALLOUT 0x0200  /* Callouts and 
timeouts */
-#defineKTR_GEOM0x0400  /* GEOM I/O events */
-#defineKTR_BUSDMA  0x0800  /* busdma(9) events */
-#defineKTR_INET6   0x1000  /* IPv6 stack */
-#defineKTR_SCHED   0x2000  /* Machine parsed sched 
info. */
-#defineKTR_BUF 0x4000  /* Buffer cache */
-#defineKTR_ALL 0x7fff
-
-/* Trace classes to compile in */
-#ifdef KTR
-#ifndef KTR_COMPILE
-#defineKTR_COMPILE (KTR_ALL)
-#endif
-#else  /* !KTR */
-#undef KTR_COMPILE
-#define KTR_COMPILE 0
-#endif /* KTR */
+#include 
 
 /*
  * Version number for ktr_entry struct.  Increment this when you break binary

Added: head/sys/sys/ktr_class.h
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ head/sys/sys/ktr_class.hMon Dec  2 22:34:47 2013(r258858)
@@ -0,0 +1,86 @@
+/*-
+ * Copyright (c) 1996 Berkeley Software Design, Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in the
+ *documentation and/or other materials provided with the distribution.
+ * 3. Berkeley Software Design Inc's name may not be used to endorse or

svn commit: r258541 - in head/sys: amd64/amd64 amd64/linux32 cam/ctl cddl/dev/dtrace cddl/dev/lockstat cddl/dev/sdt compat/linux conf dev/sfxge/common dev/xen/blkback fs/nfs fs/nfsclient i386/i386 ...

2013-11-24 Thread Attilio Rao
Author: attilio
Date: Mon Nov 25 07:38:45 2013
New Revision: 258541
URL: http://svnweb.freebsd.org/changeset/base/258541

Log:
  - For kernel compiled only with KDTRACE_HOOKS and not any lock debugging
option, unbreak the lock tracing release semantic by embedding
calls to LOCKSTAT_PROFILE_RELEASE_LOCK() direclty in the inlined
version of the releasing functions for mutex, rwlock and sxlock.
Failing to do so skips the lockstat_probe_func invokation for
unlocking.
  - As part of the LOCKSTAT support is inlined in mutex operation, for
kernel compiled without lock debugging options, potentially every
consumer must be compiled including opt_kdtrace.h.
Fix this by moving KDTRACE_HOOKS into opt_global.h and remove the
dependency by opt_kdtrace.h for all files, as now only KDTRACE_FRAMES
is linked there and it is only used as a compile-time stub [0].
  
  [0] immediately shows some new bug as DTRACE-derived support for debug
  in sfxge is broken and it was never really tested.  As it was not
  including correctly opt_kdtrace.h before it was never enabled so it
  was kept broken for a while.  Fix this by using a protection stub,
  leaving sfxge driver authors the responsibility for fixing it
  appropriately [1].
  
  Sponsored by: EMC / Isilon storage division
  Discussed with:   rstone
  [0] Reported by:  rstone
  [1] Discussed with:   philip

Modified:
  head/sys/amd64/amd64/exception.S
  head/sys/amd64/amd64/machdep.c
  head/sys/amd64/amd64/trap.c
  head/sys/amd64/linux32/linux32_dummy.c
  head/sys/cam/ctl/ctl_backend_block.c
  head/sys/cddl/dev/dtrace/dtrace_test.c
  head/sys/cddl/dev/lockstat/lockstat.c
  head/sys/cddl/dev/sdt/sdt.c
  head/sys/compat/linux/linux_emul.c
  head/sys/compat/linux/linux_fork.c
  head/sys/compat/linux/linux_futex.c
  head/sys/compat/linux/linux_mib.c
  head/sys/compat/linux/linux_misc.c
  head/sys/compat/linux/linux_sysctl.c
  head/sys/compat/linux/linux_time.c
  head/sys/compat/linux/linux_uid16.c
  head/sys/compat/linux/linux_util.c
  head/sys/conf/options
  head/sys/dev/sfxge/common/efsys.h
  head/sys/dev/xen/blkback/blkback.c
  head/sys/fs/nfs/nfs_commonkrpc.c
  head/sys/fs/nfsclient/nfs_clbio.c
  head/sys/fs/nfsclient/nfs_clnode.c
  head/sys/fs/nfsclient/nfs_clport.c
  head/sys/fs/nfsclient/nfs_clsubs.c
  head/sys/fs/nfsclient/nfs_clvnops.c
  head/sys/i386/i386/exception.s
  head/sys/i386/i386/machdep.c
  head/sys/i386/i386/trap.c
  head/sys/i386/linux/linux_dummy.c
  head/sys/kern/kern_clock.c
  head/sys/kern/kern_clocksource.c
  head/sys/kern/kern_exec.c
  head/sys/kern/kern_exit.c
  head/sys/kern/kern_fork.c
  head/sys/kern/kern_lock.c
  head/sys/kern/kern_lockstat.c
  head/sys/kern/kern_malloc.c
  head/sys/kern/kern_mutex.c
  head/sys/kern/kern_priv.c
  head/sys/kern/kern_proc.c
  head/sys/kern/kern_racct.c
  head/sys/kern/kern_rmlock.c
  head/sys/kern/kern_rwlock.c
  head/sys/kern/kern_sdt.c
  head/sys/kern/kern_sig.c
  head/sys/kern/kern_sx.c
  head/sys/kern/kern_synch.c
  head/sys/kern/kern_thread.c
  head/sys/kern/kern_timeout.c
  head/sys/kern/sched_4bsd.c
  head/sys/kern/sched_ule.c
  head/sys/kern/subr_devstat.c
  head/sys/kern/subr_sleepqueue.c
  head/sys/kern/subr_syscall.c
  head/sys/kern/subr_trap.c
  head/sys/kern/subr_turnstile.c
  head/sys/kern/vfs_cache.c
  head/sys/kern/vfs_lookup.c
  head/sys/kern/vfs_syscalls.c
  head/sys/mips/mips/exception.S
  head/sys/mips/mips/trap.c
  head/sys/modules/crypto/Makefile
  head/sys/modules/ctl/Makefile
  head/sys/modules/dtrace/dtrace_test/Makefile
  head/sys/modules/dtrace/lockstat/Makefile
  head/sys/modules/dtrace/sdt/Makefile
  head/sys/modules/ip6_mroute_mod/Makefile
  head/sys/modules/linux/Makefile
  head/sys/modules/nfscl/Makefile
  head/sys/modules/nfsclient/Makefile
  head/sys/modules/nfscommon/Makefile
  head/sys/modules/send/Makefile
  head/sys/net/vnet.c
  head/sys/netinet/in_kdtrace.c
  head/sys/netinet/ip_fastfwd.c
  head/sys/netinet/ip_input.c
  head/sys/netinet/ip_output.c
  head/sys/netinet/sctp_dtrace_declare.h
  head/sys/netinet/sctp_dtrace_define.h
  head/sys/netinet/tcp_input.c
  head/sys/netinet/tcp_output.c
  head/sys/netinet/tcp_subr.c
  head/sys/netinet/udp_usrreq.c
  head/sys/netinet6/ip6_input.c
  head/sys/netinet6/ip6_mroute.c
  head/sys/netinet6/nd6.c
  head/sys/netinet6/send.c
  head/sys/netinet6/udp6_usrreq.c
  head/sys/nfsclient/nfs_bio.c
  head/sys/nfsclient/nfs_krpc.c
  head/sys/nfsclient/nfs_subs.c
  head/sys/nfsclient/nfs_vnops.c
  head/sys/opencrypto/crypto.c
  head/sys/opencrypto/deflate.c
  head/sys/pc98/pc98/machdep.c
  head/sys/powerpc/aim/locore32.S
  head/sys/powerpc/aim/locore64.S
  head/sys/powerpc/aim/trap.c
  head/sys/security/mac/mac_audit.c
  head/sys/security/mac/mac_cred.c
  head/sys/security/mac/mac_framework.c
  head/sys/security/mac/mac_inet.c
  head/sys/security/mac/mac_net.c
  head/sys/security/mac/mac_pipe.c
  head/sys/security/mac/mac_posix_sem.c
  head/sys/security/mac/mac_posix_shm.c
  head/sys/security/mac/mac_pr

svn commit: r255868 - head/sys/kern

2013-09-25 Thread Attilio Rao
Author: attilio
Date: Wed Sep 25 13:37:52 2013
New Revision: 255868
URL: http://svnweb.freebsd.org/changeset/base/255868

Log:
  Avoid memory accesses reordering which can result in fget_unlocked()
  seeing a stale fd_ofiles table once fd_nfiles is already updated,
  resulting in OOB accesses.
  
  Approved by:  re (kib)
  Sponsored by: EMC / Isilon storage division
  Reported and tested by:   pho
  Reviewed by:  benno

Modified:
  head/sys/kern/kern_descrip.c

Modified: head/sys/kern/kern_descrip.c
==
--- head/sys/kern/kern_descrip.cWed Sep 25 02:49:18 2013
(r255867)
+++ head/sys/kern/kern_descrip.cWed Sep 25 13:37:52 2013
(r255868)
@@ -1512,12 +1512,20 @@ fdgrowtable(struct filedesc *fdp, int nf
memcpy(nmap, omap, NDSLOTS(onfiles) * sizeof(*omap));
 
/* update the pointers and counters */
-   fdp->fd_nfiles = nnfiles;
memcpy(ntable, otable, onfiles * sizeof(ntable[0]));
fdp->fd_ofiles = ntable;
fdp->fd_map = nmap;
 
/*
+* In order to have a valid pattern for fget_unlocked()
+* fdp->fd_nfiles might be the last member to be updated, otherwise
+* fget_unlocked() consumers may reference a new, higher value for
+* fdp->fd_nfiles before to access the fdp->fd_ofiles array,
+* resulting in OOB accesses.
+*/
+   atomic_store_rel_int(&fdp->fd_nfiles, nnfiles);
+
+   /*
 * Do not free the old file table, as some threads may still
 * reference entries within it.  Instead, place it on a freelist
 * which will be processed when the struct filedesc is released.
@@ -2308,7 +2316,11 @@ fget_unlocked(struct filedesc *fdp, int 
int error;
 #endif
 
-   if (fd < 0 || fd >= fdp->fd_nfiles)
+   /*
+* Avoid reads reordering and then a first access to the
+* fdp->fd_ofiles table which could result in OOB operation.
+*/
+   if (fd < 0 || fd >= atomic_load_acq_int(&fdp->fd_nfiles))
return (EBADF);
/*
 * Fetch the descriptor locklessly.  We avoid fdrop() races by
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r255797 - head/sys/kern

2013-09-22 Thread Attilio Rao
On Sun, Sep 22, 2013 at 9:34 PM, Konstantin Belousov
 wrote:
> On Sun, Sep 22, 2013 at 11:19:16PM +0300, Konstantin Belousov wrote:
>> On Sun, Sep 22, 2013 at 01:14:21PM -0700, Matthew Fleming wrote:
>> > On Sun, Sep 22, 2013 at 12:23 PM, Konstantin Belousov 
>> > wrote:
>> >> > > Author: kib
>> > > Date: Sun Sep 22 19:23:48 2013
>> > > New Revision: 255797
>> > > URL: http://svnweb.freebsd.org/changeset/base/255797
>> > >
>> > > Log:
>> > >   Increase the chance of the buffer write from the bufdaemon helper
>> > >   context to succeed.  If the locked vnode which owns the buffer to be
>> > >   written is shared locked, try the non-blocking upgrade of the lock to
>> > >   exclusive.
>> > >
>> > >   PR:   kern/178997
>> > >   Reported and tested by:   Klaus Weber <
>> > > fbsd-bugs-201...@unix-admin.de>
>> > >   Sponsored by: The FreeBSD Foundation
>> > >   MFC after:1 week
>> > >   Approved by:  re (marius)
>> > >
>> > > Modified:
>> > >   head/sys/kern/vfs_bio.c
>> > >
>> > > Modified: head/sys/kern/vfs_bio.c
>> > >
>> > > ==
>> > > --- head/sys/kern/vfs_bio.c Sun Sep 22 19:15:24 2013(r255796)
>> > > +++ head/sys/kern/vfs_bio.c Sun Sep 22 19:23:48 2013(r255797)
>> > > @@ -2624,6 +2624,8 @@ flushbufqueues(struct vnode *lvp, int ta
>> > > int hasdeps;
>> > > int flushed;
>> > > int queue;
>> > > +   int error;
>> > > +   bool unlock;
>> > >
>> > > flushed = 0;
>> > > queue = QUEUE_DIRTY;
>> > > @@ -2699,7 +2701,16 @@ flushbufqueues(struct vnode *lvp, int ta
>> > > BUF_UNLOCK(bp);
>> > > continue;
>> > > }
>> > > -   if (vn_lock(vp, LK_EXCLUSIVE | LK_NOWAIT | LK_CANRECURSE)
>> > > == 0) {
>> > > +   if (lvp == NULL) {
>> > > +   unlock = true;
>> > > +   error = vn_lock(vp, LK_EXCLUSIVE | LK_NOWAIT);
>> > > +   } else {
>> > > +   ASSERT_VOP_LOCKED(vp, "getbuf");
>> > > +   unlock = false;
>> > > +   error = VOP_ISLOCKED(vp) == LK_EXCLUSIVE ? 0 :
>> > > +   vn_lock(vp, LK_UPGRADE | LK_NOWAIT);
>> > >
>> >
>> > I don't think this is quite right.
>> >
>> > When the lock is held shared, and VOP_LOCK is implemented by lockmgr(9),
>> > (i.e. all in-tree filesystems?), LK_UPGRADE may drop the lock, and not
>> > reacquire it.  This would happen when the vnode is locked shared, the
>> > upgrade fails (2 shared owners), then lockmgr(9) will try to lock EX, which
>> > will also fail (still one shared owner).  The caller's lock is no longer
>> > held.
>> >
>> > Doesn't that scenario (LK_UPGRADE failing) cause problems both for the
>> > caller (unexpected unlock) and for flushbufqueues(), which expects the
>> > vnode lock to be held since lvp is non-NULL?
>>
>> Does it ? If the lock is dropped, the code is indeed in trouble.
>> Please note that LK_NOWAIT is specified for upgrade, and I believe
>> that this causes lockmgr to return with EBUSY without dropping
>> the lock.
>
> Yes, you are right, I reverted the patch.  Thank you for noting this.
>
> I am bitten by unreasonable behaviour of non-blocking upgrade once more.
> It has a history.
>
> Some time ago I proposed the following patch, which was turned down.
> That time, I was able to work-around the case. For the bufdaemon helper,
> I do not see any way to avoid this, except of sometimes locking the
> reader vnode exclusive in anticipation of the too high dirty buffer
> mark.

If you are speaking about me, you are mistaken, I never turned out this patch.
What I said is completely different: I said that LK_UPGRADE is a
completely wrong semantic because it can hide wrong things like the
one you hit today.
I wanted to see it removed and replaced by explicit LK_RELEASE +
LK_EXCLUSIVE operations.
Note that this would have avoided this patch.

I'm completely in favour of LK_TRYUPGRADE.

Attilio
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r254362 - head/sys/vm

2013-08-15 Thread Attilio Rao
Author: attilio
Date: Thu Aug 15 11:01:25 2013
New Revision: 254362
URL: http://svnweb.freebsd.org/changeset/base/254362

Log:
  On the recovery path for vm_page_alloc(), if a page had been requested
  wired, unwind back the wiring bits otherwise we can end up freeing a
  page that is considered wired.
  
  Sponsored by: EMC / Isilon storage division
  Reported by:  alc

Modified:
  head/sys/vm/vm_page.c

Modified: head/sys/vm/vm_page.c
==
--- head/sys/vm/vm_page.c   Thu Aug 15 10:38:10 2013(r254361)
+++ head/sys/vm/vm_page.c   Thu Aug 15 11:01:25 2013(r254362)
@@ -1611,6 +1611,10 @@ vm_page_alloc(vm_object_t object, vm_pin
if (vp != NULL)
vdrop(vp);
pagedaemon_wakeup();
+   if (req & VM_ALLOC_WIRED) {
+   atomic_subtract_int(&cnt.v_wire_count, 1);
+   m->wire_count = 0;
+   }
m->object = NULL;
vm_page_free(m);
return (NULL);
@@ -1806,8 +1810,13 @@ retry:
&deferred_vdrop_list);
if (vm_paging_needed())
pagedaemon_wakeup();
+   if ((req & VM_ALLOC_WIRED) != 0)
+   atomic_subtract_int(&cnt.v_wire_count,
+   npages);
for (m_tmp = m, m = m_ret;
m < &m_ret[npages]; m++) {
+   if ((req & VM_ALLOC_WIRED) != 0)
+   m->wire_count = 0;
if (m >= m_tmp)
m->object = NULL;
vm_page_free(m);
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r254138 - in head: share/man/man9 sys/amd64/amd64 sys/arm/arm sys/cddl/contrib/opensolaris/uts/common/fs/zfs sys/dev/agp sys/dev/drm2/i915 sys/dev/drm2/ttm sys/dev/md sys/fs/fuse sys/f

2013-08-13 Thread Attilio Rao
On Tue, Aug 13, 2013 at 4:22 PM, Ulrich Spörlein  wrote:
> On Fri, 2013-08-09 at 11:11:12 +0000, Attilio Rao wrote:
>> Author: attilio
>> Date: Fri Aug  9 11:11:11 2013
>> New Revision: 254138
>> URL: http://svnweb.freebsd.org/changeset/base/254138
>>
>> Log:
>>   The soft and hard busy mechanism rely on the vm object lock to work.
>>   Unify the 2 concept into a real, minimal, sxlock where the shared
>>   acquisition represent the soft busy and the exclusive acquisition
>>   represent the hard busy.
>>   The old VPO_WANTED mechanism becames the hard-path for this new lock
>>   and it becomes per-page rather than per-object.
>>   The vm_object lock becames an interlock for this functionality:
>>   it can be held in both read or write mode.
>>   However, if the vm_object lock is held in read mode while acquiring
>>   or releasing the busy state, the thread owner cannot make any
>>   assumption on the busy state unless it is also busying it.
>>
>>   Also:
>>   - Add a new flag to directly shared busy pages while vm_page_alloc
>> and vm_page_grab are being executed.  This will be very helpful
>> once these functions happen under a read object lock.
>>   - Move the swapping sleep into its own per-object flag
>>
>>   The KPI is heavilly changed this is why the version is bumped.
>>   It is very likely that some VM ports users will need to change
>>   their own code.
>>
>>   Sponsored by:   EMC / Isilon storage division
>>   Discussed with: alc
>>   Reviewed by:jeff, kib
>>   Tested by:  gavin, bapt (older version)
>>   Tested by:  pho, scottl
>
> The changes to sys/vm/vm_fault.c introduce a call to
> vm_page_sleep_if_busy() where the return code is not checked. The other
> 5 places in the tree check the return code, please fix this here too.
> It's CID 1062398, and I would encourage folks to get an account with
> scan.coverity.com and have an eye on newly found defects.

Not true.
The same call existed also before with exactly the same semantic.
The trick there is that it is not important to check for the return
value because we are going to retry the operation anyway.
The code looks ok to me.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"

svn commit: r254228 - head/sys/vm

2013-08-11 Thread Attilio Rao
Author: attilio
Date: Sun Aug 11 21:15:04 2013
New Revision: 254228
URL: http://svnweb.freebsd.org/changeset/base/254228

Log:
  Correct the recovery logic in vm_page_alloc_contig:
  what is really needed on this code snipped is that all the pages that
  are already fully inserted gets fully freed, while for the others the
  object removal itself might be skipped, hence the object might be set to
  NULL.
  
  Sponsored by: EMC / Isilon storage division
  Reported by:  alc, kib
  Reviewed by:  alc

Modified:
  head/sys/vm/vm_page.c

Modified: head/sys/vm/vm_page.c
==
--- head/sys/vm/vm_page.c   Sun Aug 11 20:46:05 2013(r254227)
+++ head/sys/vm/vm_page.c   Sun Aug 11 21:15:04 2013(r254228)
@@ -1807,11 +1807,9 @@ retry:
&deferred_vdrop_list);
if (vm_paging_needed())
pagedaemon_wakeup();
-   for (m = m_ret, m_tmp = m_ret;
+   for (m_tmp = m, m = m_ret;
m < &m_ret[npages]; m++) {
-   if (m_tmp < m)
-   m_tmp++;
-   else
+   if (m >= m_tmp)
m->object = NULL;
vm_page_free(m);
}
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r254141 - in head/sys: amd64/amd64 dev/drm2/i915 dev/drm2/ttm i386/i386 kern vm

2013-08-09 Thread Attilio Rao
Author: attilio
Date: Fri Aug  9 11:28:55 2013
New Revision: 254141
URL: http://svnweb.freebsd.org/changeset/base/254141

Log:
  On all the architectures, avoid to preallocate the physical memory
  for nodes used in vm_radix.
  On architectures supporting direct mapping, also avoid to pre-allocate
  the KVA for such nodes.
  
  In order to do so make the operations derived from vm_radix_insert()
  to fail and handle all the deriving failure of those.
  
  vm_radix-wise introduce a new function called vm_radix_replace(),
  which can replace a leaf node, already present, with a new one,
  and take into account the possibility, during vm_radix_insert()
  allocation, that the operations on the radix trie can recurse.
  This means that if operations in vm_radix_insert() recursed
  vm_radix_insert() will start from scratch again.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  alc (older version)
  Reviewed by:  jeff
  Tested by:pho, scottl

Modified:
  head/sys/amd64/amd64/pmap.c
  head/sys/dev/drm2/i915/i915_gem.c
  head/sys/dev/drm2/ttm/ttm_bo_vm.c
  head/sys/i386/i386/pmap.c
  head/sys/kern/subr_uio.c
  head/sys/vm/_vm_radix.h
  head/sys/vm/device_pager.c
  head/sys/vm/sg_pager.c
  head/sys/vm/vm_fault.c
  head/sys/vm/vm_object.c
  head/sys/vm/vm_object.h
  head/sys/vm/vm_page.c
  head/sys/vm/vm_page.h
  head/sys/vm/vm_radix.c
  head/sys/vm/vm_radix.h

Modified: head/sys/amd64/amd64/pmap.c
==
--- head/sys/amd64/amd64/pmap.c Fri Aug  9 11:26:26 2013(r254140)
+++ head/sys/amd64/amd64/pmap.c Fri Aug  9 11:28:55 2013(r254141)
@@ -283,7 +283,7 @@ static boolean_t pmap_enter_pde(pmap_t p
 static vm_page_t pmap_enter_quick_locked(pmap_t pmap, vm_offset_t va,
 vm_page_t m, vm_prot_t prot, vm_page_t mpte, struct rwlock **lockp);
 static void pmap_fill_ptp(pt_entry_t *firstpte, pt_entry_t newpte);
-static void pmap_insert_pt_page(pmap_t pmap, vm_page_t mpte);
+static int pmap_insert_pt_page(pmap_t pmap, vm_page_t mpte);
 static boolean_t pmap_is_modified_pvh(struct md_page *pvh);
 static boolean_t pmap_is_referenced_pvh(struct md_page *pvh);
 static void pmap_kenter_attr(vm_offset_t va, vm_paddr_t pa, int mode);
@@ -1526,12 +1526,12 @@ pmap_add_delayed_free_list(vm_page_t m, 
  * for mapping a distinct range of virtual addresses.  The pmap's collection is
  * ordered by this virtual address range.
  */
-static __inline void
+static __inline int
 pmap_insert_pt_page(pmap_t pmap, vm_page_t mpte)
 {
 
PMAP_LOCK_ASSERT(pmap, MA_OWNED);
-   vm_radix_insert(&pmap->pm_root, mpte);
+   return (vm_radix_insert(&pmap->pm_root, mpte));
 }
 
 /*
@@ -3439,7 +3439,13 @@ setpte:
("pmap_promote_pde: page table page is out of range"));
KASSERT(mpte->pindex == pmap_pde_pindex(va),
("pmap_promote_pde: page table page's pindex is wrong"));
-   pmap_insert_pt_page(pmap, mpte);
+   if (pmap_insert_pt_page(pmap, mpte)) {
+   atomic_add_long(&pmap_pde_p_failures, 1);
+   CTR2(KTR_PMAP,
+   "pmap_promote_pde: failure for va %#lx in pmap %p", va,
+   pmap);
+   return;
+   }
 
/*
 * Promote the pv entries.

Modified: head/sys/dev/drm2/i915/i915_gem.c
==
--- head/sys/dev/drm2/i915/i915_gem.c   Fri Aug  9 11:26:26 2013
(r254140)
+++ head/sys/dev/drm2/i915/i915_gem.c   Fri Aug  9 11:28:55 2013
(r254141)
@@ -64,6 +64,9 @@ __FBSDID("$FreeBSD$");
 #include 
 #include 
 
+#include 
+#include 
+
 static void i915_gem_object_flush_cpu_write_domain(
 struct drm_i915_gem_object *obj);
 static uint32_t i915_gem_get_gtt_size(struct drm_device *dev, uint32_t size,
@@ -1443,8 +1446,14 @@ retry:
vm_page_busy_sleep(m, "915pbs");
goto retry;
}
+   if (vm_page_insert(m, vm_obj, OFF_TO_IDX(offset))) {
+   DRM_UNLOCK(dev);
+   VM_OBJECT_WUNLOCK(vm_obj);
+   VM_WAIT;
+   VM_OBJECT_WLOCK(vm_obj);
+   goto retry;
+   }
m->valid = VM_PAGE_BITS_ALL;
-   vm_page_insert(m, vm_obj, OFF_TO_IDX(offset));
 have_page:
*mres = m;
vm_page_xbusy(m);

Modified: head/sys/dev/drm2/ttm/ttm_bo_vm.c
==
--- head/sys/dev/drm2/ttm/ttm_bo_vm.c   Fri Aug  9 11:26:26 2013
(r254140)
+++ head/sys/dev/drm2/ttm/ttm_bo_vm.c   Fri Aug  9 11:28:55 2013
(r254141)
@@ -47,6 +47,7 @@ __FBSDID("$FreeBSD$");
 
 #include 
 #include 
+#include 
 
 #define TTM_BO_VM_NUM_PREFAULT 16
 
@@ -221,16 +222,23 @@ reserve:
ttm_bo_unreserve(bo);
goto retry;
}
-   m->valid = VM_PAGE_BITS_ALL;
-   *mres = m;
m1 = vm_page_lookup(vm_obj, OFF_TO_IDX(offset));
if (m1 == NULL) {
-  

svn commit: r254139 - in head: share/man/man9 sys/kern

2013-08-09 Thread Attilio Rao
Author: attilio
Date: Fri Aug  9 11:24:29 2013
New Revision: 254139
URL: http://svnweb.freebsd.org/changeset/base/254139

Log:
  Give mutex(9) the ability to recurse on a per-instance basis.
  Now the MTX_RECURSE flag can be passed to the mtx_*_flag() calls.
  This helps in cases we want to narrow down to specific calls the
  possibility to recurse for some locks.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  jeff, alc
  Tested by:pho

Modified:
  head/share/man/man9/mutex.9
  head/sys/kern/kern_mutex.c

Modified: head/share/man/man9/mutex.9
==
--- head/share/man/man9/mutex.9 Fri Aug  9 11:11:11 2013(r254138)
+++ head/share/man/man9/mutex.9 Fri Aug  9 11:24:29 2013(r254139)
@@ -225,8 +225,10 @@ or
 lock, respectively, and also accept a
 .Fa flags
 argument.
-In both cases, the only flag presently available for lock acquires is
-.Dv MTX_QUIET .
+In both cases, the only flags presently available for lock acquires are
+.Dv MTX_QUIET 
+and
+.Dv MTX_RECURSE .
 If the
 .Dv MTX_QUIET
 bit is turned on in the
@@ -235,6 +237,12 @@ argument, then if
 .Dv KTR_LOCK
 tracing is being done,
 it will be silenced during the lock acquire.
+If the
+.Dv MTX_RECURSE
+bit is turned on in the
+.Fa flags
+argument, then the mutex can be acquired recursively.
+.Pp
 .Pp
 The
 .Fn mtx_trylock

Modified: head/sys/kern/kern_mutex.c
==
--- head/sys/kern/kern_mutex.c  Fri Aug  9 11:11:11 2013(r254138)
+++ head/sys/kern/kern_mutex.c  Fri Aug  9 11:24:29 2013(r254139)
@@ -218,13 +218,14 @@ __mtx_lock_flags(volatile uintptr_t *c, 
KASSERT(LOCK_CLASS(&m->lock_object) == &lock_class_mtx_sleep,
("mtx_lock() of spin mutex %s @ %s:%d", m->lock_object.lo_name,
file, line));
-   WITNESS_CHECKORDER(&m->lock_object, opts | LOP_NEWORDER | LOP_EXCLUSIVE,
-   file, line, NULL);
+   WITNESS_CHECKORDER(&m->lock_object, (opts & ~MTX_RECURSE) |
+   LOP_NEWORDER | LOP_EXCLUSIVE, file, line, NULL);
 
__mtx_lock(m, curthread, opts, file, line);
LOCK_LOG_LOCK("LOCK", &m->lock_object, opts, m->mtx_recurse, file,
line);
-   WITNESS_LOCK(&m->lock_object, opts | LOP_EXCLUSIVE, file, line);
+   WITNESS_LOCK(&m->lock_object, (opts & ~MTX_RECURSE) | LOP_EXCLUSIVE,
+   file, line);
curthread->td_locks++;
 }
 
@@ -271,9 +272,11 @@ __mtx_lock_spin_flags(volatile uintptr_t
("mtx_lock_spin() of sleep mutex %s @ %s:%d",
m->lock_object.lo_name, file, line));
if (mtx_owned(m))
-   KASSERT((m->lock_object.lo_flags & LO_RECURSABLE) != 0,
+   KASSERT((m->lock_object.lo_flags & LO_RECURSABLE) != 0 ||
+   (opts & MTX_RECURSE) != 0,
("mtx_lock_spin: recursed on non-recursive mutex %s @ %s:%d\n",
m->lock_object.lo_name, file, line));
+   opts &= ~MTX_RECURSE;
WITNESS_CHECKORDER(&m->lock_object, opts | LOP_NEWORDER | LOP_EXCLUSIVE,
file, line, NULL);
__mtx_lock_spin(m, curthread, opts, file, line);
@@ -335,12 +338,14 @@ _mtx_trylock_flags_(volatile uintptr_t *
("mtx_trylock() of spin mutex %s @ %s:%d", m->lock_object.lo_name,
file, line));
 
-   if (mtx_owned(m) && (m->lock_object.lo_flags & LO_RECURSABLE) != 0) {
+   if (mtx_owned(m) && ((m->lock_object.lo_flags & LO_RECURSABLE) != 0 ||
+   (opts & MTX_RECURSE) != 0)) {
m->mtx_recurse++;
atomic_set_ptr(&m->mtx_lock, MTX_RECURSED);
rval = 1;
} else
rval = _mtx_obtain_lock(m, (uintptr_t)curthread);
+   opts &= ~MTX_RECURSE;
 
LOCK_LOG_TRY("LOCK", &m->lock_object, opts, rval, file, line);
if (rval) {
@@ -391,15 +396,18 @@ __mtx_lock_sleep(volatile uintptr_t *c, 
m = mtxlock2mtx(c);
 
if (mtx_owned(m)) {
-   KASSERT((m->lock_object.lo_flags & LO_RECURSABLE) != 0,
+   KASSERT((m->lock_object.lo_flags & LO_RECURSABLE) != 0 ||
+   (opts & MTX_RECURSE) != 0,
("_mtx_lock_sleep: recursed on non-recursive mutex %s @ %s:%d\n",
m->lock_object.lo_name, file, line));
+   opts &= ~MTX_RECURSE;
m->mtx_recurse++;
atomic_set_ptr(&m->mtx_lock, MTX_RECURSED);
if (LOCK_LOG_TEST(&m->lock_object, opts))
CTR1(KTR_LOCK, "_mtx_lock_sleep: %p recursing", m);
return;
}
+   opts &= ~MTX_RECURSE;
 
 #ifdef HWPMC_HOOKS
PMC_SOFT_CALL( , , lock, failed);
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r254138 - in head: share/man/man9 sys/amd64/amd64 sys/arm/arm sys/cddl/contrib/opensolaris/uts/common/fs/zfs sys/dev/agp sys/dev/drm2/i915 sys/dev/drm2/ttm sys/dev/md sys/fs/fuse sys/fs...

2013-08-09 Thread Attilio Rao
Author: attilio
Date: Fri Aug  9 11:11:11 2013
New Revision: 254138
URL: http://svnweb.freebsd.org/changeset/base/254138

Log:
  The soft and hard busy mechanism rely on the vm object lock to work.
  Unify the 2 concept into a real, minimal, sxlock where the shared
  acquisition represent the soft busy and the exclusive acquisition
  represent the hard busy.
  The old VPO_WANTED mechanism becames the hard-path for this new lock
  and it becomes per-page rather than per-object.
  The vm_object lock becames an interlock for this functionality:
  it can be held in both read or write mode.
  However, if the vm_object lock is held in read mode while acquiring
  or releasing the busy state, the thread owner cannot make any
  assumption on the busy state unless it is also busying it.
  
  Also:
  - Add a new flag to directly shared busy pages while vm_page_alloc
and vm_page_grab are being executed.  This will be very helpful
once these functions happen under a read object lock.
  - Move the swapping sleep into its own per-object flag
  
  The KPI is heavilly changed this is why the version is bumped.
  It is very likely that some VM ports users will need to change
  their own code.
  
  Sponsored by: EMC / Isilon storage division
  Discussed with:   alc
  Reviewed by:  jeff, kib
  Tested by:gavin, bapt (older version)
  Tested by:pho, scottl

Added:
  head/share/man/man9/vm_page_busy.9   (contents, props changed)
Deleted:
  head/share/man/man9/vm_page_io.9
  head/share/man/man9/vm_page_sleep_if_busy.9
  head/share/man/man9/vm_page_wakeup.9
Modified:
  head/share/man/man9/Makefile
  head/share/man/man9/VOP_GETPAGES.9
  head/share/man/man9/vm_page_alloc.9
  head/sys/amd64/amd64/pmap.c
  head/sys/arm/arm/pmap-v6.c
  head/sys/arm/arm/pmap.c
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c
  head/sys/dev/agp/agp.c
  head/sys/dev/drm2/i915/i915_gem.c
  head/sys/dev/drm2/ttm/ttm_bo_vm.c
  head/sys/dev/drm2/ttm/ttm_tt.c
  head/sys/dev/md/md.c
  head/sys/fs/fuse/fuse_vnops.c
  head/sys/fs/tmpfs/tmpfs_subr.c
  head/sys/fs/tmpfs/tmpfs_vnops.c
  head/sys/i386/i386/pmap.c
  head/sys/i386/xen/pmap.c
  head/sys/ia64/ia64/pmap.c
  head/sys/kern/kern_exec.c
  head/sys/kern/subr_uio.c
  head/sys/kern/uipc_shm.c
  head/sys/kern/uipc_syscalls.c
  head/sys/kern/vfs_bio.c
  head/sys/kern/vfs_cluster.c
  head/sys/mips/mips/pmap.c
  head/sys/powerpc/aim/mmu_oea.c
  head/sys/powerpc/aim/mmu_oea64.c
  head/sys/powerpc/booke/pmap.c
  head/sys/sparc64/sparc64/pmap.c
  head/sys/sys/param.h
  head/sys/vm/phys_pager.c
  head/sys/vm/swap_pager.c
  head/sys/vm/vm_fault.c
  head/sys/vm/vm_glue.c
  head/sys/vm/vm_object.c
  head/sys/vm/vm_page.c
  head/sys/vm/vm_page.h
  head/sys/vm/vm_pageout.c
  head/sys/vm/vm_phys.c
  head/sys/vm/vnode_pager.c

Modified: head/share/man/man9/Makefile
==
--- head/share/man/man9/MakefileFri Aug  9 09:48:05 2013
(r254137)
+++ head/share/man/man9/MakefileFri Aug  9 11:11:11 2013
(r254138)
@@ -332,11 +332,8 @@ MAN=   accept_filter.9 \
vm_page_grab.9 \
vm_page_hold.9 \
vm_page_insert.9 \
-   vm_page_io.9 \
vm_page_lookup.9 \
vm_page_rename.9 \
-   vm_page_sleep_if_busy.9 \
-   vm_page_wakeup.9 \
vm_page_wire.9 \
vm_set_page_size.9 \
vmem.9 \
@@ -1465,6 +1462,21 @@ MLINKS+=vm_page_bits.9 vm_page_clear_dir
vm_page_bits.9 vm_page_test_dirty.9 \
vm_page_bits.9 vm_page_undirty.9 \
vm_page_bits.9 vm_page_zero_invalid.9
+MLINKS+=vm_page_busy.9 vm_page_busied.9 \
+   vm_page_busy.9 vm_page_busy_downgrade.9 \
+   vm_page_busy.9 vm_page_busy_sleep.9 \
+   vm_page_busy.9 vm_page_sbusied.9 \
+   vm_page_busy.9 vm_page_sbusy.9 \
+   vm_page_busy.9 vm_page_sleep_if_busy.9 \
+   vm_page_busy.9 vm_page_sunbusy.9 \
+   vm_page_busy.9 vm_page_trysbusy.9 \
+   vm_page_busy.9 vm_page_tryxbusy.9 \
+   vm_page_busy.9 vm_page_xbusied.9 \
+   vm_page_busy.9 vm_page_xbusy.9 \
+   vm_page_busy.9 vm_page_xunbusy.9 \
+   vm_page_busy.9 vm_page_assert_sbusied.9 \
+   vm_page_busy.9 vm_page_assert_unbusied.9 \
+   vm_page_busy.9 vm_page_assert_xbusied.9
 MLINKS+=vm_page_aflag.9 vm_page_aflag_clear.9 \
vm_page_aflag.9 vm_page_aflag_set.9 \
vm_page_aflag.9 vm_page_reference.9
@@ -1473,10 +1485,6 @@ MLINKS+=vm_page_free.9 vm_page_free_toq.
vm_page_free.9 vm_page_try_to_free.9
 MLINKS+=vm_page_hold.9 vm_page_unhold.9
 MLINKS+=vm_page_insert.9 vm_page_remove.9
-MLINKS+=vm_page_io.9 vm_page_io_finish.9 \
-   vm_page_io.9 vm_page_io_start.9
-MLINKS+=vm_page_wakeup.9 vm_page_busy.9 \
-   vm_page_wakeup.9 vm_page_flash.9
 MLINKS+=vm_page_wire.9 vm_page_unwire.9
 MLINKS+=VOP_ACCESS.9 VOP_ACCESSX.9
 MLINKS+=VOP_ATTRIB.9 VOP_GETATTR.9 \

Modified: head/share/man/man9/VOP_GETPAGES.9
===

svn commit: r253953 - in head/sys: cddl/contrib/opensolaris/uts/common/fs/zfs fs/tmpfs kern vm

2013-08-05 Thread Attilio Rao
Author: attilio
Date: Mon Aug  5 08:55:35 2013
New Revision: 253953
URL: http://svnweb.freebsd.org/changeset/base/253953

Log:
  Revert r253939:
  We cannot busy a page before doing pagefaults.
  Infact, it can deadlock against vnode lock, as it tries to vget().
  Other functions, right now, have an opposite lock ordering, like
  vm_object_sync(), which acquires the vnode lock first and then
  sleeps on the busy mechanism.
  
  Before this patch is reinserted we need to break this ordering.
  
  Sponsored by: EMC / Isilon storage division
  Reported by:  kib

Modified:
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c
  head/sys/fs/tmpfs/tmpfs_vnops.c
  head/sys/kern/imgact_elf.c
  head/sys/kern/kern_exec.c
  head/sys/kern/sys_process.c
  head/sys/vm/vm_extern.h
  head/sys/vm/vm_fault.c
  head/sys/vm/vm_glue.c
  head/sys/vm/vm_map.h

Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c
==
--- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c Mon Aug 
 5 08:27:35 2013(r253952)
+++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c Mon Aug 
 5 08:55:35 2013(r253953)
@@ -324,8 +324,7 @@ zfs_ioctl(vnode_t *vp, u_long com, intpt
 }
 
 static vm_page_t
-page_busy(vnode_t *vp, int64_t start, int64_t off, int64_t nbytes,
-boolean_t alloc)
+page_busy(vnode_t *vp, int64_t start, int64_t off, int64_t nbytes)
 {
vm_object_t obj;
vm_page_t pp;
@@ -347,8 +346,6 @@ page_busy(vnode_t *vp, int64_t start, in
continue;
}
} else if (pp == NULL) {
-   if (!alloc)
-   break;
pp = vm_page_alloc(obj, OFF_TO_IDX(start),
VM_ALLOC_SYSTEM | VM_ALLOC_IFCACHED |
VM_ALLOC_NOBUSY);
@@ -359,10 +356,8 @@ page_busy(vnode_t *vp, int64_t start, in
 
if (pp != NULL) {
ASSERT3U(pp->valid, ==, VM_PAGE_BITS_ALL);
-   vm_page_io_start(pp);
-   if (!alloc)
-   break;
vm_object_pip_add(obj, 1);
+   vm_page_io_start(pp);
pmap_remove_write(pp);
vm_page_clear_dirty(pp, off, nbytes);
}
@@ -372,12 +367,55 @@ page_busy(vnode_t *vp, int64_t start, in
 }
 
 static void
-page_unbusy(vm_page_t pp, boolean_t unalloc)
+page_unbusy(vm_page_t pp)
 {
 
vm_page_io_finish(pp);
-   if (unalloc)
-   vm_object_pip_subtract(pp->object, 1);
+   vm_object_pip_subtract(pp->object, 1);
+}
+
+static vm_page_t
+page_hold(vnode_t *vp, int64_t start)
+{
+   vm_object_t obj;
+   vm_page_t pp;
+
+   obj = vp->v_object;
+   zfs_vmobject_assert_wlocked(obj);
+
+   for (;;) {
+   if ((pp = vm_page_lookup(obj, OFF_TO_IDX(start))) != NULL &&
+   pp->valid) {
+   if ((pp->oflags & VPO_BUSY) != 0) {
+   /*
+* Reference the page before unlocking and
+* sleeping so that the page daemon is less
+* likely to reclaim it.
+*/
+   vm_page_reference(pp);
+   vm_page_sleep(pp, "zfsmwb");
+   continue;
+   }
+
+   ASSERT3U(pp->valid, ==, VM_PAGE_BITS_ALL);
+   vm_page_lock(pp);
+   vm_page_hold(pp);
+   vm_page_unlock(pp);
+
+   } else
+   pp = NULL;
+   break;
+   }
+   return (pp);
+}
+
+static void
+page_unhold(vm_page_t pp)
+{
+
+   vm_page_lock(pp);
+   vm_page_unhold(pp);
+   vm_page_unlock(pp);
 }
 
 static caddr_t
@@ -441,8 +479,7 @@ update_pages(vnode_t *vp, int64_t start,
 
zfs_vmobject_wlock(obj);
vm_page_undirty(pp);
-   } else if ((pp = page_busy(vp, start, off, nbytes,
-   TRUE)) != NULL) {
+   } else if ((pp = page_busy(vp, start, off, nbytes)) != NULL) {
zfs_vmobject_wunlock(obj);
 
va = zfs_map_page(pp, &sf);
@@ -451,7 +488,7 @@ update_pages(vnode_t *vp, int64_t start,
zfs_unmap_page(sf);
 
zfs_vmobject_wlock(obj);
-   page_unbusy(pp, TRUE);
+   page_unbusy(pp);
}
len -= nbytes;
off = 0;
@@ -561,7 +598,7 @@ mappedread(vnode_t *vp, int nbytes, uio_
vm_page_t pp;
uint64_t bytes = MIN(PAGESIZE - off, len);

svn commit: r253940 - in head/sys/sparc64: include sparc64

2013-08-04 Thread Attilio Rao
Author: attilio
Date: Sun Aug  4 21:17:05 2013
New Revision: 253940
URL: http://svnweb.freebsd.org/changeset/base/253940

Log:
  Remove unused member.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  alc
  Tested by:pho

Modified:
  head/sys/sparc64/include/pmap.h
  head/sys/sparc64/sparc64/pmap.c

Modified: head/sys/sparc64/include/pmap.h
==
--- head/sys/sparc64/include/pmap.h Sun Aug  4 21:07:24 2013
(r253939)
+++ head/sys/sparc64/include/pmap.h Sun Aug  4 21:17:05 2013
(r253940)
@@ -56,7 +56,6 @@ struct md_page {
struct  pmap *pmap;
uint32_t colors[DCACHE_COLORS];
int32_t color;
-   uint32_t flags;
 };
 
 struct pmap {

Modified: head/sys/sparc64/sparc64/pmap.c
==
--- head/sys/sparc64/sparc64/pmap.c Sun Aug  4 21:07:24 2013
(r253939)
+++ head/sys/sparc64/sparc64/pmap.c Sun Aug  4 21:17:05 2013
(r253940)
@@ -765,7 +765,6 @@ pmap_page_init(vm_page_t m)
 
TAILQ_INIT(&m->md.tte_list);
m->md.color = DCACHE_COLOR(VM_PAGE_TO_PHYS(m));
-   m->md.flags = 0;
m->md.pmap = NULL;
 }
 
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r253939 - in head/sys: cddl/contrib/opensolaris/uts/common/fs/zfs fs/tmpfs kern vm

2013-08-04 Thread Attilio Rao
Author: attilio
Date: Sun Aug  4 21:07:24 2013
New Revision: 253939
URL: http://svnweb.freebsd.org/changeset/base/253939

Log:
  The page hold mechanism is fast but it has couple of fallouts:
  - It does not let pages respect the LRU policy
  - It bloats the active/inactive queues of few pages
  
  Try to avoid it as much as possible with the long-term target to
  completely remove it.
  Use the soft-busy mechanism to protect page content accesses during
  short-term operations (like uiomove_fromphys()).
  
  After this change only vm_fault_quick_hold_pages() is still using the
  hold mechanism for page content access.
  There is an additional complexity there as the quick path cannot
  immediately access the page object to busy the page and the slow path
  cannot however busy more than one page a time (to avoid deadlocks).
  
  Fixing such primitive can bring to complete removal of the page hold
  mechanism.
  
  Sponsored by: EMC / Isilon storage division
  Discussed with:   alc
  Reviewed by:  jeff
  Tested by:pho

Modified:
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c
  head/sys/fs/tmpfs/tmpfs_vnops.c
  head/sys/kern/imgact_elf.c
  head/sys/kern/kern_exec.c
  head/sys/kern/sys_process.c
  head/sys/vm/vm_extern.h
  head/sys/vm/vm_fault.c
  head/sys/vm/vm_glue.c
  head/sys/vm/vm_map.h

Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c
==
--- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c Sun Aug 
 4 21:00:22 2013(r253938)
+++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c Sun Aug 
 4 21:07:24 2013(r253939)
@@ -324,7 +324,8 @@ zfs_ioctl(vnode_t *vp, u_long com, intpt
 }
 
 static vm_page_t
-page_busy(vnode_t *vp, int64_t start, int64_t off, int64_t nbytes)
+page_busy(vnode_t *vp, int64_t start, int64_t off, int64_t nbytes,
+boolean_t alloc)
 {
vm_object_t obj;
vm_page_t pp;
@@ -346,6 +347,8 @@ page_busy(vnode_t *vp, int64_t start, in
continue;
}
} else if (pp == NULL) {
+   if (!alloc)
+   break;
pp = vm_page_alloc(obj, OFF_TO_IDX(start),
VM_ALLOC_SYSTEM | VM_ALLOC_IFCACHED |
VM_ALLOC_NOBUSY);
@@ -356,8 +359,10 @@ page_busy(vnode_t *vp, int64_t start, in
 
if (pp != NULL) {
ASSERT3U(pp->valid, ==, VM_PAGE_BITS_ALL);
-   vm_object_pip_add(obj, 1);
vm_page_io_start(pp);
+   if (!alloc)
+   break;
+   vm_object_pip_add(obj, 1);
pmap_remove_write(pp);
vm_page_clear_dirty(pp, off, nbytes);
}
@@ -367,55 +372,12 @@ page_busy(vnode_t *vp, int64_t start, in
 }
 
 static void
-page_unbusy(vm_page_t pp)
+page_unbusy(vm_page_t pp, boolean_t unalloc)
 {
 
vm_page_io_finish(pp);
-   vm_object_pip_subtract(pp->object, 1);
-}
-
-static vm_page_t
-page_hold(vnode_t *vp, int64_t start)
-{
-   vm_object_t obj;
-   vm_page_t pp;
-
-   obj = vp->v_object;
-   zfs_vmobject_assert_wlocked(obj);
-
-   for (;;) {
-   if ((pp = vm_page_lookup(obj, OFF_TO_IDX(start))) != NULL &&
-   pp->valid) {
-   if ((pp->oflags & VPO_BUSY) != 0) {
-   /*
-* Reference the page before unlocking and
-* sleeping so that the page daemon is less
-* likely to reclaim it.
-*/
-   vm_page_reference(pp);
-   vm_page_sleep(pp, "zfsmwb");
-   continue;
-   }
-
-   ASSERT3U(pp->valid, ==, VM_PAGE_BITS_ALL);
-   vm_page_lock(pp);
-   vm_page_hold(pp);
-   vm_page_unlock(pp);
-
-   } else
-   pp = NULL;
-   break;
-   }
-   return (pp);
-}
-
-static void
-page_unhold(vm_page_t pp)
-{
-
-   vm_page_lock(pp);
-   vm_page_unhold(pp);
-   vm_page_unlock(pp);
+   if (unalloc)
+   vm_object_pip_subtract(pp->object, 1);
 }
 
 static caddr_t
@@ -479,7 +441,8 @@ update_pages(vnode_t *vp, int64_t start,
 
zfs_vmobject_wlock(obj);
vm_page_undirty(pp);
-   } else if ((pp = page_busy(vp, start, off, nbytes)) != NULL) {
+   } else if ((pp = page_busy(vp, start, off, nbytes,
+   TRUE)) != NULL) {
zfs_vmobject_wunlock(obj);
 
va = zfs_map_p

svn commit: r253927 - in head/sys: fs/tmpfs kern

2013-08-04 Thread Attilio Rao
Author: attilio
Date: Sun Aug  4 15:56:19 2013
New Revision: 253927
URL: http://svnweb.freebsd.org/changeset/base/253927

Log:
  Remove unnecessary soft busy of the page before to do vn_rdwr() in
  kern_sendfile() which is unnecessary.
  The page is already wired so it will not be subjected to pagefault.
  The content cannot be effectively protected as it is full of races
  already.
  Multiple accesses to the same indexes are serialized through vn_rdwr().
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  alc, jeff
  Tested by:pho

Modified:
  head/sys/fs/tmpfs/tmpfs_vnops.c
  head/sys/kern/uipc_syscalls.c

Modified: head/sys/fs/tmpfs/tmpfs_vnops.c
==
--- head/sys/fs/tmpfs/tmpfs_vnops.c Sun Aug  4 11:38:08 2013
(r253926)
+++ head/sys/fs/tmpfs/tmpfs_vnops.c Sun Aug  4 15:56:19 2013
(r253927)
@@ -448,10 +448,8 @@ tmpfs_nocacheread(vm_object_t tobj, vm_p
VM_OBJECT_WLOCK(tobj);
 
/*
-* The kern_sendfile() code calls vn_rdwr() with the page
-* soft-busied.  Ignore the soft-busy state here. Parallel
-* reads of the page content from disk are prevented by
-* VPO_BUSY.
+* Parallel reads of the page content from disk are prevented
+* by VPO_BUSY.
 *
 * Although the tmpfs vnode lock is held here, it is
 * nonetheless safe to sleep waiting for a free page.  The
@@ -460,7 +458,7 @@ tmpfs_nocacheread(vm_object_t tobj, vm_p
 * type object.
 */
m = vm_page_grab(tobj, idx, VM_ALLOC_NORMAL | VM_ALLOC_RETRY |
-   VM_ALLOC_IGN_SBUSY | VM_ALLOC_NOBUSY);
+   VM_ALLOC_NOBUSY);
if (m->valid != VM_PAGE_BITS_ALL) {
vm_page_busy(m);
if (vm_pager_has_page(tobj, idx, NULL, NULL)) {

Modified: head/sys/kern/uipc_syscalls.c
==
--- head/sys/kern/uipc_syscalls.c   Sun Aug  4 11:38:08 2013
(r253926)
+++ head/sys/kern/uipc_syscalls.c   Sun Aug  4 15:56:19 2013
(r253927)
@@ -2246,11 +2246,6 @@ retry_space:
ssize_t resid;
int readahead = sfreadahead * MAXBSIZE;
 
-   /*
-* Ensure that our page is still around
-* when the I/O completes.
-*/
-   vm_page_io_start(pg);
VM_OBJECT_WUNLOCK(obj);
 
/*
@@ -2264,11 +2259,9 @@ retry_space:
trunc_page(off), UIO_NOCOPY, IO_NODELOCKED |
IO_VMIO | ((readahead / bsize) << 
IO_SEQSHIFT),
td->td_ucred, NOCRED, &resid, td);
-   VM_OBJECT_WLOCK(obj);
-   vm_page_io_finish(pg);
-   if (!error)
-   VM_OBJECT_WUNLOCK(obj);
SFSTAT_INC(sf_iocnt);
+   if (error)
+   VM_OBJECT_WLOCK(obj);
}
if (error) {
vm_page_lock(pg);
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: should_yield problem [Was: svn commit: r251322 - head/sys/kern]

2013-07-02 Thread Attilio Rao
On Tue, Jul 2, 2013 at 7:24 PM, Andriy Gapon  wrote:
> on 02/07/2013 20:12 Attilio Rao said the following:
>> While "ticks" is signed it is used as an unsigned int.
>> td_swvolticks is always derived by ticks, which again will always be
>> used as an unisgned and then the subtractions among the 2 will be
>> consistent.
>
> So returning to my example where ticks == -1946084020 and td_swvoltick == 0 
> and
> (ticks - td->td_swvoltick) == -1946084020.  Is this consistent in your 
> opinion?

I was just pointing out that the real bug is not in the subtraction
itself but on the hogticks comparison.
This is because hogticks is not an absolute measurement but it
represents really a ticks difference.
In my opinion we should define hogticks as an unsigned int and add a
small comment saying that it is a differential.
This will fix the issue.

Thanks,
Attilio


--
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: should_yield problem [Was: svn commit: r251322 - head/sys/kern]

2013-07-02 Thread Attilio Rao
On Tue, Jul 2, 2013 at 6:48 PM, Andriy Gapon  wrote:
> on 03/06/2013 20:36 Konstantin Belousov said the following:
>> Author: kib
>> Date: Mon Jun  3 17:36:43 2013
>> New Revision: 251322
>> URL: http://svnweb.freebsd.org/changeset/base/251322
>>
>> Log:
>>   Be more generous when donating the current thread time to the owner of
>>   the vnode lock while iterating over the free vnode list.  Instead of
>>   yielding, pause for 1 tick.  The change is reported to help in some
>>   virtualized environments.
>
> Kostik,
>
> I've just run into a problem where word "generous" does not seem to be
> applicable at all unless I am mistaken about how the code works.

While "ticks" is signed it is used as an unsigned int.
td_swvolticks is always derived by ticks, which again will always be
used as an unisgned and then the subtractions among the 2 will be
consistent.

Attilio


--
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r252356 - in head: contrib/smbfs/mount_smbfs etc/defaults etc/mtree include lib lib/libprocstat rescue/rescue sbin/mount share/examples share/examples/etc share/mk sys/conf sys/kern sy

2013-06-28 Thread Attilio Rao
On Fri, Jun 28, 2013 at 11:00 PM, Davide Italiano  wrote:
> Author: davide
> Date: Fri Jun 28 21:00:08 2013
> New Revision: 252356
> URL: http://svnweb.freebsd.org/changeset/base/252356
>
> Log:
>   - Trim an unused and bogus Makefile for mount_smbfs.
>   - Reconnect with some minor modifications, in particular now selsocket()
>   internals are adapted to use sbintime units after recent'ish calloutng
>   switch.

I've updated the page at:
https://wiki.freebsd.org/NONMPSAFE_DEORBIT_VFS

which reflects completition of the project and the re-insert of smbfs.
During VFS Non-MPSAFE deorbit we lost the following in-kernel support:
CodaFS, HPFS, NTFS, NWFS, PortalFS, XFS

even if for some of those (most notably NTFS) there is an userland
module for FUSE available.

Thanks,
Attilio


--
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r252209 - in head: share/man/man9 sys/kern sys/sys

2013-06-28 Thread Attilio Rao
On Thu, Jun 27, 2013 at 4:34 PM, John Baldwin  wrote:
> On Wednesday, June 26, 2013 3:26:38 am Andre Oppermann wrote:
>> On 25.06.2013 20:44, John Baldwin wrote:
>> > Author: jhb
>> > Date: Tue Jun 25 18:44:15 2013
>> > New Revision: 252209
>> > URL: http://svnweb.freebsd.org/changeset/base/252209
>> >
>> > Log:
>> >Several improvements to rmlock(9).  Many of these are based on patches
>> >provided by Isilon.
>> >- Add an rm_assert() supporting various lock assertions similar to other
>> >  locking primitives.  Because rmlocks track readers the assertions are
>> >  always fully accurate unlike rw_assert() and sx_assert().
>> >- Flesh out the lock class methods for rmlocks to support sleeping via
>> >  condvars and rm_sleep() (but only while holding write locks), rmlock
>> >  details in 'show lock' in DDB, and the lc_owner method used by
>> >  dtrace.
>> >- Add an internal destroyed cookie so that API functions can assert
>> >  that an rmlock is not destroyed.
>> >- Make use of rm_assert() to add various assertions to the API (e.g.
>> >  to assert locks are held when an unlock routine is called).
>> >- Give RM_SLEEPABLE locks their own lock class and always use the
>> >  rmlock's own lock_object with WITNESS.
>> >- Use THREAD_NO_SLEEPING() / THREAD_SLEEPING_OK() to disallow sleeping
>> >  while holding a read lock on an rmlock.
>>
>> Thanks!
>>
>> Would it make sense to move struct rm_queue from struct pcpu itself to
>> using DPCPU as a next step?
>
> Perhaps.  It might make pcpu.h cleaner, aside from that concern I don't think
> it really matters much.

It cannot for performance reasons. I had a comment ready for this but
I'm not sure if it was ever committed.

Attilio


--
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r251586 - head/sys/arm/ti

2013-06-11 Thread Attilio Rao
On Tue, Jun 11, 2013 at 7:21 AM, Konstantin Belousov
 wrote:
> On Tue, Jun 11, 2013 at 01:10:52AM +0200, Olivier Houchard wrote:
>> On Mon, Jun 10, 2013 at 11:13:58PM +0200, Olivier Houchard wrote:
>> > On Mon, Jun 10, 2013 at 10:37:36PM +0300, Konstantin Belousov wrote:
>> > > On Mon, Jun 10, 2013 at 12:02:20PM -0500, Alan Cox wrote:
>> > > > On 06/10/2013 06:08, Olivier Houchard wrote:
>> > > > > On Mon, Jun 10, 2013 at 06:55:47AM +0300, Konstantin Belousov wrote:
>> > > > >> On Sun, Jun 09, 2013 at 10:51:12PM +, Olivier Houchard wrote:
>> > > > >>> Author: cognet
>> > > > >>> Date: Sun Jun  9 22:51:11 2013
>> > > > >>> New Revision: 251586
>> > > > >>> URL: http://svnweb.freebsd.org/changeset/base/251586
>> > > > >>>
>> > > > >>> Log:
>> > > > >>>   Increase the maximum KVM available on TI chips. Not sure why we 
>> > > > >>> suddenly need
>> > > > >>>   that much, but that lets me boot with 1GB of RAM.
>> > > > >> I suspect that the cause is the combination of limited KVA and
>> > > > >> lack of any limitation for the buffer map. I noted that ARM lacks
>> > > > >> VM_BCACHE_SIZE_MAX after a report from mav about similar (?) 
>> > > > >> problem a
>> > > > >> day ago.
>> > > > >>
>> > > > >> In essence, the buffer map is allowed to take up to ~330MB when no
>> > > > >> upper limit from VM_BCACHE_SIZE_MAX is specified.
>> > > > >
>> > > > > Hi Konstantin,
>> > > > >
>> > > > > Thanks for the hint !
>> > > > > It seems only i386 and sparc64 sets it, what would be a good value, 
>> > > > > 200M, as
>> > > > > it is on i386 ?
>> > > > >
>> > > >
>> > > > Since there are many arm platforms with less than 1 GB of kernel 
>> > > > virtual
>> > > > address (KVA) space, VM_BCACHE_SIZE_MAX should be made to scale down
>> > > > from 200 MB with the available KVA space.  See how VM_KMEM_SIZE_MAX is
>> > > > currently defined on arm.
>> > >
>> > > In fact, Ithink it does not make much sense to scale the buffer cache up.
>> > > It is mostly wasted space now.  As I measured it, on typical load you
>> > > have only 10-20% of instantiated buffers mapped.
>> > >
>> > > Alexander Motin reported that he tested the equivalent of the following
>> > > change.  With it committed, I think that r251586 could be reverted.
>> > >
>> > > diff --git a/sys/arm/include/param.h b/sys/arm/include/param.h
>> > > index 9ffb118..5c738c2 100644
>> > > --- a/sys/arm/include/param.h
>> > > +++ b/sys/arm/include/param.h
>> > > @@ -128,6 +128,11 @@
>> > >  #define USPACE_SVC_STACK_BOTTOM  (USPACE_SVC_STACK_TOP - 0x1000)
>> > >  #define USPACE_UNDEF_STACK_TOP   (USPACE_SVC_STACK_BOTTOM - 
>> > > 0x10)
>> > >  #define USPACE_UNDEF_STACK_BOTTOM(FPCONTEXTSIZE + 10)
>> > > +
>> > > +#ifndef VM_BCACHE_SIZE_MAX
>> > > +#define  VM_BCACHE_SIZE_MAX  (128 * 1024 * 1024)
>> > > +#endif
>> > > +
>> > >  /*
>> > >   * Mach derived conversion macros
>> > >   */
>> >
>> >
>> > I tested it with my changes reverted and it works indeed, so I'm fine with
>> > this being committed and my changes being reverted.
>> >
>>
>> In fact I spoke too soon. It's getting further, but I'm ending up getting
>> vm_thread_new: kstack allocation failed
>> Probably because I have a local patch that aligns the stack on 32kB, which
>> is something we have to do if we want to store curthread on the kstack.
>> It will boot if I reduce VM_DCACHE_SIZE_MAX to 64MB, but it's probably not
>> the best thing to do.
>
> The other cause of increased KVA use is the vm radix trie used to keep
> the collection of the vm object' pages. When I profiled KVA use for PAE
> on i386, which has similar problem of exhausted KVA, the radix trie
> popped up as the reason.

BTW, it would be interesting to see data from your analysis. When
radix trie was developed I get pho@ to test PAE too and no
exhaustation was reported.

Also, on a related note, I wonder how relevant is PAE support on i386
nowadays. Maybe we should consider axing it.

Attilio


--
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r251586 - head/sys/arm/ti

2013-06-11 Thread Attilio Rao
On Tue, Jun 11, 2013 at 7:21 AM, Konstantin Belousov
 wrote:
> On Tue, Jun 11, 2013 at 01:10:52AM +0200, Olivier Houchard wrote:
>> On Mon, Jun 10, 2013 at 11:13:58PM +0200, Olivier Houchard wrote:
>> > On Mon, Jun 10, 2013 at 10:37:36PM +0300, Konstantin Belousov wrote:
>> > > On Mon, Jun 10, 2013 at 12:02:20PM -0500, Alan Cox wrote:
>> > > > On 06/10/2013 06:08, Olivier Houchard wrote:
>> > > > > On Mon, Jun 10, 2013 at 06:55:47AM +0300, Konstantin Belousov wrote:
>> > > > >> On Sun, Jun 09, 2013 at 10:51:12PM +, Olivier Houchard wrote:
>> > > > >>> Author: cognet
>> > > > >>> Date: Sun Jun  9 22:51:11 2013
>> > > > >>> New Revision: 251586
>> > > > >>> URL: http://svnweb.freebsd.org/changeset/base/251586
>> > > > >>>
>> > > > >>> Log:
>> > > > >>>   Increase the maximum KVM available on TI chips. Not sure why we 
>> > > > >>> suddenly need
>> > > > >>>   that much, but that lets me boot with 1GB of RAM.
>> > > > >> I suspect that the cause is the combination of limited KVA and
>> > > > >> lack of any limitation for the buffer map. I noted that ARM lacks
>> > > > >> VM_BCACHE_SIZE_MAX after a report from mav about similar (?) 
>> > > > >> problem a
>> > > > >> day ago.
>> > > > >>
>> > > > >> In essence, the buffer map is allowed to take up to ~330MB when no
>> > > > >> upper limit from VM_BCACHE_SIZE_MAX is specified.
>> > > > >
>> > > > > Hi Konstantin,
>> > > > >
>> > > > > Thanks for the hint !
>> > > > > It seems only i386 and sparc64 sets it, what would be a good value, 
>> > > > > 200M, as
>> > > > > it is on i386 ?
>> > > > >
>> > > >
>> > > > Since there are many arm platforms with less than 1 GB of kernel 
>> > > > virtual
>> > > > address (KVA) space, VM_BCACHE_SIZE_MAX should be made to scale down
>> > > > from 200 MB with the available KVA space.  See how VM_KMEM_SIZE_MAX is
>> > > > currently defined on arm.
>> > >
>> > > In fact, Ithink it does not make much sense to scale the buffer cache up.
>> > > It is mostly wasted space now.  As I measured it, on typical load you
>> > > have only 10-20% of instantiated buffers mapped.
>> > >
>> > > Alexander Motin reported that he tested the equivalent of the following
>> > > change.  With it committed, I think that r251586 could be reverted.
>> > >
>> > > diff --git a/sys/arm/include/param.h b/sys/arm/include/param.h
>> > > index 9ffb118..5c738c2 100644
>> > > --- a/sys/arm/include/param.h
>> > > +++ b/sys/arm/include/param.h
>> > > @@ -128,6 +128,11 @@
>> > >  #define USPACE_SVC_STACK_BOTTOM  (USPACE_SVC_STACK_TOP - 0x1000)
>> > >  #define USPACE_UNDEF_STACK_TOP   (USPACE_SVC_STACK_BOTTOM - 
>> > > 0x10)
>> > >  #define USPACE_UNDEF_STACK_BOTTOM(FPCONTEXTSIZE + 10)
>> > > +
>> > > +#ifndef VM_BCACHE_SIZE_MAX
>> > > +#define  VM_BCACHE_SIZE_MAX  (128 * 1024 * 1024)
>> > > +#endif
>> > > +
>> > >  /*
>> > >   * Mach derived conversion macros
>> > >   */
>> >
>> >
>> > I tested it with my changes reverted and it works indeed, so I'm fine with
>> > this being committed and my changes being reverted.
>> >
>>
>> In fact I spoke too soon. It's getting further, but I'm ending up getting
>> vm_thread_new: kstack allocation failed
>> Probably because I have a local patch that aligns the stack on 32kB, which
>> is something we have to do if we want to store curthread on the kstack.
>> It will boot if I reduce VM_DCACHE_SIZE_MAX to 64MB, but it's probably not
>> the best thing to do.
>
> The other cause of increased KVA use is the vm radix trie used to keep
> the collection of the vm object' pages. When I profiled KVA use for PAE
> on i386, which has similar problem of exhausted KVA, the radix trie
> popped up as the reason.
>
> IMO the current sizing of the trie for the worst case is not attainable
> for any practical situation.  Anyway, this is separate.

I discussed with Jeff a way to not depend by the preallocated pool of
nodes and I think we can fix it. However I still have to finish some
things before to get into this but I will try to commit to this before
code slush.

Attilio


--
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r251471 - head/sys/vm

2013-06-06 Thread Attilio Rao
Author: attilio
Date: Thu Jun  6 18:19:26 2013
New Revision: 251471
URL: http://svnweb.freebsd.org/changeset/base/251471

Log:
  Complete r251452:
  Avoid to busy/unbusy a page in cases where there is no need to drop the
  vm_obj lock, more nominally when the page is full valid after
  vm_page_grab().
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  alc

Modified:
  head/sys/vm/swap_pager.c
  head/sys/vm/vm_glue.c

Modified: head/sys/vm/swap_pager.c
==
--- head/sys/vm/swap_pager.cThu Jun  6 14:43:19 2013(r251470)
+++ head/sys/vm/swap_pager.cThu Jun  6 18:19:26 2013(r251471)
@@ -1706,18 +1706,19 @@ swp_pager_force_pagein(vm_object_t objec
vm_page_t m;
 
vm_object_pip_add(object, 1);
-   m = vm_page_grab(object, pindex, VM_ALLOC_NORMAL|VM_ALLOC_RETRY);
+   m = vm_page_grab(object, pindex, VM_ALLOC_NORMAL | VM_ALLOC_RETRY |
+   VM_ALLOC_NOBUSY);
if (m->valid == VM_PAGE_BITS_ALL) {
vm_object_pip_subtract(object, 1);
vm_page_dirty(m);
vm_page_lock(m);
vm_page_activate(m);
vm_page_unlock(m);
-   vm_page_wakeup(m);
vm_pager_page_unswapped(m);
return;
}
 
+   vm_page_busy(m);
if (swap_pager_getpages(object, &m, 1, 0) != VM_PAGER_OK)
panic("swap_pager_force_pagein: read from swap failed");/*XXX*/
vm_object_pip_subtract(object, 1);

Modified: head/sys/vm/vm_glue.c
==
--- head/sys/vm/vm_glue.c   Thu Jun  6 14:43:19 2013(r251470)
+++ head/sys/vm/vm_glue.c   Thu Jun  6 18:19:26 2013(r251471)
@@ -241,8 +241,10 @@ vm_imgact_hold_page(vm_object_t object, 
 
VM_OBJECT_WLOCK(object);
pindex = OFF_TO_IDX(offset);
-   m = vm_page_grab(object, pindex, VM_ALLOC_NORMAL | VM_ALLOC_RETRY);
+   m = vm_page_grab(object, pindex, VM_ALLOC_NORMAL | VM_ALLOC_RETRY |
+   VM_ALLOC_NOBUSY);
if (m->valid != VM_PAGE_BITS_ALL) {
+   vm_page_busy(m);
ma[0] = m;
rv = vm_pager_get_pages(object, ma, 1, 0);
m = vm_page_lookup(object, pindex);
@@ -255,11 +257,11 @@ vm_imgact_hold_page(vm_object_t object, 
m = NULL;
goto out;
}
+   vm_page_wakeup(m);
}
vm_page_lock(m);
vm_page_hold(m);
vm_page_unlock(m);
-   vm_page_wakeup(m);
 out:
VM_OBJECT_WUNLOCK(object);
return (m);
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r251397 - head/sys/vm

2013-06-04 Thread Attilio Rao
Author: attilio
Date: Tue Jun  4 22:47:01 2013
New Revision: 251397
URL: http://svnweb.freebsd.org/changeset/base/251397

Log:
  In vm_object_split(), busy and consequently unbusy the pages only when
  swap_pager_copy() is invoked, otherwise there is no reason to do so.
  This will eliminate the necessity to busy pages most of the times.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  alc

Modified:
  head/sys/vm/vm_object.c

Modified: head/sys/vm/vm_object.c
==
--- head/sys/vm/vm_object.c Tue Jun  4 22:32:33 2013(r251396)
+++ head/sys/vm/vm_object.c Tue Jun  4 22:47:01 2013(r251397)
@@ -1390,7 +1390,8 @@ retry:
vm_page_rename(m, new_object, idx);
vm_page_unlock(m);
/* page automatically made dirty by rename and cache handled */
-   vm_page_busy(m);
+   if (orig_object->type == OBJT_SWAP)
+   vm_page_busy(m);
}
if (orig_object->type == OBJT_SWAP) {
/*
@@ -1398,6 +1399,8 @@ retry:
 * and new_object's locks are released and reacquired. 
 */
swap_pager_copy(orig_object, new_object, offidxstart, 0);
+   TAILQ_FOREACH(m, &new_object->memq, listq)
+   vm_page_wakeup(m);
 
/*
 * Transfer any cached pages from orig_object to new_object.
@@ -1413,8 +1416,6 @@ retry:
new_object);
}
VM_OBJECT_WUNLOCK(orig_object);
-   TAILQ_FOREACH(m, &new_object->memq, listq)
-   vm_page_wakeup(m);
VM_OBJECT_WUNLOCK(new_object);
entry->object.vm_object = new_object;
entry->offset = 0LL;
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r251077 - head/sys/vm

2013-05-28 Thread Attilio Rao
Author: attilio
Date: Tue May 28 22:07:23 2013
New Revision: 251077
URL: http://svnweb.freebsd.org/changeset/base/251077

Log:
  o Change the locking scheme for swp_bcount.
It can now be accessed with a write lock on the object containing it OR
with a read lock on the object containing it along with the swhash_mtx.
  o Remove some duplicate assertions for swap_pager_freespace() and
swap_pager_unswapped() but keep the object locking references for
documentation.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  alc

Modified:
  head/sys/vm/swap_pager.c

Modified: head/sys/vm/swap_pager.c
==
--- head/sys/vm/swap_pager.cTue May 28 22:00:37 2013(r251076)
+++ head/sys/vm/swap_pager.cTue May 28 22:07:23 2013(r251077)
@@ -822,12 +822,13 @@ swp_pager_freeswapspace(daddr_t blk, int
  * The external callers of this routine typically have already destroyed
  * or renamed vm_page_t's associated with this range in the object so
  * we should be ok.
+ *
+ * The object must be locked.
  */
 void
 swap_pager_freespace(vm_object_t object, vm_pindex_t start, vm_size_t size)
 {
 
-   VM_OBJECT_ASSERT_WLOCKED(object);
swp_pager_meta_free(object, start, size);
 }
 
@@ -999,7 +1000,7 @@ swap_pager_haspage(vm_object_t object, v
 {
daddr_t blk0;
 
-   VM_OBJECT_ASSERT_WLOCKED(object);
+   VM_OBJECT_ASSERT_LOCKED(object);
/*
 * do we have good backing store at the requested index ?
 */
@@ -1065,12 +1066,13 @@ swap_pager_haspage(vm_object_t object, v
  * depends on it.
  *
  * This routine may not sleep.
+ *
+ * The object containing the page must be locked.
  */
 static void
 swap_pager_unswapped(vm_page_t m)
 {
 
-   VM_OBJECT_ASSERT_WLOCKED(m->object);
swp_pager_meta_ctl(m->object, m->pindex, SWM_FREE);
 }
 
@@ -1916,7 +1918,7 @@ static void
 swp_pager_meta_free(vm_object_t object, vm_pindex_t index, daddr_t count)
 {
 
-   VM_OBJECT_ASSERT_WLOCKED(object);
+   VM_OBJECT_ASSERT_LOCKED(object);
if (object->type != OBJT_SWAP)
return;
 
@@ -2021,7 +2023,7 @@ swp_pager_meta_ctl(vm_object_t object, v
daddr_t r1;
int idx;
 
-   VM_OBJECT_ASSERT_WLOCKED(object);
+   VM_OBJECT_ASSERT_LOCKED(object);
/*
 * The meta data only exists of the object is OBJT_SWAP
 * and even then might not be allocated yet.
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r250909 - head/sys/vm

2013-05-22 Thread Attilio Rao
Author: attilio
Date: Wed May 22 15:11:00 2013
New Revision: 250909
URL: http://svnweb.freebsd.org/changeset/base/250909

Log:
  Acquire read lock on the src object for vm_fault_copy_entry().
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  alc

Modified:
  head/sys/vm/vm_fault.c

Modified: head/sys/vm/vm_fault.c
==
--- head/sys/vm/vm_fault.c  Wed May 22 12:59:39 2013(r250908)
+++ head/sys/vm/vm_fault.c  Wed May 22 15:11:00 2013(r250909)
@@ -1318,7 +1318,7 @@ vm_fault_copy_entry(vm_map_t dst_map, vm
 * (Because the source is wired down, the page will be in
 * memory.)
 */
-   VM_OBJECT_WLOCK(src_object);
+   VM_OBJECT_RLOCK(src_object);
object = src_object;
pindex = src_pindex + dst_pindex;
while ((src_m = vm_page_lookup(object, pindex)) == NULL &&
@@ -1327,15 +1327,15 @@ vm_fault_copy_entry(vm_map_t dst_map, vm
/*
 * Allow fallback to backing objects if we are reading.
 */
-   VM_OBJECT_WLOCK(backing_object);
+   VM_OBJECT_RLOCK(backing_object);
pindex += OFF_TO_IDX(object->backing_object_offset);
-   VM_OBJECT_WUNLOCK(object);
+   VM_OBJECT_RUNLOCK(object);
object = backing_object;
}
if (src_m == NULL)
panic("vm_fault_copy_wired: page missing");
pmap_copy_page(src_m, dst_m);
-   VM_OBJECT_WUNLOCK(object);
+   VM_OBJECT_RUNLOCK(object);
dst_m->valid = VM_PAGE_BITS_ALL;
dst_m->dirty = VM_PAGE_BITS_ALL;
VM_OBJECT_WUNLOCK(dst_object);
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r250885 - head/sys/kern

2013-05-21 Thread Attilio Rao
Author: attilio
Date: Tue May 21 20:54:03 2013
New Revision: 250885
URL: http://svnweb.freebsd.org/changeset/base/250885

Log:
  vm_object locking is not needed there as pages are already wired.
  
  Sponsored by: EMC / Isilon storage division
  Submitted by: alc

Modified:
  head/sys/kern/vfs_bio.c

Modified: head/sys/kern/vfs_bio.c
==
--- head/sys/kern/vfs_bio.c Tue May 21 20:38:19 2013(r250884)
+++ head/sys/kern/vfs_bio.c Tue May 21 20:54:03 2013(r250885)
@@ -4211,7 +4211,6 @@ vfs_bio_bzero_buf(struct buf *bp, int ba
} else {
BUF_CHECK_UNMAPPED(bp);
n = PAGE_SIZE - (base & PAGE_MASK);
-   VM_OBJECT_WLOCK(bp->b_bufobj->bo_object);
for (i = base / PAGE_SIZE; size > 0 && i < bp->b_npages; ++i) {
m = bp->b_pages[i];
if (n > size)
@@ -4221,7 +4220,6 @@ vfs_bio_bzero_buf(struct buf *bp, int ba
size -= n;
n = PAGE_SIZE;
}
-   VM_OBJECT_WUNLOCK(bp->b_bufobj->bo_object);
}
 }
 
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r250884 - in head/sys: amd64/amd64 arm/arm i386/i386 i386/xen ia64/ia64 mips/mips powerpc/aim powerpc/booke sparc64/sparc64 vm

2013-05-21 Thread Attilio Rao
Author: attilio
Date: Tue May 21 20:38:19 2013
New Revision: 250884
URL: http://svnweb.freebsd.org/changeset/base/250884

Log:
  o Relax locking assertions for vm_page_find_least()
  o Relax locking assertions for pmap_enter_object() and add them also
to architectures that currently don't have any
  o Introduce VM_OBJECT_LOCK_DOWNGRADE() which is basically a downgrade
operation on the per-object rwlock
  o Use all the mechanisms above to make vm_map_pmap_enter() to work
mostl of the times only with readlocks.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  alc

Modified:
  head/sys/amd64/amd64/pmap.c
  head/sys/arm/arm/pmap-v6.c
  head/sys/arm/arm/pmap.c
  head/sys/i386/i386/pmap.c
  head/sys/i386/xen/pmap.c
  head/sys/ia64/ia64/pmap.c
  head/sys/mips/mips/pmap.c
  head/sys/powerpc/aim/mmu_oea.c
  head/sys/powerpc/aim/mmu_oea64.c
  head/sys/powerpc/booke/pmap.c
  head/sys/sparc64/sparc64/pmap.c
  head/sys/vm/vm_map.c
  head/sys/vm/vm_object.h
  head/sys/vm/vm_page.c

Modified: head/sys/amd64/amd64/pmap.c
==
--- head/sys/amd64/amd64/pmap.c Tue May 21 19:59:37 2013(r250883)
+++ head/sys/amd64/amd64/pmap.c Tue May 21 20:38:19 2013(r250884)
@@ -3722,7 +3722,8 @@ pmap_enter_object(pmap_t pmap, vm_offset
vm_page_t m, mpte;
vm_pindex_t diff, psize;
 
-   VM_OBJECT_ASSERT_WLOCKED(m_start->object);
+   VM_OBJECT_ASSERT_LOCKED(m_start->object);
+
psize = atop(end - start);
mpte = NULL;
m = m_start;

Modified: head/sys/arm/arm/pmap-v6.c
==
--- head/sys/arm/arm/pmap-v6.c  Tue May 21 19:59:37 2013(r250883)
+++ head/sys/arm/arm/pmap-v6.c  Tue May 21 20:38:19 2013(r250884)
@@ -2931,6 +2931,8 @@ pmap_enter_object(pmap_t pmap, vm_offset
vm_page_t m;
vm_pindex_t diff, psize;
 
+   VM_OBJECT_ASSERT_LOCKED(m_start->object);
+
psize = atop(end - start);
m = m_start;
rw_wlock(&pvh_global_lock);

Modified: head/sys/arm/arm/pmap.c
==
--- head/sys/arm/arm/pmap.c Tue May 21 19:59:37 2013(r250883)
+++ head/sys/arm/arm/pmap.c Tue May 21 20:38:19 2013(r250884)
@@ -3587,6 +3587,8 @@ pmap_enter_object(pmap_t pmap, vm_offset
vm_page_t m;
vm_pindex_t diff, psize;
 
+   VM_OBJECT_ASSERT_LOCKED(m_start->object);
+
psize = atop(end - start);
m = m_start;
rw_wlock(&pvh_global_lock);

Modified: head/sys/i386/i386/pmap.c
==
--- head/sys/i386/i386/pmap.c   Tue May 21 19:59:37 2013(r250883)
+++ head/sys/i386/i386/pmap.c   Tue May 21 20:38:19 2013(r250884)
@@ -3677,7 +3677,8 @@ pmap_enter_object(pmap_t pmap, vm_offset
vm_page_t m, mpte;
vm_pindex_t diff, psize;
 
-   VM_OBJECT_ASSERT_WLOCKED(m_start->object);
+   VM_OBJECT_ASSERT_LOCKED(m_start->object);
+
psize = atop(end - start);
mpte = NULL;
m = m_start;

Modified: head/sys/i386/xen/pmap.c
==
--- head/sys/i386/xen/pmap.cTue May 21 19:59:37 2013(r250883)
+++ head/sys/i386/xen/pmap.cTue May 21 20:38:19 2013(r250884)
@@ -2871,7 +2871,8 @@ pmap_enter_object(pmap_t pmap, vm_offset
multicall_entry_t *mclp = mcl;
int error, count = 0;
 
-   VM_OBJECT_ASSERT_WLOCKED(m_start->object);
+   VM_OBJECT_ASSERT_LOCKED(m_start->object);
+
psize = atop(end - start);
mpte = NULL;
m = m_start;

Modified: head/sys/ia64/ia64/pmap.c
==
--- head/sys/ia64/ia64/pmap.c   Tue May 21 19:59:37 2013(r250883)
+++ head/sys/ia64/ia64/pmap.c   Tue May 21 20:38:19 2013(r250884)
@@ -1802,7 +1802,8 @@ pmap_enter_object(pmap_t pmap, vm_offset
vm_page_t m;
vm_pindex_t diff, psize;
 
-   VM_OBJECT_ASSERT_WLOCKED(m_start->object);
+   VM_OBJECT_ASSERT_LOCKED(m_start->object);
+
psize = atop(end - start);
m = m_start;
rw_wlock(&pvh_global_lock);

Modified: head/sys/mips/mips/pmap.c
==
--- head/sys/mips/mips/pmap.c   Tue May 21 19:59:37 2013(r250883)
+++ head/sys/mips/mips/pmap.c   Tue May 21 20:38:19 2013(r250884)
@@ -2399,7 +2399,8 @@ pmap_enter_object(pmap_t pmap, vm_offset
vm_page_t m, mpte;
vm_pindex_t diff, psize;
 
-   VM_OBJECT_ASSERT_WLOCKED(m_start->object);
+   VM_OBJECT_ASSERT_LOCKED(m_start->object);
+
psize = atop(end - start);
mpte = NULL;
m = m_start;

Modified: head/sys/powerpc/aim/mmu_oea.c
==

svn commit: r250751 - head/sys/kern

2013-05-17 Thread Attilio Rao
Author: attilio
Date: Fri May 17 20:03:55 2013
New Revision: 250751
URL: http://svnweb.freebsd.org/changeset/base/250751

Log:
  Use readlocking now that assertions on vm_page_lookup() are relaxed.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  alc
  Tested by:flo, pho

Modified:
  head/sys/kern/vfs_bio.c

Modified: head/sys/kern/vfs_bio.c
==
--- head/sys/kern/vfs_bio.c Fri May 17 19:37:16 2013(r250750)
+++ head/sys/kern/vfs_bio.c Fri May 17 20:03:55 2013(r250751)
@@ -2772,7 +2772,7 @@ inmem(struct vnode * vp, daddr_t blkno)
size = vp->v_mount->mnt_stat.f_iosize;
off = (vm_ooffset_t)blkno * 
(vm_ooffset_t)vp->v_mount->mnt_stat.f_iosize;
 
-   VM_OBJECT_WLOCK(obj);
+   VM_OBJECT_RLOCK(obj);
for (toff = 0; toff < vp->v_mount->mnt_stat.f_iosize; toff += tinc) {
m = vm_page_lookup(obj, OFF_TO_IDX(off + toff));
if (!m)
@@ -2784,11 +2784,11 @@ inmem(struct vnode * vp, daddr_t blkno)
(vm_offset_t) ((toff + off) & PAGE_MASK), tinc) == 0)
goto notinmem;
}
-   VM_OBJECT_WUNLOCK(obj);
+   VM_OBJECT_RUNLOCK(obj);
return 1;
 
 notinmem:
-   VM_OBJECT_WUNLOCK(obj);
+   VM_OBJECT_RUNLOCK(obj);
return (0);
 }
 
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r250601 - in head/sys: sys vm x86/acpica

2013-05-13 Thread Attilio Rao
Author: attilio
Date: Mon May 13 15:40:51 2013
New Revision: 250601
URL: http://svnweb.freebsd.org/changeset/base/250601

Log:
  o Add accessor functions to add and remove pages from a specific
freelist.
  o Split the pool of free pages queues really by domain and not rely on
definition of VM_RAW_NFREELIST.
  o For MAXMEMDOM > 1, wrap the RR allocation logic into a specific
function that is called when calculating the allocation domain.
The RR counter is kept, currently, per-thread.
In the future it is expected that such function evolves in a real
policy decision referee, based on specific informations retrieved by
per-thread and per-vm_object attributes.
  o Add the concept of "probed domains" under the form of vm_ndomains.
It is responsibility for every architecture willing to support multiple
memory domains to correctly probe vm_ndomains along with mem_affinity
segments attributes.  Those two values are supposed to remain always
consistent.
Please also note that vm_ndomains and td_dom_rr_idx are both int
because segments already store domains as int.  Ideally u_int would
have much more sense. Probabilly this should be cleaned up in the
future.
  o Apply RR domain selection also to vm_phys_zero_pages_idle().
  
  Sponsored by: EMC / Isilon storage division
  Partly obtained from: jeff
  Reviewed by:  alc
  Tested by:jeff

Modified:
  head/sys/sys/proc.h
  head/sys/vm/vm_phys.c
  head/sys/vm/vm_phys.h
  head/sys/x86/acpica/srat.c

Modified: head/sys/sys/proc.h
==
--- head/sys/sys/proc.h Mon May 13 15:18:36 2013(r250600)
+++ head/sys/sys/proc.h Mon May 13 15:40:51 2013(r250601)
@@ -274,6 +274,7 @@ struct thread {
pid_t   td_dbg_forked;  /* (c) Child pid for debugger. */
u_int   td_vp_reserv;   /* (k) Count of reserved vnodes. */
int td_no_sleeping; /* (k) Sleeping disabled count. */
+   int td_dom_rr_idx;  /* (k) RR Numa domain selection. */
 #definetd_endzero td_sigmask
 
 /* Copied during fork1() or create_thread(). */

Modified: head/sys/vm/vm_phys.c
==
--- head/sys/vm/vm_phys.c   Mon May 13 15:18:36 2013(r250600)
+++ head/sys/vm/vm_phys.c   Mon May 13 15:40:51 2013(r250601)
@@ -48,6 +48,9 @@ __FBSDID("$FreeBSD$");
 #include 
 #include 
 #include 
+#if MAXMEMDOM > 1
+#include 
+#endif
 #include 
 #include 
 #include 
@@ -62,13 +65,6 @@ __FBSDID("$FreeBSD$");
 #include 
 #include 
 
-/*
- * VM_FREELIST_DEFAULT is split into MAXMEMDOM lists, one for each
- * domain.  These extra lists are stored at the end of the regular
- * free lists starting with VM_NFREELIST.
- */
-#define VM_RAW_NFREELIST   (VM_NFREELIST + MAXMEMDOM - 1)
-
 struct vm_freelist {
struct pglist pl;
int lcnt;
@@ -84,6 +80,8 @@ struct vm_phys_seg {
 
 struct mem_affinity *mem_affinity;
 
+int vm_ndomains = 1;
+
 static struct vm_phys_seg vm_phys_segs[VM_PHYSSEG_MAX];
 
 static int vm_phys_nsegs;
@@ -98,9 +96,7 @@ static struct mtx vm_phys_fictitious_reg
 MALLOC_DEFINE(M_FICT_PAGES, "", "");
 
 static struct vm_freelist
-vm_phys_free_queues[VM_RAW_NFREELIST][VM_NFREEPOOL][VM_NFREEORDER];
-static struct vm_freelist
-(*vm_phys_lookup_lists[MAXMEMDOM][VM_RAW_NFREELIST])[VM_NFREEPOOL][VM_NFREEORDER];
+vm_phys_free_queues[MAXMEMDOM][VM_NFREELIST][VM_NFREEPOOL][VM_NFREEORDER];
 
 static int vm_nfreelists = VM_FREELIST_DEFAULT + 1;
 
@@ -116,11 +112,8 @@ static int sysctl_vm_phys_segs(SYSCTL_HA
 SYSCTL_OID(_vm, OID_AUTO, phys_segs, CTLTYPE_STRING | CTLFLAG_RD,
 NULL, 0, sysctl_vm_phys_segs, "A", "Phys Seg Info");
 
-#if MAXMEMDOM > 1
-static int sysctl_vm_phys_lookup_lists(SYSCTL_HANDLER_ARGS);
-SYSCTL_OID(_vm, OID_AUTO, phys_lookup_lists, CTLTYPE_STRING | CTLFLAG_RD,
-NULL, 0, sysctl_vm_phys_lookup_lists, "A", "Phys Lookup Lists");
-#endif
+SYSCTL_INT(_vm, OID_AUTO, ndomains, CTLFLAG_RD,
+&vm_ndomains, 0, "Number of physical memory domains available.");
 
 static vm_page_t vm_phys_alloc_domain_pages(int domain, int flind, int pool,
 int order);
@@ -131,6 +124,22 @@ static int vm_phys_paddr_to_segind(vm_pa
 static void vm_phys_split_pages(vm_page_t m, int oind, struct vm_freelist *fl,
 int order);
 
+static __inline int
+vm_rr_selectdomain(void)
+{
+#if MAXMEMDOM > 1
+   struct thread *td;
+
+   td = curthread;
+
+   td->td_dom_rr_idx++;
+   td->td_dom_rr_idx %= vm_ndomains;
+   return (td->td_dom_rr_idx);
+#else
+   return (0);
+#endif
+}
+
 /*
  * Outputs the state of the physical memory allocator, specifically,
  * the amount of physical memory in each free list.
@@ -140,31 +149,37 @@ sysctl_vm_phys_free(SYSCTL_HANDLER_ARGS)
 {
struct sbuf sbuf;
struct vm_freelist *fl;
-   int error, flind, oind, pind;
+   int dom, error, f

Re: svn commit: r250411 - in head/sys: conf kern sys

2013-05-11 Thread Attilio Rao
On Sat, May 11, 2013 at 4:34 PM, Attilio Rao  wrote:
> On Fri, May 10, 2013 at 9:33 PM, John Baldwin  wrote:
>> On Friday, May 10, 2013 2:51:20 pm Marcel Moolenaar wrote:
>>>
>>> On May 10, 2013, at 9:11 AM, John Baldwin  wrote:
>>>
>>> > On Friday, May 10, 2013 11:46:54 am Marcel Moolenaar wrote:
>>> >>>
>>> >>> 2) vnode locks from a local filesystem that report a LOR with a "devfs"
>>> >>>  vnode.  Typical reports are either "ufs" -> "devfs" or in some cases
>>> >>>  "ufs" -> "devfs" -> "ufs".  As with 1), I would much rather tag the
>>> >>>  offending location than to disable all WITNESS checking on vnode locks.
>>> >>
>>> >> With more file system types in use, this will get mixed up with the
>>> >> other file systems and noise you get is rather severe. It is a big
>>> >> problem for us at Juniper.
>>> >
>>> > Note, it is very specific that the second lock is always "devfs".  I think
>>> > that points to this being isolated to a few specific places, not a generic
>>> > ordering problem.
>>>
>>> Alas, that's not the case. These LORs are reported between ufs and unionfs,
>>> or ufs and isofs, etc. It's not just between something and devfs.
>>
>> Ugh, I have only seen them with devfs so had presumed them to be more
>> localized (and thus more easily targeted).  In that case your change
>> may be as fine-grained as we can get.  I would also like to still keep
>> WITNESS checking between vnode locks and other lock types, and LK_NOWITNESS
>> would remove that, so between your change and Attilio's approach I do
>> prefer yours.
>>
>>> I'm not sure the only options we have are to ignore the problem
>>> or implement a general fix. If we set out to silence witness for
>>> the known false positives then it's ok to handle them on a case
>>> by case basis. We'll see patterns soon enough and then re-code
>>> the solutions in terms of those patterns. If we're lucky we see
>>> a single general solution, but if not, then it's fine to have a
>>> handful of special case. The worse we can do is not address it
>>> at all.
>>
>> I was assuming that the reversals were far more specific, and knowing about
>> other false positives like the dirhash one, I want a generic way to do more
>> fine-grained marking of false positives.  If there were only a few places we
>> would need to mark to fix the reversals you see, then I would prefer the
>> suspend/resume approach for your case.  However, the reversals you are 
>> masking
>> sound too widespread to support that.
>
> The solution to this is what I proposed: pass down a LK_NOWITNESS for
> instances where you don't want to check for witness.
> This is per lock-call.
> You localize the fix to the instance you want to shut down and you
> skip witness for every false-positive.
> When you commit the LK_NOWITNESS you mention explicitely the reason
> why the reported LOR is a false positive so it is also documented on
> svn.
>
> I still don't understand what you are objecting. Marcel objections'
> were completely no-sense (Don't want to involve Witness) but at least
> I expect some decent one by you.

And finally, all the logic to do this seems already implemented. I
didn't recall that.
See this:
http://svnweb.freebsd.org/base?view=revision&revision=179554

What is missing is similar logic for sx(9) and rwlock(9). We can
simply add _flags() variant of such functions if we need it.
But Marcel's approach is not helping on that side as well, if not for
a simple red-herring effect.

Attilio


--
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r250411 - in head/sys: conf kern sys

2013-05-11 Thread Attilio Rao
On Fri, May 10, 2013 at 9:33 PM, John Baldwin  wrote:
> On Friday, May 10, 2013 2:51:20 pm Marcel Moolenaar wrote:
>>
>> On May 10, 2013, at 9:11 AM, John Baldwin  wrote:
>>
>> > On Friday, May 10, 2013 11:46:54 am Marcel Moolenaar wrote:
>> >>>
>> >>> 2) vnode locks from a local filesystem that report a LOR with a "devfs"
>> >>>  vnode.  Typical reports are either "ufs" -> "devfs" or in some cases
>> >>>  "ufs" -> "devfs" -> "ufs".  As with 1), I would much rather tag the
>> >>>  offending location than to disable all WITNESS checking on vnode locks.
>> >>
>> >> With more file system types in use, this will get mixed up with the
>> >> other file systems and noise you get is rather severe. It is a big
>> >> problem for us at Juniper.
>> >
>> > Note, it is very specific that the second lock is always "devfs".  I think
>> > that points to this being isolated to a few specific places, not a generic
>> > ordering problem.
>>
>> Alas, that's not the case. These LORs are reported between ufs and unionfs,
>> or ufs and isofs, etc. It's not just between something and devfs.
>
> Ugh, I have only seen them with devfs so had presumed them to be more
> localized (and thus more easily targeted).  In that case your change
> may be as fine-grained as we can get.  I would also like to still keep
> WITNESS checking between vnode locks and other lock types, and LK_NOWITNESS
> would remove that, so between your change and Attilio's approach I do
> prefer yours.
>
>> I'm not sure the only options we have are to ignore the problem
>> or implement a general fix. If we set out to silence witness for
>> the known false positives then it's ok to handle them on a case
>> by case basis. We'll see patterns soon enough and then re-code
>> the solutions in terms of those patterns. If we're lucky we see
>> a single general solution, but if not, then it's fine to have a
>> handful of special case. The worse we can do is not address it
>> at all.
>
> I was assuming that the reversals were far more specific, and knowing about
> other false positives like the dirhash one, I want a generic way to do more
> fine-grained marking of false positives.  If there were only a few places we
> would need to mark to fix the reversals you see, then I would prefer the
> suspend/resume approach for your case.  However, the reversals you are masking
> sound too widespread to support that.

The solution to this is what I proposed: pass down a LK_NOWITNESS for
instances where you don't want to check for witness.
This is per lock-call.
You localize the fix to the instance you want to shut down and you
skip witness for every false-positive.
When you commit the LK_NOWITNESS you mention explicitely the reason
why the reported LOR is a false positive so it is also documented on
svn.

I still don't understand what you are objecting. Marcel objections'
were completely no-sense (Don't want to involve Witness) but at least
I expect some decent one by you.

Attilio


--
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r250411 - in head/sys: conf kern sys

2013-05-11 Thread Attilio Rao
On Fri, May 10, 2013 at 3:52 PM, John Baldwin  wrote:
> On Thursday, May 09, 2013 4:56:33 pm Marcel Moolenaar wrote:
>>
>> On May 9, 2013, at 9:46 AM, Attilio Rao  wrote:
>>
>> > On Thu, May 9, 2013 at 6:28 PM, Marcel Moolenaar 
> wrote:
>> >> Author: marcel
>> >> Date: Thu May  9 16:28:18 2013
>> >> New Revision: 250411
>> >> URL: http://svnweb.freebsd.org/changeset/base/250411
>> >>
>> >> Log:
>> >>  Add option WITNESS_NO_VNODE to suppress printing LORs between VNODE
>> >>  locks. To support this, VNODE locks are created with the LK_IS_VNODE
>> >>  flag. This flag is propagated down using the LO_IS_VNODE flag.
>> >>
>> >>  Note that WITNESS still records the LOR. Only the printing and the
>> >>  optional entering into the kernel debugger is bypassed with the
>> >>  WITNESS_NO_VNODE option.
>> >
>> > This is the wrong way to deal with such problem and I avoided to do
>> > something like that on purpose.
>>
>> I disagree. We have known LOR messages between VNODE locks that
>> pollute the console and so far we haven't fixed the root cause
>> in some form or shape. Silencing this known case is good to
>> maximize the attention LORs need to be given while still have
>> witness involved to catch locking problems with vnodes that are
>> of a different nature.
>>
>> >
>> > The way to fix this is to implement LK_NOWITNESS on a per-lock basis
>> > into lockmgr, propagate the same concept to the vn_lock() (which
>> > should be basically done automatically) and finally identify the
>> > false-positive case and commit for them explicitely LK_NOWITNESS on a
>> > per-call basis, explaining in detail why the single case reported is a
>> > false-positive.
>>
>> This is worse. You want witness involved.
>
> Well, I disagree with both of you a bit.  I mostly agree with Attilio in
> that the committed change is a really large sledgehammer.  If we want to
> ignore all LORs for a large number of locks in the system we might as well
> remove WITNESS altogether.  However, I think LK_NOWITNESS is also too large
> of a sledgehammer for this as well.  AFAIK there are two vnode-related LORs
> that I can think of:

At this point I think that my e-mail is completely ambiguous because
what I'm proposing is the same way we shut down specific LOR with
mutexes using mtx_lock_flags().
I don't really understand what are you objecting about.

Attilio


--
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r250411 - in head/sys: conf kern sys

2013-05-09 Thread Attilio Rao
On Thu, May 9, 2013 at 10:56 PM, Marcel Moolenaar  wrote:
>
> On May 9, 2013, at 9:46 AM, Attilio Rao  wrote:
>
>> On Thu, May 9, 2013 at 6:28 PM, Marcel Moolenaar  wrote:
>>> Author: marcel
>>> Date: Thu May  9 16:28:18 2013
>>> New Revision: 250411
>>> URL: http://svnweb.freebsd.org/changeset/base/250411
>>>
>>> Log:
>>>  Add option WITNESS_NO_VNODE to suppress printing LORs between VNODE
>>>  locks. To support this, VNODE locks are created with the LK_IS_VNODE
>>>  flag. This flag is propagated down using the LO_IS_VNODE flag.
>>>
>>>  Note that WITNESS still records the LOR. Only the printing and the
>>>  optional entering into the kernel debugger is bypassed with the
>>>  WITNESS_NO_VNODE option.
>>
>> This is the wrong way to deal with such problem and I avoided to do
>> something like that on purpose.
>
> I disagree. We have known LOR messages between VNODE locks that
> pollute the console and so far we haven't fixed the root cause
> in some form or shape. Silencing this known case is good to
> maximize the attention LORs need to be given while still have
> witness involved to catch locking problems with vnodes that are
> of a different nature.
>
>>
>> The way to fix this is to implement LK_NOWITNESS on a per-lock basis
>> into lockmgr, propagate the same concept to the vn_lock() (which
>> should be basically done automatically) and finally identify the
>> false-positive case and commit for them explicitely LK_NOWITNESS on a
>> per-call basis, explaining in detail why the single case reported is a
>> false-positive.
>
> This is worse. You want witness involved.
>
>> Please revert this patch asap.
>
> This change does not inhibit people from fixing the problem at the
> root cause, and in the mean time maximize witness' effectiveness.
> Calling for a backout is unwarranted and unnecessarily aggressive.

I completely disagree with the whole content of your e-mail.
Thanks for disrupting a useful tool along with other commits which
happened in the past by other people about invariants effectiveness.

Attilio


--
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r250411 - in head/sys: conf kern sys

2013-05-09 Thread Attilio Rao
On Thu, May 9, 2013 at 6:28 PM, Marcel Moolenaar  wrote:
> Author: marcel
> Date: Thu May  9 16:28:18 2013
> New Revision: 250411
> URL: http://svnweb.freebsd.org/changeset/base/250411
>
> Log:
>   Add option WITNESS_NO_VNODE to suppress printing LORs between VNODE
>   locks. To support this, VNODE locks are created with the LK_IS_VNODE
>   flag. This flag is propagated down using the LO_IS_VNODE flag.
>
>   Note that WITNESS still records the LOR. Only the printing and the
>   optional entering into the kernel debugger is bypassed with the
>   WITNESS_NO_VNODE option.

This is the wrong way to deal with such problem and I avoided to do
something like that on purpose.

The way to fix this is to implement LK_NOWITNESS on a per-lock basis
into lockmgr, propagate the same concept to the vn_lock() (which
should be basically done automatically) and finally identify the
false-positive case and commit for them explicitely LK_NOWITNESS on a
per-call basis, explaining in detail why the single case reported is a
false-positive.

Please revert this patch asap.

Attilio


--
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r250395 - head/sys/sys

2013-05-08 Thread Attilio Rao
Author: attilio
Date: Thu May  9 00:04:59 2013
New Revision: 250395
URL: http://svnweb.freebsd.org/changeset/base/250395

Log:
  Generalize the bitset operations, present in cpuset and offer a KPI to
  redefine such operations for different consumers.
  This will be used when NUMA support will be finished and numaset
  will need to be used.
  
  Sponsored by: EMC / Isilon storage division
  Obtained from:jeff
  Reviewed by:  alc

Added:
  head/sys/sys/_bitset.h   (contents, props changed)
  head/sys/sys/bitset.h   (contents, props changed)
Modified:
  head/sys/sys/_cpuset.h
  head/sys/sys/cpuset.h

Added: head/sys/sys/_bitset.h
==
--- /dev/null   00:00:00 1970   (empty, because file is newly added)
+++ head/sys/sys/_bitset.h  Thu May  9 00:04:59 2013(r250395)
@@ -0,0 +1,61 @@
+/*-
+ * Copyright (c) 2008, Jeffrey Roberson 
+ * All rights reserved.
+ *
+ * Copyright (c) 2008 Nokia Corporation
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *notice unmodified, this list of conditions, and the following
+ *disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in the
+ *documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
+ * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
+ * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+ * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * $FreeBSD$
+ */
+
+#ifndef _SYS__BITSET_H_
+#define_SYS__BITSET_H_
+
+/*
+ * Macros addressing word and bit within it, tuned to make compiler
+ * optimize cases when SETSIZE fits into single machine word.
+ */
+#define_BITSET_BITS(sizeof(long) * NBBY)
+
+#define__bitset_words(_s)  (howmany(_s, _BITSET_BITS))
+
+#define__bitset_mask(_s, n)
\
+   (1L << ((__bitset_words((_s)) == 1) ?   \
+   (__size_t)(n) : ((n) % _BITSET_BITS)))
+
+#define__bitset_word(_s, n)
\
+   ((__bitset_words((_s)) == 1) ? 0 : ((n) / _BITSET_BITS))
+
+#defineBITSET_DEFINE(t, _s)
\
+struct t { \
+long__bits[__bitset_words((_s))];  \
+};
+
+#defineBITSET_T_INITIALIZER(x) 
\
+   { .__bits = { x } }
+
+#defineBITSET_FSET(n)  
\
+   [ 0 ... ((n) - 1) ] = (-1L)
+
+#endif /* !_SYS__BITSET_H_ */

Modified: head/sys/sys/_cpuset.h
==
--- head/sys/sys/_cpuset.h  Wed May  8 23:30:24 2013(r250394)
+++ head/sys/sys/_cpuset.h  Thu May  9 00:04:59 2013(r250395)
@@ -32,6 +32,8 @@
 #ifndef _SYS__CPUSET_H_
 #define_SYS__CPUSET_H_
 
+#include 
+
 #ifdef _KERNEL
 #defineCPU_SETSIZE MAXCPU
 #endif
@@ -42,17 +44,13 @@
 #defineCPU_SETSIZE CPU_MAXSIZE
 #endif
 
-#define_NCPUBITS   (sizeof(long) * NBBY)   /* bits per mask */
-#define_NCPUWORDS  howmany(CPU_SETSIZE, _NCPUBITS)
-
-typedefstruct _cpuset {
-   long__bits[howmany(CPU_SETSIZE, _NCPUBITS)];
-} cpuset_t;
+#define_NCPUBITS   _BITSET_BITS
+#define_NCPUWORDS  __bitset_words(CPU_SETSIZE)
 
-#defineCPUSET_FSET 
\
-   [ 0 ... (_NCPUWORDS - 1) ] = (-1L)
+BITSET_DEFINE(_cpuset, CPU_SETSIZE);
+typedef struct _cpuset cpuset_t;
 
-#defineCPUSET_T_INITIALIZER(x) 
\
-   { .__bits = { x } }
+#defineCPUSET_FSET BITSET_FSET(_NCPUWORDS)
+#defineCPUSET_T_INITIALIZERBITSET_T_INITIALIZER
 
 #endif /* !_SYS__CPUSET_H_ */

Added: head/sys/sys/bitset.h
==
--- /dev/null   00:00:00 1970   (empty, becaus

Re: svn commit: r250339 - head/sys/x86/acpica

2013-05-08 Thread Attilio Rao
On Wed, May 8, 2013 at 10:53 PM, John Baldwin  wrote:
> On Wednesday, May 08, 2013 2:21:12 pm Attilio Rao wrote:
>> On Wed, May 8, 2013 at 6:01 PM, John Baldwin  wrote:
>> > On Tuesday, May 07, 2013 6:49:57 pm Attilio Rao wrote:
>> >> Author: attilio
>> >> Date: Tue May  7 22:49:56 2013
>> >> New Revision: 250339
>> >> URL: http://svnweb.freebsd.org/changeset/base/250339
>> >>
>> >> Log:
>> >>   Add functions to do ACPI System Locality Information Table parsing
>> >>   and printing at boot.
>> >>   For reference on table informations and purposes please review ACPI 
>> >> specs.
>> >>
>> >>   Sponsored by:   EMC / Isilon storage division
>> >>   Obtained from:  jeff
>> >>   Reviewed by:jhb (earlier version)
>> >
>> > Eh, I don't recall reviewing this.  However, acpidump already dumps this
>> > table, and we should dump tables via acpidump, not in dmesg.
>>
>> Now that I read better this patch is based upon a patch you submitted
>> originally (see the private thread NUMA WIP).
>> We already print some useful ACPI table information at boot really
>> (APIC ACPI table case) and this is only printed out when having
>> MAXMEMDOM > 1. Finally I have also sent you this patch and you didn't
>> object.
>
> Humm, that was in an old thread, and I never responded to say "ok" to
> this.
>
> However, the patch I posted in that thread is the patch to dump the SLIT
> in acpidump and it was comitted back in r241198.  That is the place to dump
> these tables, not in dmesg.  We do not dump the raw contents of any other
> ACPI tables in dmesg, and have actively moved other tables out to userland
> (e.g. $PIR is dumped via a userland tool only).
>
>> However if you really dislike it I can print under bootverbose maybe?
>
> No, it does not belong in dmesg at all.  Code to use its contents and honor
> it when setting the domain lists would be fine :), but printing it when you
> can more easily get it via acpidump -t is just clutter.

Reverted in r 250389.

Attilio


--
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r250389 - head/sys/x86/acpica

2013-05-08 Thread Attilio Rao
Author: attilio
Date: Wed May  8 21:06:47 2013
New Revision: 250389
URL: http://svnweb.freebsd.org/changeset/base/250389

Log:
  Revert r250339 as apparently it is more clutter than help.
  
  Sponsored by: EMC / Isilon storage division
  Requested by: jhb

Modified:
  head/sys/x86/acpica/srat.c

Modified: head/sys/x86/acpica/srat.c
==
--- head/sys/x86/acpica/srat.c  Wed May  8 21:04:19 2013(r250388)
+++ head/sys/x86/acpica/srat.c  Wed May  8 21:06:47 2013(r250389)
@@ -331,48 +331,6 @@ srat_walk_table(acpi_subtable_handler *h
acpi_walk_subtables(srat + 1, (char *)srat + srat->Header.Length,
handler, arg);
 }
- 
-static void
-acpi_handle_slit(ACPI_TABLE_SLIT *slit)
-{
-   UINT64 i, j;
-
-   printf("ACPI System Locality Information Table: %ju localities\n",
-   (uintmax_t)slit->LocalityCount);
-   printf("  ");
-   for (i = 0; i < slit->LocalityCount; i++)
-   printf(" %3ju", (uintmax_t)i);
-   printf("\n +");
-   for (i = 0; i < slit->LocalityCount; i++)
-   printf("");
-   printf("\n");
-   for (i = 0; i < slit->LocalityCount; i++) {
-   printf(" %3ju |", (uintmax_t)i);
-   for (j = 0; j < slit->LocalityCount; j++)
-   printf(" %3u",
-   slit->Entry[i * slit->LocalityCount + j]);
-   printf("\n");
-   }
-}
-
-static void
-parse_slit(void *arg __unused)
-{
-   ACPI_TABLE_SLIT *slit;
-   vm_paddr_t slit_physaddr;
-
-   if (resource_disabled("slit", 0))
-   return;
-
-   slit_physaddr = acpi_find_table(ACPI_SIG_SLIT);
-   if (slit_physaddr == 0)
-   return;
-   slit = acpi_map_table(slit_physaddr, ACPI_SIG_SLIT);
-   acpi_handle_slit(slit);
-   acpi_unmap_table(slit);
-}
-
-SYSINIT(parse_slit, SI_SUB_VM - 1, SI_ORDER_SECOND, parse_slit, NULL);
 
 /*
  * Setup per-CPU ACPI IDs.
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r250339 - head/sys/x86/acpica

2013-05-08 Thread Attilio Rao
On Wed, May 8, 2013 at 6:01 PM, John Baldwin  wrote:
> On Tuesday, May 07, 2013 6:49:57 pm Attilio Rao wrote:
>> Author: attilio
>> Date: Tue May  7 22:49:56 2013
>> New Revision: 250339
>> URL: http://svnweb.freebsd.org/changeset/base/250339
>>
>> Log:
>>   Add functions to do ACPI System Locality Information Table parsing
>>   and printing at boot.
>>   For reference on table informations and purposes please review ACPI specs.
>>
>>   Sponsored by:   EMC / Isilon storage division
>>   Obtained from:  jeff
>>   Reviewed by:jhb (earlier version)
>
> Eh, I don't recall reviewing this.  However, acpidump already dumps this
> table, and we should dump tables via acpidump, not in dmesg.

Now that I read better this patch is based upon a patch you submitted
originally (see the private thread NUMA WIP).
We already print some useful ACPI table information at boot really
(APIC ACPI table case) and this is only printed out when having
MAXMEMDOM > 1. Finally I have also sent you this patch and you didn't
object.

However if you really dislike it I can print under bootverbose maybe?

Attilio


--
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r250338 - in head/sys: amd64/include arm/include conf i386/include ia64/include mips/include powerpc/include sparc64/include vm x86/acpica

2013-05-08 Thread Attilio Rao
On Wed, May 8, 2013 at 5:21 AM, Adrian Chadd  wrote:
> Hi,
>
> This broke a fresh buildworld for me. sys/vm/vm_phys.c still
> references VM_NDOMAIN.

Fixed in r250361. Sorry for the breakage.

Thanks,
Attilio


--
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r250361 - head/sys/vm

2013-05-08 Thread Attilio Rao
Author: attilio
Date: Wed May  8 10:55:39 2013
New Revision: 250361
URL: http://svnweb.freebsd.org/changeset/base/250361

Log:
  Fix-up r250338 by completing the removal of VM_NDOMAIN in favor of
  MAXMEMDOM.
  This unbreak builds.
  
  Sponsored by: EMC / Isilon storage division
  Reported by:  adrian, jeli

Modified:
  head/sys/vm/vm_phys.c

Modified: head/sys/vm/vm_phys.c
==
--- head/sys/vm/vm_phys.c   Wed May  8 10:53:17 2013(r250360)
+++ head/sys/vm/vm_phys.c   Wed May  8 10:55:39 2013(r250361)
@@ -464,7 +464,7 @@ vm_phys_alloc_pages(int pool, int order)
 vm_page_t
 vm_phys_alloc_freelist_pages(int flind, int pool, int order)
 {
-#if VM_NDOMAIN > 1
+#if MAXMEMDOM > 1
vm_page_t m;
int i, ndomains;
 #endif
@@ -477,7 +477,7 @@ vm_phys_alloc_freelist_pages(int flind, 
KASSERT(order < VM_NFREEORDER,
("vm_phys_alloc_freelist_pages: order %d is out of range", order));
 
-#if VM_NDOMAIN > 1
+#if MAXMEMDOM > 1
/*
 * This routine expects to be called with a VM_FREELIST_* constant.
 * On a system with multiple domains we need to adjust the flind
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r250339 - head/sys/x86/acpica

2013-05-07 Thread Attilio Rao
Author: attilio
Date: Tue May  7 22:49:56 2013
New Revision: 250339
URL: http://svnweb.freebsd.org/changeset/base/250339

Log:
  Add functions to do ACPI System Locality Information Table parsing
  and printing at boot.
  For reference on table informations and purposes please review ACPI specs.
  
  Sponsored by: EMC / Isilon storage division
  Obtained from:jeff
  Reviewed by:  jhb (earlier version)

Modified:
  head/sys/x86/acpica/srat.c

Modified: head/sys/x86/acpica/srat.c
==
--- head/sys/x86/acpica/srat.c  Tue May  7 22:46:24 2013(r250338)
+++ head/sys/x86/acpica/srat.c  Tue May  7 22:49:56 2013(r250339)
@@ -331,6 +331,48 @@ srat_walk_table(acpi_subtable_handler *h
acpi_walk_subtables(srat + 1, (char *)srat + srat->Header.Length,
handler, arg);
 }
+ 
+static void
+acpi_handle_slit(ACPI_TABLE_SLIT *slit)
+{
+   UINT64 i, j;
+
+   printf("ACPI System Locality Information Table: %ju localities\n",
+   (uintmax_t)slit->LocalityCount);
+   printf("  ");
+   for (i = 0; i < slit->LocalityCount; i++)
+   printf(" %3ju", (uintmax_t)i);
+   printf("\n +");
+   for (i = 0; i < slit->LocalityCount; i++)
+   printf("");
+   printf("\n");
+   for (i = 0; i < slit->LocalityCount; i++) {
+   printf(" %3ju |", (uintmax_t)i);
+   for (j = 0; j < slit->LocalityCount; j++)
+   printf(" %3u",
+   slit->Entry[i * slit->LocalityCount + j]);
+   printf("\n");
+   }
+}
+
+static void
+parse_slit(void *arg __unused)
+{
+   ACPI_TABLE_SLIT *slit;
+   vm_paddr_t slit_physaddr;
+
+   if (resource_disabled("slit", 0))
+   return;
+
+   slit_physaddr = acpi_find_table(ACPI_SIG_SLIT);
+   if (slit_physaddr == 0)
+   return;
+   slit = acpi_map_table(slit_physaddr, ACPI_SIG_SLIT);
+   acpi_handle_slit(slit);
+   acpi_unmap_table(slit);
+}
+
+SYSINIT(parse_slit, SI_SUB_VM - 1, SI_ORDER_SECOND, parse_slit, NULL);
 
 /*
  * Setup per-CPU ACPI IDs.
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r250338 - in head/sys: amd64/include arm/include conf i386/include ia64/include mips/include powerpc/include sparc64/include vm x86/acpica

2013-05-07 Thread Attilio Rao
Author: attilio
Date: Tue May  7 22:46:24 2013
New Revision: 250338
URL: http://svnweb.freebsd.org/changeset/base/250338

Log:
  Rename VM_NDOMAIN into MAXMEMDOM and move it into machine/param.h in
  order to match the MAXCPU concept.  The change should also be useful
  for consolidation and consistency.
  
  Sponsored by: EMC / Isilon storage division
  Obtained from:jeff
  Reviewed by:  alc

Modified:
  head/sys/amd64/include/param.h
  head/sys/amd64/include/vmparam.h
  head/sys/arm/include/param.h
  head/sys/arm/include/vmparam.h
  head/sys/conf/NOTES
  head/sys/conf/options
  head/sys/i386/include/param.h
  head/sys/i386/include/vmparam.h
  head/sys/ia64/include/param.h
  head/sys/ia64/include/vmparam.h
  head/sys/mips/include/param.h
  head/sys/mips/include/vmparam.h
  head/sys/powerpc/include/param.h
  head/sys/powerpc/include/vmparam.h
  head/sys/sparc64/include/param.h
  head/sys/sparc64/include/vmparam.h
  head/sys/vm/vm_phys.c
  head/sys/x86/acpica/srat.c

Modified: head/sys/amd64/include/param.h
==
--- head/sys/amd64/include/param.h  Tue May  7 22:05:57 2013
(r250337)
+++ head/sys/amd64/include/param.h  Tue May  7 22:46:24 2013
(r250338)
@@ -71,6 +71,10 @@
 #define MAXCPU 1
 #endif
 
+#ifndef MAXMEMDOM
+#defineMAXMEMDOM   1
+#endif
+
 #defineALIGNBYTES  _ALIGNBYTES
 #defineALIGN(p)_ALIGN(p)
 /*

Modified: head/sys/amd64/include/vmparam.h
==
--- head/sys/amd64/include/vmparam.hTue May  7 22:05:57 2013
(r250337)
+++ head/sys/amd64/include/vmparam.hTue May  7 22:46:24 2013
(r250338)
@@ -121,13 +121,6 @@
 #defineVM_NFREEORDER   13
 
 /*
- * Only one memory domain.
- */
-#ifndef VM_NDOMAIN
-#defineVM_NDOMAIN  1
-#endif
-
-/*
  * Enable superpage reservations: 1 level.
  */
 #ifndefVM_NRESERVLEVEL

Modified: head/sys/arm/include/param.h
==
--- head/sys/arm/include/param.hTue May  7 22:05:57 2013
(r250337)
+++ head/sys/arm/include/param.hTue May  7 22:46:24 2013
(r250338)
@@ -80,6 +80,10 @@
 #defineMAXCPU  1
 #endif /* SMP || KLD_MODULE */
 
+#ifndef MAXMEMDOM
+#defineMAXMEMDOM   1
+#endif
+
 #defineALIGNBYTES  _ALIGNBYTES
 #defineALIGN(p)_ALIGN(p)
 /*

Modified: head/sys/arm/include/vmparam.h
==
--- head/sys/arm/include/vmparam.h  Tue May  7 22:05:57 2013
(r250337)
+++ head/sys/arm/include/vmparam.h  Tue May  7 22:46:24 2013
(r250338)
@@ -109,13 +109,6 @@
 #defineVM_NFREEORDER   9
 
 /*
- * Only one memory domain.
- */
-#ifndef VM_NDOMAIN
-#defineVM_NDOMAIN  1
-#endif
-
-/*
  * Disable superpage reservations.
  */
 #ifndefVM_NRESERVLEVEL

Modified: head/sys/conf/NOTES
==
--- head/sys/conf/NOTES Tue May  7 22:05:57 2013(r250337)
+++ head/sys/conf/NOTES Tue May  7 22:46:24 2013(r250338)
@@ -224,6 +224,10 @@ optionsSMP # Symmetric 
MultiProcesso
 # A default value should be already present, for every architecture.
 optionsMAXCPU=32
 
+# MAXMEMDOM defines the maximum number of memory domains that can boot in the
+# system.  A default value should already be defined by every architecture.
+optionsMAXMEMDOM=1
+
 # ADAPTIVE_MUTEXES changes the behavior of blocking mutexes to spin
 # if the thread that currently owns the mutex is executing on another
 # CPU.  This behavior is enabled by default, so this option can be used

Modified: head/sys/conf/options
==
--- head/sys/conf/options   Tue May  7 22:05:57 2013(r250337)
+++ head/sys/conf/options   Tue May  7 22:46:24 2013(r250338)
@@ -568,6 +568,7 @@ DIAGNOSTIC  opt_global.h
 INVARIANT_SUPPORT  opt_global.h
 INVARIANTS opt_global.h
 MAXCPU opt_global.h
+MAXMEMDOM  opt_global.h
 MAXPHYSopt_global.h
 MCLSHIFT   opt_global.h
 MUTEX_DEBUGopt_global.h
@@ -584,7 +585,6 @@ VFS_BIO_DEBUG   opt_global.h
 VM_KMEM_SIZE   opt_vm.h
 VM_KMEM_SIZE_SCALE opt_vm.h
 VM_KMEM_SIZE_MAX   opt_vm.h
-VM_NDOMAIN opt_vm.h
 VM_NRESERVLEVELopt_vm.h
 VM_LEVEL_0_ORDER   opt_vm.h
 NO_SWAPPINGopt_vm.h

Modified: head/sys/i386/include/param.h
==
--- head/sys/i386/include/param.h   Tue May  7 22:05:57 2013  

Re: svn commit: r250237 - in head/sys: fs/smbfs netsmb

2013-05-04 Thread Attilio Rao
On Sat, May 4, 2013 at 4:18 PM, Davide Italiano  wrote:
> Author: davide
> Date: Sat May  4 14:18:10 2013
> New Revision: 250237
> URL: http://svnweb.freebsd.org/changeset/base/250237
>
> Log:
>   Overhaul locking in netsmb, getting rid of the obsolete lockmgr() primitive.
>   This solves a long standing LOR between smb_conn and smb_vc.

I've only skimmed through the patch so I may be missing something
important, but can you please clarify how did you replace LK_DRAIN in,
for example, smb_co_rele()?

Attilio


--
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r249277 - head/sys/kern

2013-04-09 Thread Attilio Rao
On Tue, Apr 9, 2013 at 5:32 PM, Konstantin Belousov  wrote:
> On Mon, Apr 08, 2013 at 07:58:32PM +0000, Attilio Rao wrote:
>> Author: attilio
>> Date: Mon Apr  8 19:58:32 2013
>> New Revision: 249277
>> URL: http://svnweb.freebsd.org/changeset/base/249277
>>
>> Log:
>>   Switch some "low-hanging fruit" to acquire read lock on vmobjects
>>   rather than write locks.
>
> Could you, please, change the assertion types for object page accessors,
> like vm_page_lookup(), as the first step ?

This is going to happen pretty soon.
Alan and I have some code that depends on such assertions changing and
we want to do it in structured way.

Attilio


--
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r249278 - head/sys/vm

2013-04-08 Thread Attilio Rao
Author: attilio
Date: Mon Apr  8 20:02:27 2013
New Revision: 249278
URL: http://svnweb.freebsd.org/changeset/base/249278

Log:
  The per-page act_count can be made very-easily protected by the
  per-page lock rather than vm_object lock, without any further overhead.
  Make the formal switch.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  alc
  Tested by:pho

Modified:
  head/sys/vm/vm_page.h
  head/sys/vm/vm_pageout.c

Modified: head/sys/vm/vm_page.h
==
--- head/sys/vm/vm_page.h   Mon Apr  8 19:58:32 2013(r249277)
+++ head/sys/vm/vm_page.h   Mon Apr  8 20:02:27 2013(r249278)
@@ -143,7 +143,7 @@ struct vm_page {
uint8_t aflags; /* access is atomic */
uint8_t oflags; /* page VPO_* flags (O) */
uint16_t flags; /* page PG_* flags (P) */
-   u_char  act_count;  /* page usage count (O) */
+   u_char  act_count;  /* page usage count (P) */
u_char  busy;   /* page busy count (O) */
/* NOTE that these must support one bit per DEV_BSIZE in a page!!! */
/* so, on normal X86 kernels, they must be at least 8 bits wide */

Modified: head/sys/vm/vm_pageout.c
==
--- head/sys/vm/vm_pageout.cMon Apr  8 19:58:32 2013(r249277)
+++ head/sys/vm/vm_pageout.cMon Apr  8 20:02:27 2013(r249278)
@@ -1015,9 +1015,9 @@ vm_pageout_scan(int pass)
} else if ((m->aflags & PGA_REFERENCED) == 0 &&
(actcount = pmap_ts_referenced(m)) != 0) {
vm_page_activate(m);
-   vm_page_unlock(m);
-   m->act_count += actcount + ACT_ADVANCE;
VM_OBJECT_WUNLOCK(object);
+   m->act_count += actcount + ACT_ADVANCE;
+   vm_page_unlock(m);
goto relock_queues;
}
 
@@ -1031,9 +1031,9 @@ vm_pageout_scan(int pass)
vm_page_aflag_clear(m, PGA_REFERENCED);
actcount = pmap_ts_referenced(m);
vm_page_activate(m);
-   vm_page_unlock(m);
-   m->act_count += actcount + ACT_ADVANCE + 1;
VM_OBJECT_WUNLOCK(object);
+   m->act_count += actcount + ACT_ADVANCE + 1;
+   vm_page_unlock(m);
goto relock_queues;
}
 
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r249277 - head/sys/kern

2013-04-08 Thread Attilio Rao
Author: attilio
Date: Mon Apr  8 19:58:32 2013
New Revision: 249277
URL: http://svnweb.freebsd.org/changeset/base/249277

Log:
  Switch some "low-hanging fruit" to acquire read lock on vmobjects
  rather than write locks.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  alc
  Tested by:pho

Modified:
  head/sys/kern/imgact_elf.c
  head/sys/kern/kern_proc.c
  head/sys/kern/sys_process.c

Modified: head/sys/kern/imgact_elf.c
==
--- head/sys/kern/imgact_elf.c  Mon Apr  8 19:57:21 2013(r249276)
+++ head/sys/kern/imgact_elf.c  Mon Apr  8 19:58:32 2013(r249277)
@@ -1277,15 +1277,15 @@ each_writable_segment(td, func, closure)
continue;
 
/* Ignore memory-mapped devices and such things. */
-   VM_OBJECT_WLOCK(object);
+   VM_OBJECT_RLOCK(object);
while ((backing_object = object->backing_object) != NULL) {
-   VM_OBJECT_WLOCK(backing_object);
-   VM_OBJECT_WUNLOCK(object);
+   VM_OBJECT_RLOCK(backing_object);
+   VM_OBJECT_RUNLOCK(object);
object = backing_object;
}
ignore_entry = object->type != OBJT_DEFAULT &&
object->type != OBJT_SWAP && object->type != OBJT_VNODE;
-   VM_OBJECT_WUNLOCK(object);
+   VM_OBJECT_RUNLOCK(object);
if (ignore_entry)
continue;
 

Modified: head/sys/kern/kern_proc.c
==
--- head/sys/kern/kern_proc.c   Mon Apr  8 19:57:21 2013(r249276)
+++ head/sys/kern/kern_proc.c   Mon Apr  8 19:58:32 2013(r249277)
@@ -1995,7 +1995,7 @@ sysctl_kern_proc_ovmmap(SYSCTL_HANDLER_A
kve->kve_private_resident = 0;
obj = entry->object.vm_object;
if (obj != NULL) {
-   VM_OBJECT_WLOCK(obj);
+   VM_OBJECT_RLOCK(obj);
if (obj->shadow_count == 1)
kve->kve_private_resident =
obj->resident_page_count;
@@ -2010,9 +2010,9 @@ sysctl_kern_proc_ovmmap(SYSCTL_HANDLER_A
 
for (lobj = tobj = obj; tobj; tobj = tobj->backing_object) {
if (tobj != obj)
-   VM_OBJECT_WLOCK(tobj);
+   VM_OBJECT_RLOCK(tobj);
if (lobj != obj)
-   VM_OBJECT_WUNLOCK(lobj);
+   VM_OBJECT_RUNLOCK(lobj);
lobj = tobj;
}
 
@@ -2072,11 +2072,11 @@ sysctl_kern_proc_ovmmap(SYSCTL_HANDLER_A
break;
}
if (lobj != obj)
-   VM_OBJECT_WUNLOCK(lobj);
+   VM_OBJECT_RUNLOCK(lobj);
 
kve->kve_ref_count = obj->ref_count;
kve->kve_shadow_count = obj->shadow_count;
-   VM_OBJECT_WUNLOCK(obj);
+   VM_OBJECT_RUNLOCK(obj);
if (vp != NULL) {
vn_fullpath(curthread, vp, &fullpath,
&freepath);
@@ -2162,7 +2162,7 @@ sysctl_kern_proc_vmmap(SYSCTL_HANDLER_AR
kve->kve_private_resident = 0;
obj = entry->object.vm_object;
if (obj != NULL) {
-   VM_OBJECT_WLOCK(obj);
+   VM_OBJECT_RLOCK(obj);
if (obj->shadow_count == 1)
kve->kve_private_resident =
obj->resident_page_count;
@@ -2183,9 +2183,9 @@ sysctl_kern_proc_vmmap(SYSCTL_HANDLER_AR
 
for (lobj = tobj = obj; tobj; tobj = tobj->backing_object) {
if (tobj != obj)
-   VM_OBJECT_WLOCK(tobj);
+   VM_OBJECT_RLOCK(tobj);
if (lobj != obj)
-   VM_OBJECT_WUNLOCK(lobj);
+   VM_OBJECT_RUNLOCK(lobj);
lobj = tobj;
}
 
@@ -2247,11 +2247,11 @@ sysctl_kern_proc_vmmap(SYSCTL_HANDLER_AR
break;
}
if (lobj != obj)
-   VM_OBJECT_WUNLOCK(lobj);
+   VM_OBJECT_RUNLOCK(lobj);
 
kve->kve_ref_count = obj->ref_count;
kve->kve_shadow_count = obj->shadow_count;
-   VM_OBJECT_WUNLOCK(obj);
+   VM_OBJECT_RUNLOCK(obj);
if (vp != NULL) {
 

svn commit: r248591 - head/sys/kern

2013-03-21 Thread Attilio Rao
Author: attilio
Date: Thu Mar 21 19:58:25 2013
New Revision: 248591
URL: http://svnweb.freebsd.org/changeset/base/248591

Log:
  Fix a bug in UMTX_PROFILING:
  UMTX_PROFILING should really analyze the distribution of locks as they
  index entries in the umtxq_chains hash-table.
  However, the current implementation does add/dec the length counters
  for *every* thread insert/removal, measuring at all really userland
  contention and not the hash distribution.
  
  Fix this by correctly add/dec the length counters in the points where
  it is really needed.
  
  Please note that this bug brought us questioning in the past the quality
  of the umtx hash table distribution.
  To date with all the benchmarks I could try I was not able to reproduce
  any issue about the hash distribution on umtx.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  jeff, davide
  MFC after:2 weeks

Modified:
  head/sys/kern/kern_umtx.c

Modified: head/sys/kern/kern_umtx.c
==
--- head/sys/kern/kern_umtx.c   Thu Mar 21 18:59:02 2013(r248590)
+++ head/sys/kern/kern_umtx.c   Thu Mar 21 19:58:25 2013(r248591)
@@ -542,19 +542,19 @@ umtxq_insert_queue(struct umtx_q *uq, in
uh = uq->uq_spare_queue;
uh->key = uq->uq_key;
LIST_INSERT_HEAD(&uc->uc_queue[q], uh, link);
+#ifdef UMTX_PROFILING
+   uc->length++;
+   if (uc->length > uc->max_length) {
+   uc->max_length = uc->length;
+   if (uc->max_length > max_length)
+   max_length = uc->max_length;
+   }
+#endif
}
uq->uq_spare_queue = NULL;
 
TAILQ_INSERT_TAIL(&uh->head, uq, uq_link);
uh->length++;
-#ifdef UMTX_PROFILING
-   uc->length++;
-   if (uc->length > uc->max_length) {
-   uc->max_length = uc->length;
-   if (uc->max_length > max_length)
-   max_length = uc->max_length;
-   }
-#endif
uq->uq_flags |= UQF_UMTXQ;
uq->uq_cur_queue = uh;
return;
@@ -572,13 +572,13 @@ umtxq_remove_queue(struct umtx_q *uq, in
uh = uq->uq_cur_queue;
TAILQ_REMOVE(&uh->head, uq, uq_link);
uh->length--;
-#ifdef UMTX_PROFILING
-   uc->length--;
-#endif
uq->uq_flags &= ~UQF_UMTXQ;
if (TAILQ_EMPTY(&uh->head)) {
KASSERT(uh->length == 0,
("inconsistent umtxq_queue length"));
+#ifdef UMTX_PROFILING
+   uc->length--;
+#endif
LIST_REMOVE(uh, link);
} else {
uh = LIST_FIRST(&uc->uc_spare_queue);
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r248449 - in head/sys: amd64/amd64 amd64/include conf i386/i386 i386/include i386/xen vm

2013-03-17 Thread Attilio Rao
Author: attilio
Date: Mon Mar 18 00:25:02 2013
New Revision: 248449
URL: http://svnweb.freebsd.org/changeset/base/248449

Log:
  Sync back vmcontention branch into HEAD:
  Replace the per-object resident and cached pages splay tree with a
  path-compressed multi-digit radix trie.
  Along with this, switch also the x86-specific handling of idle page
  tables to using the radix trie.
  
  This change is supposed to do the following:
  - Allowing the acquisition of read locking for lookup operations of the
resident/cached pages collections as the per-vm_page_t splay iterators
are now removed.
  - Increase the scalability of the operations on the page collections.
  
  The radix trie does rely on the consumers locking to ensure atomicity of
  its operations.  In order to avoid deadlocks the bisection nodes are
  pre-allocated in the UMA zone.  This can be done safely because the
  algorithm needs at maximum one new node per insert which means the
  maximum number of the desired nodes is the number of available physical
  frames themselves.  However, not all the times a new bisection node is
  really needed.
  
  The radix trie implements path-compression because UFS indirect blocks
  can lead to several objects with a very sparse trie, increasing the number
  of levels to usually scan.  It also helps in the nodes pre-fetching by
  introducing the single node per-insert property.
  
  This code is not generalized (yet) because of the possible loss of
  performance by having much of the sizes in play configurable.
  However, efforts to make this code more general and then reusable in
  further different consumers might be really done.
  
  The only KPI change is the removal of the function vm_page_splay() which
  is now reaped.
  The only KBI change, instead, is the removal of the left/right iterators
  from struct vm_page, which are now reaped.
  
  Further technical notes broken into mealpieces can be retrieved from the
  svn branch:
  http://svn.freebsd.org/base/user/attilio/vmcontention/
  
  Sponsored by: EMC / Isilon storage division
  In collaboration with:alc, jeff
  Tested by:flo, pho, jhb, davide
  Tested by:ian (arm)
  Tested by:andreast (powerpc)

Added:
  head/sys/vm/_vm_radix.h
 - copied unchanged from r248448, 
user/attilio/vmcontention/sys/vm/_vm_radix.h
  head/sys/vm/vm_radix.c
 - copied unchanged from r248448, 
user/attilio/vmcontention/sys/vm/vm_radix.c
  head/sys/vm/vm_radix.h
 - copied unchanged from r248448, 
user/attilio/vmcontention/sys/vm/vm_radix.h
Modified:
  head/sys/amd64/amd64/pmap.c
  head/sys/amd64/include/pmap.h
  head/sys/conf/files
  head/sys/i386/i386/pmap.c
  head/sys/i386/include/pmap.h
  head/sys/i386/xen/pmap.c
  head/sys/vm/vm_object.c
  head/sys/vm/vm_object.h
  head/sys/vm/vm_page.c
  head/sys/vm/vm_page.h
  head/sys/vm/vm_reserv.c

Modified: head/sys/amd64/amd64/pmap.c
==
--- head/sys/amd64/amd64/pmap.c Sun Mar 17 23:53:06 2013(r248448)
+++ head/sys/amd64/amd64/pmap.c Mon Mar 18 00:25:02 2013(r248449)
@@ -131,6 +131,7 @@ __FBSDID("$FreeBSD$");
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 
@@ -1497,7 +1498,8 @@ pmap_free_zero_pages(vm_page_t free)
 
while (free != NULL) {
m = free;
-   free = m->right;
+   free = (void *)m->object;
+   m->object = NULL;
/* Preserve the page's PG_ZERO setting. */
vm_page_free_toq(m);
}
@@ -1516,7 +1518,7 @@ pmap_add_delayed_free_list(vm_page_t m, 
m->flags |= PG_ZERO;
else
m->flags &= ~PG_ZERO;
-   m->right = *free;
+   m->object = (void *)*free;
*free = m;
 }

@@ -1526,31 +1528,12 @@ pmap_add_delayed_free_list(vm_page_t m, 
  * for mapping a distinct range of virtual addresses.  The pmap's collection is
  * ordered by this virtual address range.
  */
-static void
+static __inline void
 pmap_insert_pt_page(pmap_t pmap, vm_page_t mpte)
 {
-   vm_page_t root;
 
PMAP_LOCK_ASSERT(pmap, MA_OWNED);
-   root = pmap->pm_root;
-   if (root == NULL) {
-   mpte->left = NULL;
-   mpte->right = NULL;
-   } else {
-   root = vm_page_splay(mpte->pindex, root);
-   if (mpte->pindex < root->pindex) {
-   mpte->left = root->left;
-   mpte->right = root;
-   root->left = NULL;
-   } else if (mpte->pindex == root->pindex)
-   panic("pmap_insert_pt_page: pindex already inserted");
-   else {
-   mpte->right = root->right;
-   mpte->left = root;
-   root->right = NULL;
-   }
-   }
-   pmap->pm_root = mpte;
+   vm_radix_insert(&pmap->pm_root, mpte);
 }
 
 /*
@@ -1558,19 +1541,12 @@ pmap_insert

svn commit: r248210 - head/sys/sys

2013-03-12 Thread Attilio Rao
Author: attilio
Date: Tue Mar 12 14:30:36 2013
New Revision: 248210
URL: http://svnweb.freebsd.org/changeset/base/248210

Log:
  Bump __FreeBSD_version after r248084, breaking VM KPI to introduce
  read/write lockers.
  
  Sponsored by: EMC / Isilon storage division
  Requested by: flo

Modified:
  head/sys/sys/param.h

Modified: head/sys/sys/param.h
==
--- head/sys/sys/param.hTue Mar 12 14:21:52 2013(r248209)
+++ head/sys/sys/param.hTue Mar 12 14:30:36 2013(r248210)
@@ -58,7 +58,7 @@
  * in the range 5 to 9.
  */
 #undef __FreeBSD_version
-#define __FreeBSD_version 129  /* Master, propagated to newvers */
+#define __FreeBSD_version 130  /* Master, propagated to newvers */
 
 /*
  * __FreeBSD_kernel__ indicates that this system uses the kernel of FreeBSD,
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r248197 - head/sys/vm

2013-03-12 Thread Attilio Rao
Author: attilio
Date: Tue Mar 12 12:20:49 2013
New Revision: 248197
URL: http://svnweb.freebsd.org/changeset/base/248197

Log:
  Simplify vm_page_is_valid().
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  alc

Modified:
  head/sys/vm/vm_page.c

Modified: head/sys/vm/vm_page.c
==
--- head/sys/vm/vm_page.c   Tue Mar 12 12:19:23 2013(r248196)
+++ head/sys/vm/vm_page.c   Tue Mar 12 12:20:49 2013(r248197)
@@ -2898,10 +2898,7 @@ vm_page_is_valid(vm_page_t m, int base, 
 
VM_OBJECT_ASSERT_WLOCKED(m->object);
bits = vm_page_bits(base, size);
-   if (m->valid && ((m->valid & bits) == bits))
-   return 1;
-   else
-   return 0;
+   return (m->valid != 0 && (m->valid & bits) == bits);
 }
 
 /*
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r248105 - head/sys/kern

2013-03-09 Thread Attilio Rao
Author: attilio
Date: Sat Mar  9 15:31:19 2013
New Revision: 248105
URL: http://svnweb.freebsd.org/changeset/base/248105

Log:
  Improve UMTX_PROFILING:
  - Use u_int values for length and max_length values
  - Add a way to reset the max_length heuristic in order to have the
possibility to reuse the mechanism consecutively without rebooting
the machine
  - Add a way to quick display top5 contented buckets in the system for
the max_length value.
This should give a quick overview on the quality of the hash table
distribution.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  jeff, davide

Modified:
  head/sys/kern/kern_umtx.c

Modified: head/sys/kern/kern_umtx.c
==
--- head/sys/kern/kern_umtx.c   Sat Mar  9 15:04:44 2013(r248104)
+++ head/sys/kern/kern_umtx.c   Sat Mar  9 15:31:19 2013(r248105)
@@ -39,6 +39,7 @@ __FBSDID("$FreeBSD$");
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -64,6 +65,11 @@ __FBSDID("$FreeBSD$");
 #define _UMUTEX_TRY1
 #define _UMUTEX_WAIT   2
 
+#ifdef UMTX_PROFILING
+#defineUPROF_PERC_BIGGER(w, f, sw, sf) 
\
+   (((w) > (sw)) || ((w) == (sw) && (f) > (sf)))
+#endif
+
 /* Priority inheritance mutex info. */
 struct umtx_pi {
/* Owner thread */
@@ -157,8 +163,8 @@ struct umtxq_chain {
TAILQ_HEAD(,umtx_pi)uc_pi_list;
 
 #ifdef UMTX_PROFILING
-   int length;
-   int max_length;
+   u_int   length;
+   u_int   max_length;
 #endif
 };
 
@@ -252,6 +258,117 @@ umtx_init_profiling(void) 
"max_length1", CTLFLAG_RD, &umtxq_chains[1][i].max_length, 
0, NULL);
}
 }
+
+static int
+sysctl_debug_umtx_chains_peaks(SYSCTL_HANDLER_ARGS)
+{
+   char buf[512];
+   struct sbuf sb;
+   struct umtxq_chain *uc;
+   u_int fract, i, j, tot, whole;
+   u_int sf0, sf1, sf2, sf3, sf4;
+   u_int si0, si1, si2, si3, si4;
+   u_int sw0, sw1, sw2, sw3, sw4;
+
+   sbuf_new(&sb, buf, sizeof(buf), SBUF_FIXEDLEN);
+   for (i = 0; i < 2; i++) {
+   tot = 0;
+   for (j = 0; j < UMTX_CHAINS; ++j) {
+   uc = &umtxq_chains[i][j];
+   mtx_lock(&uc->uc_lock);
+   tot += uc->max_length;
+   mtx_unlock(&uc->uc_lock);
+   }
+   if (tot == 0)
+   sbuf_printf(&sb, "%u) Empty ", i);
+   else {
+   sf0 = sf1 = sf2 = sf3 = sf4 = 0;
+   si0 = si1 = si2 = si3 = si4 = 0;
+   sw0 = sw1 = sw2 = sw3 = sw4 = 0;
+   for (j = 0; j < UMTX_CHAINS; j++) {
+   uc = &umtxq_chains[i][j];
+   mtx_lock(&uc->uc_lock);
+   whole = uc->max_length * 100;
+   mtx_unlock(&uc->uc_lock);
+   fract = (whole % tot) * 100;
+   if (UPROF_PERC_BIGGER(whole, fract, sw0, sf0)) {
+   sf0 = fract;
+   si0 = j;
+   sw0 = whole;
+   } else if (UPROF_PERC_BIGGER(whole, fract, sw1,
+   sf1)) {
+   sf1 = fract;
+   si1 = j;
+   sw1 = whole;
+   } else if (UPROF_PERC_BIGGER(whole, fract, sw2,
+   sf2)) {
+   sf2 = fract;
+   si2 = j;
+   sw2 = whole;
+   } else if (UPROF_PERC_BIGGER(whole, fract, sw3,
+   sf3)) {
+   sf3 = fract;
+   si3 = j;
+   sw3 = whole;
+   } else if (UPROF_PERC_BIGGER(whole, fract, sw4,
+   sf4)) {
+   sf4 = fract;
+   si4 = j;
+   sw4 = whole;
+   }
+   }
+   sbuf_printf(&sb, "queue %u:\n", i);
+   sbuf_printf(&sb, "1st: %u.%u%% idx: %u\n", sw0 / tot,
+   sf0 / tot, si0);
+   sbuf_printf(&sb, "2nd: %u.%u%% idx: %u\n", sw1 / tot,
+   sf1 / tot, si1);
+   sbuf_printf(&sb, "3rd: %u.%u%% idx: %u\n", sw2 / tot,
+ 

Re: svn commit: r248097 - in head: . lib/libncp lib/libprocstat share/examples/nwclient sys/fs/nwfs sys/modules/ncp sys/modules/nwfs sys/netncp usr.bin/ncplist usr.bin/ncplogin usr.sbin/mount_nwfs

2013-03-09 Thread Attilio Rao
On Sat, Mar 9, 2013 at 4:02 PM, Robert Watson  wrote:
> Hi Attilio:
>
> It's really great to see the continued progress towards the goal
> entirely-MPSAFE VFS in 10.x -- we owe you a huge vote of thanks for pursuing
> this!

Thanks for the kind words.

The VFS can be considered completely MPSAFE by date, the only
remaining thing to do is sweeping out smbfs/netsmb.
However a known FreeBSD shops has patches to make smbfs MPSAFE and I'd
rather give them more time to commit them and re-add smbfs as a MPSAFE
filesystem rather than get into their way and remove the support.

This is the last bit to sort out.

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r248097 - in head: . lib/libncp lib/libprocstat share/examples/nwclient sys/fs/nwfs sys/modules/ncp sys/modules/nwfs sys/netncp usr.bin/ncplist usr.bin/ncplogin usr.sbin/mount_nwfs

2013-03-09 Thread Attilio Rao
Author: attilio
Date: Sat Mar  9 12:45:36 2013
New Revision: 248097
URL: http://svnweb.freebsd.org/changeset/base/248097

Log:
  Garbage collect NWFS and NCP bits which are now completely disconnected
  from the tree since few months.
  
  This patch is not targeted for MFC.

Deleted:
  head/lib/libncp/
  head/lib/libprocstat/nwfs.c
  head/share/examples/nwclient/
  head/sys/fs/nwfs/
  head/sys/modules/ncp/
  head/sys/modules/nwfs/
  head/sys/netncp/
  head/usr.bin/ncplist/
  head/usr.bin/ncplogin/
  head/usr.sbin/mount_nwfs/
Modified:
  head/ObsoleteFiles.inc

Modified: head/ObsoleteFiles.inc
==
--- head/ObsoleteFiles.inc  Sat Mar  9 12:34:45 2013(r248096)
+++ head/ObsoleteFiles.inc  Sat Mar  9 12:45:36 2013(r248097)
@@ -38,6 +38,43 @@
 #   xargs -n1 | sort | uniq -d;
 # done
 
+# 20130902: NWFS and NCP supports removed
+OLD_FILES+=usr/bin/ncplist
+OLD_FILES+=usr/bin/ncplogin
+OLD_FILES+=usr/bin/ncplogout
+OLD_FILES+=usr/include/fs/nwfs/nwfs.h
+OLD_FILES+=usr/include/fs/nwfs/nwfs_mount.h
+OLD_FILES+=usr/include/fs/nwfs/nwfs_node.h
+OLD_FILES+=usr/include/fs/nwfs/nwfs_subr.h
+OLD_DIRS+=usr/include/fs/nwfs
+OLD_FILES+=usr/include/netncp/ncp.h
+OLD_FILES+=usr/include/netncp/ncp_cfg.h
+OLD_FILES+=usr/include/netncp/ncp_conn.h
+OLD_FILES+=usr/include/netncp/ncp_file.h
+OLD_FILES+=usr/include/netncp/ncp_lib.h
+OLD_FILES+=usr/include/netncp/ncp_ncp.h
+OLD_FILES+=usr/include/netncp/ncp_nls.h
+OLD_FILES+=usr/include/netncp/ncp_rcfile.h
+OLD_FILES+=usr/include/netncp/ncp_rq.h
+OLD_FILES+=usr/include/netncp/ncp_sock.h
+OLD_FILES+=usr/include/netncp/ncp_subr.h
+OLD_FILES+=usr/include/netncp/ncp_user.h
+OLD_FILES+=usr/include/netncp/ncpio.h
+OLD_FILES+=usr/include/netncp/nwerror.h
+OLD_DIRS+=usr/include/netncp
+OLD_FILES+=usr/lib/libncp.a
+OLD_FILES+=usr/lib/libncp.so
+OLD_LIBS+=usr/lib/libncp.so.4
+OLD_FILES+=usr/lib/libncp_p.a
+OLD_FILES+=usr/lib32/libncp.a
+OLD_FILES+=usr/lib32/libncp.so
+OLD_LIBS+=usr/lib32/libncp.so.4
+OLD_FILES+=usr/lib32/libncp_p.a
+OLD_FILES+=usr/sbin/mount_nwfs
+OLD_FILES+=usr/share/man/man1/ncplist.1.gz
+OLD_FILES+=usr/share/man/man1/ncplogin.1.gz
+OLD_FILES+=usr/share/man/man1/ncplogout.1.gz
+OLD_FILES+=usr/share/man/man8/mount_nwfs.8.gz
 # 20130302: NTFS support removed
 OLD_FILES+=rescue/mount_ntfs
 OLD_FILES+=sbin/mount_ntfs
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r248084 - in head/sys: amd64/amd64 arm/arm cddl/compat/opensolaris/kern cddl/compat/opensolaris/sys cddl/contrib/opensolaris/uts/common/fs/zfs cddl/contrib/opensolaris/uts/common/fs/zfs...

2013-03-08 Thread Attilio Rao
Author: attilio
Date: Sat Mar  9 02:32:23 2013
New Revision: 248084
URL: http://svnweb.freebsd.org/changeset/base/248084

Log:
  Switch the vm_object mutex to be a rwlock.  This will enable in the
  future further optimizations where the vm_object lock will be held
  in read mode most of the time the page cache resident pool of pages
  are accessed for reading purposes.
  
  The change is mostly mechanical but few notes are reported:
  * The KPI changes as follow:
- VM_OBJECT_LOCK() -> VM_OBJECT_WLOCK()
- VM_OBJECT_TRYLOCK() -> VM_OBJECT_TRYWLOCK()
- VM_OBJECT_UNLOCK() -> VM_OBJECT_WUNLOCK()
- VM_OBJECT_LOCK_ASSERT(MA_OWNED) -> VM_OBJECT_ASSERT_WLOCKED()
  (in order to avoid visibility of implementation details)
- The read-mode operations are added:
  VM_OBJECT_RLOCK(), VM_OBJECT_TRYRLOCK(), VM_OBJECT_RUNLOCK(),
  VM_OBJECT_ASSERT_RLOCKED(), VM_OBJECT_ASSERT_LOCKED()
  * The vm/vm_pager.h namespace pollution avoidance (forcing requiring
sys/mutex.h in consumers directly to cater its inlining functions
using VM_OBJECT_LOCK()) imposes that all the vm/vm_pager.h
consumers now must include also sys/rwlock.h.
  * zfs requires a quite convoluted fix to include FreeBSD rwlocks into
the compat layer because the name clash between FreeBSD and solaris
versions must be avoided.
At this purpose zfs redefines the vm_object locking functions
directly, isolating the FreeBSD components in specific compat stubs.
  
  The KPI results heavilly broken by this commit.  Thirdy part ports must
  be updated accordingly (I can think off-hand of VirtualBox, for example).
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  jeff
  Reviewed by:  pjd (ZFS specific review)
  Discussed with:   alc
  Tested by:pho

Added:
  head/sys/cddl/compat/opensolaris/kern/opensolaris_vm.c
 - copied unchanged from r248082, 
user/attilio/vmobj-rwlock/sys/cddl/compat/opensolaris/kern/opensolaris_vm.c
  head/sys/cddl/compat/opensolaris/sys/freebsd_rwlock.h
 - copied unchanged from r248082, 
user/attilio/vmobj-rwlock/sys/cddl/compat/opensolaris/sys/freebsd_rwlock.h
  head/sys/cddl/compat/opensolaris/sys/vm.h
 - copied unchanged from r248082, 
user/attilio/vmobj-rwlock/sys/cddl/compat/opensolaris/sys/vm.h
Modified:
  head/sys/amd64/amd64/machdep.c
  head/sys/amd64/amd64/pmap.c
  head/sys/arm/arm/machdep.c
  head/sys/arm/arm/pmap-v6.c
  head/sys/arm/arm/pmap.c
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_context.h
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c
  head/sys/compat/linprocfs/linprocfs.c
  head/sys/conf/files
  head/sys/dev/agp/agp.c
  head/sys/dev/agp/agp_i810.c
  head/sys/dev/drm/drmP.h
  head/sys/dev/drm2/drmP.h
  head/sys/dev/drm2/i915/i915_gem.c
  head/sys/dev/drm2/ttm/ttm_bo_vm.c
  head/sys/dev/drm2/ttm/ttm_tt.c
  head/sys/dev/hwpmc/hwpmc_mod.c
  head/sys/dev/md/md.c
  head/sys/dev/netmap/netmap.c
  head/sys/dev/sound/pcm/dsp.c
  head/sys/fs/fuse/fuse_io.c
  head/sys/fs/fuse/fuse_vnops.c
  head/sys/fs/nfsclient/nfs_clbio.c
  head/sys/fs/nfsclient/nfs_clnode.c
  head/sys/fs/nfsclient/nfs_clvnops.c
  head/sys/fs/nfsserver/nfs_nfsdport.c
  head/sys/fs/procfs/procfs_map.c
  head/sys/fs/tmpfs/tmpfs_subr.c
  head/sys/fs/tmpfs/tmpfs_vnops.c
  head/sys/i386/i386/machdep.c
  head/sys/i386/i386/pmap.c
  head/sys/i386/xen/pmap.c
  head/sys/ia64/ia64/machdep.c
  head/sys/ia64/ia64/pmap.c
  head/sys/kern/imgact_elf.c
  head/sys/kern/kern_exec.c
  head/sys/kern/kern_proc.c
  head/sys/kern/kern_sharedpage.c
  head/sys/kern/kern_shutdown.c
  head/sys/kern/subr_uio.c
  head/sys/kern/sys_process.c
  head/sys/kern/sysv_shm.c
  head/sys/kern/uipc_shm.c
  head/sys/kern/uipc_syscalls.c
  head/sys/kern/vfs_aio.c
  head/sys/kern/vfs_bio.c
  head/sys/kern/vfs_cluster.c
  head/sys/kern/vfs_default.c
  head/sys/kern/vfs_subr.c
  head/sys/kern/vfs_syscalls.c
  head/sys/kern/vfs_vnops.c
  head/sys/mips/mips/machdep.c
  head/sys/mips/mips/pmap.c
  head/sys/modules/zfs/Makefile
  head/sys/nfsclient/nfs_bio.c
  head/sys/nfsclient/nfs_vnops.c
  head/sys/nfsserver/nfs_serv.c
  head/sys/ofed/drivers/infiniband/core/umem.c
  head/sys/ofed/include/linux/linux_compat.c
  head/sys/pc98/pc98/machdep.c
  head/sys/powerpc/aim/machdep.c
  head/sys/powerpc/aim/mmu_oea.c
  head/sys/powerpc/aim/mmu_oea64.c
  head/sys/powerpc/booke/machdep.c
  head/sys/powerpc/booke/pmap.c
  head/sys/security/mac/mac_process.c
  head/sys/sparc64/sparc64/machdep.c
  head/sys/sparc64/sparc64/pmap.c
  head/sys/ufs/ffs/ffs_rawread.c
  head/sys/ufs/ffs/ffs_vnops.c
  head/sys/vm/default_pager.c
  head/sys/vm/device_pager.c
  head/sys/vm/phys_pager.c
  head/sys/vm/sg_pager.c
  head/sys/vm/swap_pager.c
  head/sys/vm/uma_core.c
  head/sys/vm/vm_fault.c
  head/sys/vm/vm_glue.c
  head/sys/vm/vm_init.c
  head/sys/vm/vm_kern.c
  head/sys/vm/vm_map.c
  head/sys/vm/vm_meter.c
  head/sys/vm/vm_mmap.c
  head/sys/vm/vm_object.c
  head/sys/vm/vm_object.h
  head/sys/vm/vm_page.c
  head/sys/vm/v

svn commit: r248082 - in head/sys: cddl/compat/opensolaris/sys vm

2013-03-08 Thread Attilio Rao
Author: attilio
Date: Sat Mar  9 02:05:29 2013
New Revision: 248082
URL: http://svnweb.freebsd.org/changeset/base/248082

Log:
  Merge from vmc-playground:
  Introduce a new KPI that verifies if the page cache is empty for a
  specified vm_object.  This KPI does not make assumptions about the
  locking in order to be used also for building assertions at init and
  destroy time.
  It is mostly used to hide implementation details of the page cache.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  jeff
  Reviewed by:  alc (vm_radix based version)
  Tested by:flo, pho, jhb, davide

Modified:
  head/sys/cddl/compat/opensolaris/sys/vnode.h
  head/sys/vm/vm_object.c
  head/sys/vm/vm_object.h
  head/sys/vm/vm_page.c

Modified: head/sys/cddl/compat/opensolaris/sys/vnode.h
==
--- head/sys/cddl/compat/opensolaris/sys/vnode.hSat Mar  9 01:54:26 
2013(r248081)
+++ head/sys/cddl/compat/opensolaris/sys/vnode.hSat Mar  9 02:05:29 
2013(r248082)
@@ -76,7 +76,7 @@ vn_is_readonly(vnode_t *vp)
 #definevn_has_cached_data(vp)  \
((vp)->v_object != NULL && \
 ((vp)->v_object->resident_page_count > 0 || \
- (vp)->v_object->cache != NULL))
+ !vm_object_cache_is_empty((vp)->v_object)))
 #definevn_exists(vp)   do { } while (0)
 #definevn_invalid(vp)  do { } while (0)
 #definevn_renamepath(tdvp, svp, tnm, lentnm)   do { } while (0)

Modified: head/sys/vm/vm_object.c
==
--- head/sys/vm/vm_object.c Sat Mar  9 01:54:26 2013(r248081)
+++ head/sys/vm/vm_object.c Sat Mar  9 02:05:29 2013(r248082)
@@ -173,7 +173,7 @@ vm_object_zdtor(void *mem, int size, voi
("object %p has reservations",
object));
 #endif
-   KASSERT(object->cache == NULL,
+   KASSERT(vm_object_cache_is_empty(object),
("object %p has cached pages",
object));
KASSERT(object->paging_in_progress == 0,
@@ -752,7 +752,7 @@ vm_object_terminate(vm_object_t object)
if (__predict_false(!LIST_EMPTY(&object->rvq)))
vm_reserv_break_all(object);
 #endif
-   if (__predict_false(object->cache != NULL))
+   if (__predict_false(!vm_object_cache_is_empty(object)))
vm_page_cache_free(object, 0, 0);
 
/*
@@ -1377,7 +1377,7 @@ retry:
 * should still be OBJT_DEFAULT and orig_object should not
 * contain any cached pages within the specified range.
 */
-   if (__predict_false(orig_object->cache != NULL))
+   if (__predict_false(!vm_object_cache_is_empty(orig_object)))
vm_page_cache_transfer(orig_object, offidxstart,
new_object);
}
@@ -1726,7 +1726,8 @@ vm_object_collapse(vm_object_t object)
/*
 * Free any cached pages from backing_object.
 */
-   if (__predict_false(backing_object->cache != 
NULL))
+   if (__predict_false(
+   !vm_object_cache_is_empty(backing_object)))
vm_page_cache_free(backing_object, 0, 
0);
}
/*
@@ -1920,7 +1921,7 @@ again:
}
vm_object_pip_wakeup(object);
 skipmemq:
-   if (__predict_false(object->cache != NULL))
+   if (__predict_false(!vm_object_cache_is_empty(object)))
vm_page_cache_free(object, start, end);
 }
 

Modified: head/sys/vm/vm_object.h
==
--- head/sys/vm/vm_object.h Sat Mar  9 01:54:26 2013(r248081)
+++ head/sys/vm/vm_object.h Sat Mar  9 02:05:29 2013(r248082)
@@ -229,6 +229,13 @@ void vm_object_pip_wakeup(vm_object_t ob
 void vm_object_pip_wakeupn(vm_object_t object, short i);
 void vm_object_pip_wait(vm_object_t object, char *waitid);
 
+static __inline boolean_t
+vm_object_cache_is_empty(vm_object_t object)
+{
+
+   return (object->cache == NULL);
+}
+
 vm_object_t vm_object_allocate (objtype_t, vm_pindex_t);
 boolean_t vm_object_coalesce(vm_object_t, vm_ooffset_t, vm_size_t, vm_size_t,
boolean_t);

Modified: head/sys/vm/vm_page.c
==
--- head/sys/vm/vm_page.c   Sat Mar  9 01:54:26 2013(r248081)
+++ head/sys/vm/vm_page.c   Sat Mar  9 02:05:29 2013(r248082)
@@ -1129,7 +1129,7 @@ vm_page_cache_free(vm_object_t object, v
boolean_t empty;
 
mtx_lock(&vm_page_queue_free_mtx);
-   if (__predict_false(object->cache == NULL)) {
+   if (__predict_false(vm_object_cache_is_empty(object)

svn commit: r247788 - head/sys/vm

2013-03-04 Thread Attilio Rao
Author: attilio
Date: Mon Mar  4 13:10:59 2013
New Revision: 247788
URL: http://svnweb.freebsd.org/changeset/base/247788

Log:
  Merge from vmcontention:
  As vm objects are type-stable there is no need to initialize the
  resident splay tree pointer and the cache splay tree pointer in
  _vm_object_allocate() but this could be done in the init UMA zone
  handler.
  
  The destructor UMA zone handler, will further check if the condition is
  retained at every destruction and catch for bugs.
  
  Sponsored by: EMC / Isilon storage division
  Submitted by: alc

Modified:
  head/sys/vm/vm_object.c

Modified: head/sys/vm/vm_object.c
==
--- head/sys/vm/vm_object.c Mon Mar  4 12:48:41 2013(r247787)
+++ head/sys/vm/vm_object.c Mon Mar  4 13:10:59 2013(r247788)
@@ -165,8 +165,9 @@ vm_object_zdtor(void *mem, int size, voi
 
object = (vm_object_t)mem;
KASSERT(TAILQ_EMPTY(&object->memq),
-   ("object %p has resident pages",
-   object));
+   ("object %p has resident pages in its memq", object));
+   KASSERT(object->root == NULL,
+   ("object %p has resident pages in its tree", object));
 #if VM_NRESERVLEVEL > 0
KASSERT(LIST_EMPTY(&object->rvq),
("object %p has reservations",
@@ -197,9 +198,11 @@ vm_object_zinit(void *mem, int size, int
mtx_init(&object->mtx, "vm object", NULL, MTX_DEF | MTX_DUPOK);
 
/* These are true for any object that has been freed */
+   object->root = NULL;
object->paging_in_progress = 0;
object->resident_page_count = 0;
object->shadow_count = 0;
+   object->cache = NULL;
return (0);
 }
 
@@ -210,7 +213,6 @@ _vm_object_allocate(objtype_t type, vm_p
TAILQ_INIT(&object->memq);
LIST_INIT(&object->shadow_head);
 
-   object->root = NULL;
object->type = type;
switch (type) {
case OBJT_DEAD:
@@ -247,7 +249,6 @@ _vm_object_allocate(objtype_t type, vm_p
 #if VM_NRESERVLEVEL > 0
LIST_INIT(&object->rvq);
 #endif
-   object->cache = NULL;
 
mtx_lock(&vm_object_list_mtx);
TAILQ_INSERT_TAIL(&vm_object_list, object, object_list);
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


Re: svn commit: r247709 - head

2013-03-03 Thread Attilio Rao
On Sun, Mar 3, 2013 at 12:09 PM, Antoine Brodin  wrote:
> Author: antoine
> Date: Sun Mar  3 11:09:55 2013
> New Revision: 247709
> URL: http://svnweb.freebsd.org/changeset/base/247709
>
> Log:
>   Add more obsolete files.

Pointy hat to me (most of).

Thanks,
Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r247678 - head/sys/i386/xen

2013-03-02 Thread Attilio Rao
Author: attilio
Date: Sun Mar  3 01:02:57 2013
New Revision: 247678
URL: http://svnweb.freebsd.org/changeset/base/247678

Log:
  Fix-up r247622 by also renaming pv_list iterator into the xen
  pmap verbatim copy.
  
  Sponsored by: EMC / Isilon storage division
  Reported by:  tinderbox

Modified:
  head/sys/i386/xen/pmap.c

Modified: head/sys/i386/xen/pmap.c
==
--- head/sys/i386/xen/pmap.cSun Mar  3 00:37:34 2013(r247677)
+++ head/sys/i386/xen/pmap.cSun Mar  3 01:02:57 2013(r247678)
@@ -2038,7 +2038,7 @@ pmap_pv_reclaim(pmap_t locked_pmap)
vm_page_dirty(m);
if ((tpte & PG_A) != 0)
vm_page_aflag_set(m, PGA_REFERENCED);
-   TAILQ_REMOVE(&m->md.pv_list, pv, pv_list);
+   TAILQ_REMOVE(&m->md.pv_list, pv, pv_next);
if (TAILQ_EMPTY(&m->md.pv_list))
vm_page_aflag_clear(m, PGA_WRITEABLE);
pc->pc_map[field] |= 1UL << bit;
@@ -2239,9 +2239,9 @@ pmap_pvh_remove(struct md_page *pvh, pma
pv_entry_t pv;
 
rw_assert(&pvh_global_lock, RA_WLOCKED);
-   TAILQ_FOREACH(pv, &pvh->pv_list, pv_list) {
+   TAILQ_FOREACH(pv, &pvh->pv_list, pv_next) {
if (pmap == PV_PMAP(pv) && va == pv->pv_va) {
-   TAILQ_REMOVE(&pvh->pv_list, pv, pv_list);
+   TAILQ_REMOVE(&pvh->pv_list, pv, pv_next);
break;
}
}
@@ -2281,7 +2281,7 @@ pmap_try_insert_pv_entry(pmap_t pmap, vm
if (pv_entry_count < pv_entry_high_water && 
(pv = get_pv_entry(pmap, TRUE)) != NULL) {
pv->pv_va = va;
-   TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_list);
+   TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_next);
return (TRUE);
} else
return (FALSE);
@@ -2503,7 +2503,7 @@ pmap_remove_all(vm_page_t m)
vm_page_dirty(m);
pmap_unuse_pt(pmap, pv->pv_va, &free);
pmap_invalidate_page(pmap, pv->pv_va);
-   TAILQ_REMOVE(&m->md.pv_list, pv, pv_list);
+   TAILQ_REMOVE(&m->md.pv_list, pv, pv_next);
free_pv_entry(pmap, pv);
PMAP_UNLOCK(pmap);
}
@@ -2770,7 +2770,7 @@ pmap_enter(pmap_t pmap, vm_offset_t va, 
if (pv == NULL)
pv = get_pv_entry(pmap, FALSE);
pv->pv_va = va;
-   TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_list);
+   TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_next);
pa |= PG_MANAGED;
} else if (pv != NULL)
free_pv_entry(pmap, pv);
@@ -3465,7 +3465,7 @@ pmap_page_exists_quick(pmap_t pmap, vm_p
("pmap_page_exists_quick: page %p is not managed", m));
rv = FALSE;
rw_wlock(&pvh_global_lock);
-   TAILQ_FOREACH(pv, &m->md.pv_list, pv_list) {
+   TAILQ_FOREACH(pv, &m->md.pv_list, pv_next) {
if (PV_PMAP(pv) == pmap) {
rv = TRUE;
break;
@@ -3497,7 +3497,7 @@ pmap_page_wired_mappings(vm_page_t m)
return (count);
rw_wlock(&pvh_global_lock);
sched_pin();
-   TAILQ_FOREACH(pv, &m->md.pv_list, pv_list) {
+   TAILQ_FOREACH(pv, &m->md.pv_list, pv_next) {
pmap = PV_PMAP(pv);
PMAP_LOCK(pmap);
pte = pmap_pte_quick(pmap, pv->pv_va);
@@ -3602,7 +3602,7 @@ pmap_remove_pages(pmap_t pmap)
if (tpte & PG_M)
vm_page_dirty(m);
 
-   TAILQ_REMOVE(&m->md.pv_list, pv, pv_list);
+   TAILQ_REMOVE(&m->md.pv_list, pv, pv_next);
if (TAILQ_EMPTY(&m->md.pv_list))
vm_page_aflag_clear(m, PGA_WRITEABLE);
 
@@ -3662,7 +3662,7 @@ pmap_is_modified(vm_page_t m)
return (rv);
rw_wlock(&pvh_global_lock);
sched_pin();
-   TAILQ_FOREACH(pv, &m->md.pv_list, pv_list) {
+   TAILQ_FOREACH(pv, &m->md.pv_list, pv_next) {
pmap = PV_PMAP(pv);
PMAP_LOCK(pmap);
pte = pmap_pte_quick(pmap, pv->pv_va);
@@ -3723,7 +3723,7 @@ pmap_is_referenced(vm_page_t m)
rv = FALSE;
rw_wlock(&pvh_global_lock);
sched_pin();
-   TAILQ_FOREACH(pv, &m->md.pv_list, pv_list) {
+   TAILQ_FOREACH(pv, &m->md.pv_list, pv_next) {
pmap = PV_PMAP(pv);
PMAP_LOCK(pmap);
pte = pmap_pte_quick(pmap, pv->pv_va);
@@ -3793,7 +3793,7 @@ pmap_remove_write(vm_page_t m)
return;
rw_wlock(&pvh_global_

svn commit: r247665 - in head: . lib/libprocstat sbin/mount_ntfs sys/fs/ntfs sys/modules/ntfs sys/modules/ntfs_iconv

2013-03-02 Thread Attilio Rao
Author: attilio
Date: Sat Mar  2 18:40:04 2013
New Revision: 247665
URL: http://svnweb.freebsd.org/changeset/base/247665

Log:
  Garbage collect NTFS bits which are now completely disconnected from
  the tree since few months.
  
  This patch is not targeted for MFC.

Deleted:
  head/lib/libprocstat/ntfs.c
  head/sbin/mount_ntfs/
  head/sys/fs/ntfs/
  head/sys/modules/ntfs/
  head/sys/modules/ntfs_iconv/
Modified:
  head/ObsoleteFiles.inc

Modified: head/ObsoleteFiles.inc
==
--- head/ObsoleteFiles.inc  Sat Mar  2 18:38:29 2013(r247664)
+++ head/ObsoleteFiles.inc  Sat Mar  2 18:40:04 2013(r247665)
@@ -38,6 +38,16 @@
 #   xargs -n1 | sort | uniq -d;
 # done
 
+# 20130302: NTFS support removed
+OLD_FILES+=usr/include/fs/ntfs/ntfs.h
+OLD_FILES+=usr/include/fs/ntfs/ntfs_compr.h
+OLD_FILES+=usr/include/fs/ntfs/ntfs_ihash.h
+OLD_FILES+=usr/include/fs/ntfs/ntfs_inode.h
+OLD_FILES+=usr/include/fs/ntfs/ntfs_subr.h
+OLD_FILES+=usr/include/fs/ntfs/ntfs_vfsops.h
+OLD_FILES+=usr/include/fs/ntfs/ntfsmount.h
+OLD_DIRS+=usr/include/fs/ntfs
+OLD_FILES+=usr/share/man/man8/mount_ntfs.8.gz
 # 20130302: PORTALFS support removed
 OLD_FILES+=usr/include/fs/portalfs/portal.h
 OLD_DIRS+=usr/include/fs/portalfs
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r247662 - head/sys/geom/label

2013-03-02 Thread Attilio Rao
Author: attilio
Date: Sat Mar  2 18:23:59 2013
New Revision: 247662
URL: http://svnweb.freebsd.org/changeset/base/247662

Log:
  Remove ntfs headers dependency for g_label_ntfs.c by redefining the
  used structs and values.
  
  This patch is not targeted for MFC.

Modified:
  head/sys/geom/label/g_label_ntfs.c

Modified: head/sys/geom/label/g_label_ntfs.c
==
--- head/sys/geom/label/g_label_ntfs.c  Sat Mar  2 18:18:30 2013
(r247661)
+++ head/sys/geom/label/g_label_ntfs.c  Sat Mar  2 18:23:59 2013
(r247662)
@@ -32,21 +32,72 @@ __FBSDID("$FreeBSD$");
 #include 
 #include 
 
-#include 
-
 #include 
 #include 
 
+#defineNTFS_A_VOLUMENAME   0x60
+#defineNTFS_FILEMAGIC  ((uint32_t)(0x454C4946))
+#defineNTFS_VOLUMEINO  3
+
 #define G_LABEL_NTFS_DIR   "ntfs"
 
+struct ntfs_attr {
+   uint32_ta_type;
+   uint32_treclen;
+   uint8_t a_flag;
+   uint8_t a_namelen;
+   uint8_t a_nameoff;
+   uint8_t reserved1;
+   uint8_t a_compression;
+   uint8_t reserved2;
+   uint16_ta_index;
+   uint16_ta_datalen;
+   uint16_treserved3;
+   uint16_ta_dataoff;
+   uint16_ta_indexed;
+};
+
+struct ntfs_filerec {
+   uint32_tfr_hdrmagic;
+   uint16_tfr_hdrfoff;
+   uint16_tfr_hdrfnum;
+   uint8_t reserved[8];
+   uint16_tfr_seqnum;
+   uint16_tfr_nlink;
+   uint16_tfr_attroff;
+   uint16_tfr_flags;
+   uint32_tfr_size;
+   uint32_tfr_allocated;
+   uint64_tfr_mainrec;
+   uint16_tfr_attrnum;
+};
+
+struct ntfs_bootfile {
+   uint8_t reserved1[3];
+   uint8_t bf_sysid[8];
+   uint16_tbf_bps;
+   uint8_t bf_spc;
+   uint8_t reserved2[7];
+   uint8_t bf_media;
+   uint8_t reserved3[2];
+   uint16_tbf_spt;
+   uint16_tbf_heads;
+   uint8_t reserver4[12];
+   uint64_tbf_spv;
+   uint64_tbf_mftcn;
+   uint64_tbf_mftmirrcn;
+   uint8_t bf_mftrecsz;
+   uint32_tbf_ibsz;
+   uint32_tbf_volsn;
+};
 
 static void
 g_label_ntfs_taste(struct g_consumer *cp, char *label, size_t size)
 {
struct g_provider *pp;
-   struct bootfile *bf;
-   struct filerec *fr;
-   struct attr *atr;
+   struct ntfs_bootfile *bf;
+   struct ntfs_filerec *fr;
+   struct ntfs_attr *atr;
off_t voloff;
char *filerecp, *ap;
char mftrecsz, vnchar;
@@ -58,7 +109,7 @@ g_label_ntfs_taste(struct g_consumer *cp
pp = cp->provider;
filerecp = NULL;
 
-   bf = (struct bootfile *)g_read_data(cp, 0, pp->sectorsize, NULL);
+   bf = (struct ntfs_bootfile *)g_read_data(cp, 0, pp->sectorsize, NULL);
if (bf == NULL || strncmp(bf->bf_sysid, "NTFS", 8) != 0)
goto done;
 
@@ -75,16 +126,16 @@ g_label_ntfs_taste(struct g_consumer *cp
filerecp = g_read_data(cp, voloff, recsize, NULL);
if (filerecp == NULL)
goto done;
-   fr = (struct filerec *)filerecp;
+   fr = (struct ntfs_filerec *)filerecp;
 
-   if (fr->fr_fixup.fh_magic != NTFS_FILEMAGIC)
+   if (fr->fr_hdrmagic != NTFS_FILEMAGIC)
goto done;
 
for (ap = filerecp + fr->fr_attroff;
-   atr = (struct attr *)ap, atr->a_hdr.a_type != -1;
-   ap += atr->a_hdr.reclen) {
-   if (atr->a_hdr.a_type == NTFS_A_VOLUMENAME) {
-   if(atr->a_r.a_datalen >= size *2){
+   atr = (struct ntfs_attr *)ap, atr->a_type != -1;
+   ap += atr->reclen) {
+   if (atr->a_type == NTFS_A_VOLUMENAME) {
+   if(atr->a_datalen >= size *2){
label[0] = 0;
goto done;
}
@@ -92,8 +143,8 @@ g_label_ntfs_taste(struct g_consumer *cp
 *UNICODE to ASCII.
 * Should we need to use iconv(9)?
 */
-   for (j = 0; j < atr->a_r.a_datalen; j++) {
-   vnchar = *(ap + atr->a_r.a_dataoff + j);
+   for (j = 0; j < atr->a_datalen; j++) {
+   vnchar = *(ap + atr->a_dataoff + j);
if (j & 1) {
if (vnchar) {
label[0] = 0;
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r247640 - in head: . share/examples/portal sys/fs/portalfs sys/modules/portalfs usr.sbin/mount_portalfs

2013-03-02 Thread Attilio Rao
Author: attilio
Date: Sat Mar  2 16:43:28 2013
New Revision: 247640
URL: http://svnweb.freebsd.org/changeset/base/247640

Log:
  Garbage collect PORTALFS bits which are now completely disconnected from
  the tree since few months.
  
  This patch is not targeted for MFC.

Deleted:
  head/share/examples/portal/
  head/sys/fs/portalfs/
  head/sys/modules/portalfs/
  head/usr.sbin/mount_portalfs/
Modified:
  head/ObsoleteFiles.inc

Modified: head/ObsoleteFiles.inc
==
--- head/ObsoleteFiles.inc  Sat Mar  2 16:39:00 2013(r247639)
+++ head/ObsoleteFiles.inc  Sat Mar  2 16:43:28 2013(r247640)
@@ -38,6 +38,13 @@
 #   xargs -n1 | sort | uniq -d;
 # done
 
+# 20130302: PORTALFS support removed
+OLD_FILES+=usr/include/fs/portalfs/portal.h
+OLD_DIRS+=usr/include/fs/portalfs
+OLD_FILES+=usr/share/examples/portal/README
+OLD_FILES+=usr/share/examples/portal/portal.conf
+OLD_DIRS+=usr/share/examples/portal
+OLD_FILES+=usr/share/man/man8/mount_portalfs.8.gz
 # 20130302: CODAFS support removed
 OLD_FILES+=usr/share/man/man4/coda.4.gz
 # 20130302: XFS support removed
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r247635 - in head: . share/man/man4 sys/fs/coda sys/modules/coda sys/modules/coda5

2013-03-02 Thread Attilio Rao
Author: attilio
Date: Sat Mar  2 16:30:18 2013
New Revision: 247635
URL: http://svnweb.freebsd.org/changeset/base/247635

Log:
  Garbage collect CODAFS bits which are now completely disconnected from
  the tree since few months.
  
  This patch is not targeted for MFC.

Deleted:
  head/share/man/man4/coda.4
  head/sys/fs/coda/
  head/sys/modules/coda/
  head/sys/modules/coda5/
Modified:
  head/ObsoleteFiles.inc

Modified: head/ObsoleteFiles.inc
==
--- head/ObsoleteFiles.inc  Sat Mar  2 16:18:40 2013(r247634)
+++ head/ObsoleteFiles.inc  Sat Mar  2 16:30:18 2013(r247635)
@@ -38,6 +38,8 @@
 #   xargs -n1 | sort | uniq -d;
 # done
 
+# 20130302: CODAFS support removed
+OLD_FILES+=usr/share/man/man4/coda.4.gz
 # 20130302: XFS support removed
 OLD_FILES+=usr/share/man/man5/xfs.5.gz
 # 20130116: removed long unused directories for .1aout section manpages
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r247631 - in head: . share/man/man5 sys/gnu/fs/xfs sys/modules/xfs

2013-03-02 Thread Attilio Rao
Author: attilio
Date: Sat Mar  2 15:33:54 2013
New Revision: 247631
URL: http://svnweb.freebsd.org/changeset/base/247631

Log:
  Garbage collect XFS bits which are now already completely disconnected
  from the tree since few months.
  
  This is not targeted for MFC.

Deleted:
  head/share/man/man5/xfs.5
  head/sys/gnu/fs/xfs/
  head/sys/modules/xfs/
Modified:
  head/ObsoleteFiles.inc

Modified: head/ObsoleteFiles.inc
==
--- head/ObsoleteFiles.inc  Sat Mar  2 15:33:06 2013(r247630)
+++ head/ObsoleteFiles.inc  Sat Mar  2 15:33:54 2013(r247631)
@@ -38,6 +38,8 @@
 #   xargs -n1 | sort | uniq -d;
 # done
 
+# 20130302: XFS support removed
+OLD_FILES+=usr/share/man/man5/xfs.5.gz
 # 20130116: removed long unused directories for .1aout section manpages
 OLD_FILES+=usr/share/man/en.ISO8859-1/man1aout
 OLD_FILES+=usr/share/man/en.UTF-8/man1aout
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r247628 - in head: sbin/mount_hpfs sys/fs/hpfs sys/modules/hpfs

2013-03-02 Thread Attilio Rao
Author: attilio
Date: Sat Mar  2 14:54:33 2013
New Revision: 247628
URL: http://svnweb.freebsd.org/changeset/base/247628

Log:
  Garbage collect HPFS bits which are now already completely disconnected
  from the tree since few months (please note that the userland bits
  were already disconnected since a long time, thus there is no need
  to update the OLD* entries).
  
  This is not targeted for MFC.

Deleted:
  head/sbin/mount_hpfs/
  head/sys/fs/hpfs/
  head/sys/modules/hpfs/
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


svn commit: r247622 - in head/sys: amd64/amd64 amd64/include i386/i386 i386/include

2013-03-02 Thread Attilio Rao
Author: attilio
Date: Sat Mar  2 14:19:08 2013
New Revision: 247622
URL: http://svnweb.freebsd.org/changeset/base/247622

Log:
  Merge from vmc-playground branch:
  Rename the pv_entry_t iterator from pv_list to pv_next.
  Besides being more correct technically (as the name seems to suggest
  this is a list while it is an iterator), it will also be needed by
  vm_radix work to avoid a nameclash on macro expansions.
  
  Sponsored by: EMC / Isilon storage division
  Reviewed by:  alc, jeff
  Tested by:flo, pho, jhb, davide

Modified:
  head/sys/amd64/amd64/pmap.c
  head/sys/amd64/include/pmap.h
  head/sys/i386/i386/pmap.c
  head/sys/i386/include/pmap.h

Modified: head/sys/amd64/amd64/pmap.c
==
--- head/sys/amd64/amd64/pmap.c Sat Mar  2 13:08:13 2013(r247621)
+++ head/sys/amd64/amd64/pmap.c Sat Mar  2 14:19:08 2013(r247622)
@@ -,7 +,7 @@ reclaim_pv_chunk(pmap_t locked_pmap, str
if ((tpte & PG_A) != 0)
vm_page_aflag_set(m, PGA_REFERENCED);
CHANGE_PV_LIST_LOCK_TO_VM_PAGE(lockp, m);
-   TAILQ_REMOVE(&m->md.pv_list, pv, pv_list);
+   TAILQ_REMOVE(&m->md.pv_list, pv, pv_next);
if (TAILQ_EMPTY(&m->md.pv_list) &&
(m->flags & PG_FICTITIOUS) == 0) {
pvh = pa_to_pvh(VM_PAGE_TO_PHYS(m));
@@ -2506,9 +2506,9 @@ pmap_pvh_remove(struct md_page *pvh, pma
pv_entry_t pv;
 
rw_assert(&pvh_global_lock, RA_LOCKED);
-   TAILQ_FOREACH(pv, &pvh->pv_list, pv_list) {
+   TAILQ_FOREACH(pv, &pvh->pv_list, pv_next) {
if (pmap == PV_PMAP(pv) && va == pv->pv_va) {
-   TAILQ_REMOVE(&pvh->pv_list, pv, pv_list);
+   TAILQ_REMOVE(&pvh->pv_list, pv, pv_next);
break;
}
}
@@ -2547,7 +2547,7 @@ pmap_pv_demote_pde(pmap_t pmap, vm_offse
pv = pmap_pvh_remove(pvh, pmap, va);
KASSERT(pv != NULL, ("pmap_pv_demote_pde: pv not found"));
m = PHYS_TO_VM_PAGE(pa);
-   TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_list);
+   TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_next);
/* Instantiate the remaining NPTEPG - 1 pv entries. */
PV_STAT(atomic_add_long(&pv_entry_allocs, NPTEPG - 1));
va_last = va + NBPDR - PAGE_SIZE;
@@ -2565,7 +2565,7 @@ pmap_pv_demote_pde(pmap_t pmap, vm_offse
m++;
KASSERT((m->oflags & VPO_UNMANAGED) == 0,
("pmap_pv_demote_pde: page %p is not managed", m));
-   TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_list);
+   TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_next);
if (va == va_last)
goto out;
}
@@ -2613,7 +2613,7 @@ pmap_pv_promote_pde(pmap_t pmap, vm_offs
pv = pmap_pvh_remove(&m->md, pmap, va);
KASSERT(pv != NULL, ("pmap_pv_promote_pde: pv not found"));
pvh = pa_to_pvh(pa);
-   TAILQ_INSERT_TAIL(&pvh->pv_list, pv, pv_list);
+   TAILQ_INSERT_TAIL(&pvh->pv_list, pv, pv_next);
/* Free the remaining NPTEPG - 1 pv entries. */
va_last = va + NBPDR - PAGE_SIZE;
do {
@@ -2654,7 +2654,7 @@ pmap_try_insert_pv_entry(pmap_t pmap, vm
if ((pv = get_pv_entry(pmap, NULL)) != NULL) {
pv->pv_va = va;
CHANGE_PV_LIST_LOCK_TO_VM_PAGE(lockp, m);
-   TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_list);
+   TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_next);
return (TRUE);
} else
return (FALSE);
@@ -2678,7 +2678,7 @@ pmap_pv_insert_pde(pmap_t pmap, vm_offse
pv->pv_va = va;
CHANGE_PV_LIST_LOCK_TO_PHYS(lockp, pa);
pvh = pa_to_pvh(pa);
-   TAILQ_INSERT_TAIL(&pvh->pv_list, pv, pv_list);
+   TAILQ_INSERT_TAIL(&pvh->pv_list, pv, pv_next);
return (TRUE);
} else
return (FALSE);
@@ -3157,7 +3157,7 @@ small_mappings:
vm_page_dirty(m);
pmap_unuse_pt(pmap, pv->pv_va, *pde, &free);
pmap_invalidate_page(pmap, pv->pv_va);
-   TAILQ_REMOVE(&m->md.pv_list, pv, pv_list);
+   TAILQ_REMOVE(&m->md.pv_list, pv, pv_next);
free_pv_entry(pmap, pv);
PMAP_UNLOCK(pmap);
}
@@ -3602,7 +3602,7 @@ retry:
pv = get_pv_entry(pmap, &lock);
pv->pv_va = va;
CHANGE_PV_LIST_LOCK_TO_PHYS(&lock, pa);
-   TAILQ_INSERT_TAIL(&m->md.pv_list, pv, pv_list);
+   TAILQ_INSERT_TAIL(&m->md.pv_li

Re: svn commit: r247588 - in head/sys: dev/mps kern sys

2013-03-01 Thread Attilio Rao
On Fri, Mar 1, 2013 at 11:03 PM, John Baldwin  wrote:
> Author: jhb
> Date: Fri Mar  1 22:03:31 2013
> New Revision: 247588
> URL: http://svnweb.freebsd.org/changeset/base/247588
>
> Log:
>   Replace the TDP_NOSLEEPING flag with a counter so that the
>   THREAD_NO_SLEEPING() and THREAD_SLEEPING_OK() macros can nest.
>
>   Reviewed by:  attilio

My cleanup had some small tweaks like:
- No trailing white space cleanup in mps
- td_no_sleeping u_int rather than int
- s/TDP_UNUSED9/TDP_UNUSED09 (consistency with others definitions)
- There is no need to use braces around curthread

Also the rmlock part doesn't belong to this patchset.

http://www.freebsd.org/~attilio/jhbnosleep.patch

Attilio


-- 
Peace can only be achieved by understanding - A. Einstein
___
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"


  1   2   3   4   5   6   7   8   >