> I see. I think dropping the ill_locks don't present a problem, but the
> ipsq_lock is trickier since dropping it in ipsq_dq() opens up some races.
> I'll put this issue on the TODO list.
I've now made the change, by moving the second ipsq_exit() to the end of
ipsq_dq(), after all locks are dropped. If you could take another look at
ipsq_dq(), ipsq_enter(), and ipsq_exit() in ip_if.c, I'd really appreciate
it:
http://cr.grommit.com/~meem/ipmp-cv-dev/
*** NOTE: please be sure to reload the webrev pages ***
Along the way, I addressed a large oversight in the way threads sleeping
in ipsq_enter() are awoken. The fix parallels the current Nevada logic:
when ipx_writer is cleared on an xop, all of the IPSQs associated with the
xop are walked, and the associated ill->ill_cv's are awoken:
/* ... */
ipx->ipx_writer = NULL;
VERIFY(--ipx->ipx_reentry_cnt == 0);
ipx->ipx_ipsq_queued = B_FALSE;
emptied = B_TRUE;
mutex_exit(&ipx->ipx_lock);
mutex_exit(&ipsq->ipsq_lock);
if (emptied) {
xopipsq = ipsq;
do {
if ((phyi = xopipsq->ipsq_phyint) == NULL)
continue;
illv4 = phyi->phyint_illv4;
illv6 = phyi->phyint_illv6;
GRAB_ILL_LOCKS(illv4, illv6);
if (illv4 != NULL)
cv_broadcast(&illv4->ill_cv);
if (illv6 != NULL)
cv_broadcast(&illv6->ill_cv);
RELEASE_ILL_LOCKS(illv4, illv6);
} while ((xopipsq = xopipsq->ipsq_next) != ipsq);
}
(Recall that in the new code, the binding between IPSQ and ill/phyint
is fixed, and is only set to NULL when the phyint itself is freed,
at which point there will never be any threads to wake up.)
--
meem
