On 12/23/2012 10:12 PM, Oleg Nesterov wrote:
> On 12/23, Srivatsa S. Bhat wrote:
>>
>> On 12/20/2012 07:12 PM, Oleg Nesterov wrote:
>>>
>>> We need mb() + rmb(). Plust cli/sti unless this arch has optimized
>>> this_cpu_add() like x86 (as you pointed out).
>>>
>>
>> Hey, IIUC, we actually don't need mb() in the reader!! Just an rmb() will do.
>
> Well. I don't think so. But when it comes to the barriers I am never sure
> until Paul confirms my understanding ;)
>
>> #define reader_nested_percpu()
>> \
>> (__this_cpu_read(reader_percpu_refcnt) & READER_REFCNT_MASK)
>>
>> #define writer_active()
>> \
>> (__this_cpu_read(writer_signal))
>>
>>
>> #define READER_PRESENT (1UL << 16)
>> #define READER_REFCNT_MASK (READER_PRESENT - 1)
>>
>> void get_online_cpus_atomic(void)
>> {
>> preempt_disable();
>>
>> /*
>> * First and foremost, make your presence known to the writer.
>> */
>> this_cpu_add(reader_percpu_refcnt, READER_PRESENT);
>>
>> /*
>> * If we are already using per-cpu refcounts, it is not safe to switch
>> * the synchronization scheme. So continue using the refcounts.
>> */
>> if (reader_nested_percpu()) {
>> this_cpu_inc(reader_percpu_refcnt);
>> } else {
>> smp_rmb();
>> if (unlikely(writer_active())) {
>> ... //take hotplug_rwlock
>> }
>> }
>>
>> ...
>>
>> /* Prevent reordering of any subsequent reads of cpu_online_mask. */
>> smp_rmb();
>> }
>>
>> The smp_rmb() before writer_active() ensures that LOAD(writer_signal) follows
>> LOAD(reader_percpu_refcnt) (at the 'if' condition). And in turn, that load is
>> automatically going to follow the STORE(reader_percpu_refcnt)
>
> But why this STORE should be visible on another CPU before we
> LOAD(writer_signal)?
>
> Lets discuss the simple and artificial example. Suppose we have
>
> int X, Y;
>
> int func(void)
> {
> X = 1; // suppose that nobody else can change it
> mb();
> return Y;
> }
>
> Now you are saying that we can change it and avoid the costly mb():
>
> int func(void)
> {
> X = 1;
>
> if (X != 1)
> BUG();
>
> rmb();
> return Y;
> }
>
> I doubt. rmb() can only guarantee that the preceding LOAD's should be
> completed. Without mb() it is possible that this CPU won't write X to
> memory at all.
>
Oh, ok :-( Thanks for correcting me and for the detailed explanation!
For a moment, I really thought we had it solved at last! ;-(
Regards,
Srivatsa S. Bhat
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
