Re: [ceph-users] Question about BUG #11332

[Gregory Farnum]

On Thu, Nov 23, 2017 at 1:55 AM 许雪寒  wrote:

> Hi, everyone.
>
>  We also encountered this problem: http://tracker.ceph.com/issues/11332.
> And we found that this seems to be caused by the lack of mutual exclusion
> between applying "trim" and handling subscriptions. Since
> "build_incremental" operations doesn't go through the "PAXOS" procedure,
> and applying "trim" contains two phases, which are modifying "mondbstore"
> and updating "cached_first_committed", there could be a chance for
> "send_incremental" operations to happen between them. What's more,
> "build_incremental" operations also contain two phases, getting
> "cached_first_committed" and getting actual incrementals for MonDBStore.
> So, if "build_incremental" do happens concurrently with applying "trim", it
> could get an out-dated "cached_first_committed" and try to read a full map
> whose already trimmed.
>
> Is this right?
>

I don't think this is right. Keep in mind that the monitors are basically a
single-threaded event-driven machine. Both trimming and building
incrementals happen in direct response to receiving messages, in the main
dispatch loop, and while trimming is happening the PaxosService is not
readable. So it won't be invoking build_incremental() and they won't run
concurrently.
-Greg


>
> If it is, we think maybe all “READ” operations in monitor should be
> synchronized with paxos commit. Right? Should some kind of read-write
> locking mechanism be used here?
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Question about BUG #11332

[许雪寒]

Hi, everyone.

 We also encountered this problem: http://tracker.ceph.com/issues/11332. And we 
found that this seems to be caused by the lack of mutual exclusion between 
applying "trim" and handling subscriptions. Since "build_incremental" 
operations doesn't go through the "PAXOS" procedure, and applying "trim" 
contains two phases, which are modifying "mondbstore" and updating 
"cached_first_committed", there could be a chance for "send_incremental" 
operations to happen between them. What's more, "build_incremental" operations 
also contain two phases, getting "cached_first_committed" and getting actual 
incrementals for MonDBStore. So, if "build_incremental" do happens concurrently 
with applying "trim", it could get an out-dated "cached_first_committed" and 
try to read a full map whose already trimmed.

Is this right?

If it is, we think maybe all “READ” operations in monitor should be 
synchronized with paxos commit. Right? Should some kind of read-write locking 
mechanism be used here?
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com