On Sun, Apr 9, 2017 at 11:14 PM, Yu Li <[email protected]> wrote: > Correct me if I'm wrong, but I think we should assume no other but the > single operation when checking whether it's idempotent. Similar to the > wikipedia > example <https://en.wikipedia.org/wiki/Idempotence#Examples>: "A function > looking up a customer's name and address in a database > <https://en.wikipedia.org/wiki/Database> is typically idempotent, since > this will not cause the database to change", I think all Get/MultiGet/Scan > operations in hbase are idempotent. > > About "speculative rpc handling", I doubt whether it benefits in hbase. > Normally if a request already arrives at server side but with slow > execution, the problem might be: > 1. The server is too busy and request get queued > 2. The processing itself is slow due to the request pattern or some > hardware failure > I don't think a speculative execution of the request could help in any of > the above cases. It's different from the speculative task execution in MR, > there we could choose another node to execute the task while here we have > no choice. >
We have a different use case here. Basically we are trying to enforce scheduling at HBase. Consider the following scenario: both client-1 and client-2 are competing for I/O resources. But client-2 are also issuing a bunch of requests that do not require any I/O resources (say, data is cached). Since we have idle CPU/memory, we want to serve these cached requests for client-2, but we do not want client-2 to use more than its fair share of I/O. Unfortunately, at the time we pick RPC call to handle, we don't know whether an RPC would cause I/O or not. So we think we can abort a request if it requires I/O resources that are not allocated to it, and re-schedule it later based on our scheduling policy. > > OTOH, we already have timeout mechanism to make sure server resource won't > be wasted: > 1. For scan > - When a request handling timeouts, server will stop further > processing, refer to RSRpcServices#getTimeLimit and > ScannerContext#checkTimeLimit > - If the client went away during processing, server will also stop > processing, check the SimpleRpcServer#disconnectSince and > RegionScannerImpl#nextInternal methods for more details. > > 2. For single Get > - Controlled by rpc and operation timeout > > 3. For MultiGet > - I think this is something we could improve. On client side we have > timeout mechanism but on server side there seems to be no relative > interrupt logic. > > > Best Regards, > Yu > > On 10 April 2017 at 11:12, Jerry He <[email protected]> wrote: > > > Again, it depends on how you abort and 'idempotent' can have different > > definitions. > > > > For example, even if you are only concerned about read, > > there are resources on the HRegion that the read touches or acquires > > (scanner, lock, mvcc etc) that hopefully will be cleaned/releases with > the > > abort. > > Or you may have it in a bad/inconsistent state. > > > > Thanks. > > > > Jerry > > > > > > On Sun, Apr 9, 2017 at 7:14 PM, 张铎(Duo Zhang) <[email protected]> > > wrote: > > > > > I think this depends on how you model the problem. At server side, if > you > > > re-execute a read operation with a new mvcc, then you may read a value > > that > > > should not be visible if you use the old mvcc. If you define this as an > > > error then I think there will be conflicts. > > > > > > But at client side, there is guarantee that the request you send first > > will > > > be executed first. So as long as the read request does not return, I > > think > > > it is OK to read a value which is written by a write request which is > > sent > > > after the read request? > > > > > > Thanks. > > > > > > 2017-04-10 9:52 GMT+08:00 杨苏立 Yang Su Li <[email protected]>: > > > > > > > We are only concerned about read operations here. Are you suggesting > > they > > > > are completely idempotent? > > > > Are there any read-after-write conflicts? > > > > > > > > Thanks > > > > > > > > Sui > > > > > > > > On Sun, Apr 9, 2017 at 8:48 PM, 张铎(Duo Zhang) <[email protected] > > > > > > wrote: > > > > > > > > > It depends on how you about the rpc request. For hbase, there will > be > > > no > > > > > write conflict, but a write operation can only be finished iff all > > the > > > > > write operations with a lower mvcc number have been finished. So if > > you > > > > > just stop a write operation without recovering the mvcc(I do not > know > > > how > > > > > to recover but I think you need to something...) then the writes > will > > > be > > > > > stuck. > > > > > > > > > > And one more thing, for read operation you may interrupt it at any > > > time, > > > > > but for write operation, I do not think you can re-execute it with > a > > > new > > > > > mvcc number if the WAL entry has already been flushed out. That > > means, > > > > the > > > > > re-execution process will be different if you about the write > > operation > > > > at > > > > > different stages. > > > > > > > > > > Thanks. > > > > > > > > > > 2017-04-10 6:47 GMT+08:00 杨苏立 Yang Su Li <[email protected]>: > > > > > > > > > > > We are trying to implement speculative rpc handling for our > > > workloads. > > > > So > > > > > > we want allow RPC Handler to stop executing an RPC call, put it > > back > > > to > > > > > the > > > > > > queue, and later re-execute it. > > > > > > > > > > > > If at time t1, we execute and RPC call half way, aborts, and put > > the > > > > call > > > > > > back to the queue. > > > > > > Then at time t2 another RPC handler picks the call and re-execute > > it. > > > > > > I understand that we might get a different mvcc number and > > different > > > > > > results at t2 compared to we execute it at t1. > > > > > > My question is that: would this situation any different compared > to > > > the > > > > > > situation where the call was never executed at t1, and is > executed > > at > > > > t2 > > > > > > for the first time. > > > > > > > > > > > > > > > > > > My guess is that since at t1 we may already gotten an mvcc > number, > > so > > > > it > > > > > > might potentially cause some write conflicts and certain write > > > > operations > > > > > > to retry. But correctness wise, is there any difference? > > > > > > > > > > > > Thanks a lot! > > > > > > > > > > > > Suli > > > > > > > > > > > > > > > > > > On Sun, Apr 9, 2017 at 5:14 PM, Jerry He <[email protected]> > > wrote: > > > > > > > > > > > > > I don't know what your intention and your context are. > > > > > > > > > > > > > > You may get a different mvcc number and get different results > > next > > > > time > > > > > > > around if there are concurrent writes. > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Jerry > > > > > > > > > > > > > > On Sun, Apr 9, 2017 at 12:48 PM 杨苏立 Yang Su Li < > > [email protected] > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > I am wondering, for read requests like Get/MultiGet/Scan, is > > the > > > > RPC > > > > > > > > handling idempotent in HBase? > > > > > > > > > > > > > > > > More specifically, if in the middle of RPC handling we stop > the > > > > > > handling > > > > > > > > threads, puts the RPC call back to the queue, and later > another > > > RPC > > > > > > > Handler > > > > > > > > picks up this call and starts all over again, will the result > > be > > > > the > > > > > > same > > > > > > > > as if this call is being handled for the first time now? Or > are > > > > their > > > > > > any > > > > > > > > unexpected side effects? > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > Suli > > > > > > > > > > > > > > > > -- > > > > > > > > Suli Yang > > > > > > > > > > > > > > > > Department of Physics > > > > > > > > University of Wisconsin Madison > > > > > > > > > > > > > > > > 4257 Chamberlin Hall > > > > > > > > Madison WI 53703 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Suli Yang > > > > > > > > > > > > Department of Physics > > > > > > University of Wisconsin Madison > > > > > > > > > > > > 4257 Chamberlin Hall > > > > > > Madison WI 53703 > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Suli Yang > > > > > > > > Department of Physics > > > > University of Wisconsin Madison > > > > > > > > 4257 Chamberlin Hall > > > > Madison WI 53703 > > > > > > > > > > -- Suli Yang Department of Physics University of Wisconsin Madison 4257 Chamberlin Hall Madison WI 53703
