Currently ConstraintProcessor latches onto prePut() to perform validation
check.
>From HRegion.doMiniBatchPut() where prePut() is called:
/* Run coprocessor pre hook outside of locks to avoid deadlock */
So to make use of Constraint in Suraj's scenario, we have some decisions to
make about various factors.
Cheers
On Sun, Dec 4, 2011 at 8:39 AM, Suraj Varma <svarma...@gmail.com> wrote:
> Jesse:
> >> Quick soln - write a CP to check the single row (blocking the put).
>
> Yeah - given that I want this to be atomically done, I'm wondering if
> this would even work (because, I believe I'd need to unlock the row so
> that the checkAndMutate can take the lock - so, there is a brief
> window between where there is no lock being held - and some other
> thread could take that lock). One option would be to pass in a lock to
> checkAndMutate ... but that would increase the locking period and may
> have performance implications, I think.
>
> I see a lot of potential in the Constraints implementation - it would
> really open up CAS operations to do functional constraint checking,
> rather than just value comparisons.
>
> --Suraj
>
> On Sun, Dec 4, 2011 at 8:32 AM, Suraj Varma <svarma...@gmail.com> wrote:
> > Thanks - I see that the lock is taken internal to checkAndMutate.
> >
> > I'm wondering whether it is a better idea to actually pass in a
> > Constraint (or even Constraints) as the checkAndMutate argument. Right
> > now it is taking in an Comparator and a CompareOp for verification.
> > But, this could just be a special case of Constraint which is
> > evaluated within the lock.
> >
> > In other words, we could open up a richer Constraint checking api
> > where any "functional" Constraint check can be performed in the
> > checkAndPut operation.
> >
> > This would also not have the same performance impact of taking a
> > rowLock in preCheckAndPut and release in postCheckAndPut. And - it is
> > really (in my mind) implementing the compare-and-set more generically.
> >
> > I also see the potential of passing in multiple constraints (say
> > upper/lower bounds in Increment/Decrement operations) etc.
> >
> > --Suraj
> >
> >
> > On Sat, Dec 3, 2011 at 7:44 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> >> From HRegionServer.checkAndPut():
> >> if (region.getCoprocessorHost() != null) {
> >> Boolean result = region.getCoprocessorHost()
> >> .preCheckAndPut(row, family, qualifier, CompareOp.EQUAL,
> comparator,
> >> put);
> >> ...
> >> boolean result = checkAndMutate(regionName, row, family, qualifier,
> >> CompareOp.EQUAL, new BinaryComparator(value), put,
> >> lock);
> >> We can see that the lock isn't taken for preCheckAndPut().
> >>
> >> To satisfy Suraj's requirement, I think a slight change to
> checkAndPut() is
> >> needed so that atomicity can be achieved across preCheckAndPut() and
> >> checkAndMutate().
> >>
> >> Cheers
> >>
> >> On Sat, Dec 3, 2011 at 4:54 PM, Suraj Varma <svarma...@gmail.com>
> wrote:
> >>
> >>> Just so my question is clear ... everything I'm suggesting is in the
> >>> context of a single row (not cross row / table). - so, yes, I'm
> >>> guessing obtaining a RowLock on the region side during preCheckAndPut
> >>> / postCheckAndPut would certainly work. Which was why I was asking
> >>> whether the pre/postCheckAndPut obtains the row lock or whether the
> >>> row lock is only obtained within checkAndPut.
> >>>
> >>> Let's say the coprocessor takes a rowlock in preCheckAndPut ... will
> >>> that even work? i.e. can the same rowlock be inherited by the
> >>> checkAndPut api within that thread's context? Or will preCheckAndPut
> >>> have to release the lock so that checkAndPut can take it (which won't
> >>> work for my case, as it has to be atomic between the preCheck and
> >>> Put.)
> >>>
> >>> Thanks for pointing me to the Constraints functionality - I'll take a
> >>> look at whether it could potentially work.
> >>> --Suraj
> >>>
> >>> On Sat, Dec 3, 2011 at 10:25 AM, Jesse Yates <jesse.k.ya...@gmail.com>
> >>> wrote:
> >>> > I think the feature you are looking for is a Constraint. Currently
> they
> >>> are
> >>> > being added to 0.94 in
> >>> > HBASE-4605<https://issues.apache.org/jira/browse/HBASE-4605>;
> >>> > they are almost ready to be rolled in, and backporting to 0.92 is
> >>> > definitely doable.
> >>> >
> >>> > However, Constraints aren't going to be quite flexible enough to
> >>> > efficiently support what you are describing. For instance, with a
> >>> > constraint, you are ideally just checking the put value against some
> >>> simple
> >>> > constraint (never over 10 or always an integer), but looking at the
> >>> current
> >>> > state of the table before allowing the put would currently require
> >>> creating
> >>> > a full blown connection to the local table through another HTable.
> >>> >
> >>> > In the short term, you could write a simple coprocessor to do this
> >>> checking
> >>> > and then move over to constraints (which are a simpler, more
> flexible,
> >>> way
> >>> > of doing this) when the necessary features have been added.
> >>> >
> >>> > It is worth discussing if it makes sense to have access to the local
> >>> region
> >>> > through a constraint, though that breaks the idea a little bit, it
> would
> >>> > certainly be useful and not overly wasteful in terms of runtime.
> >>> >
> >>> > Supposing the feature would be added to talk to the local table, and
> >>> since
> >>> > the puts are going to be serialized on the regionserver (at least to
> that
> >>> > single row you are trying to update), you will never get a situation
> >>> where
> >>> > the value added is over the threshold. If you were really worried
> about
> >>> the
> >>> > atomicity of the operation, then when doing the put, first get the
> >>> RowLock,
> >>> > then do the put and release the RowLock. However, that latter method
> is
> >>> > going to be really slow, so should only be used as a stop gap if the
> >>> > constraint doesn't work as expected, until a patch is made for
> >>> constraints.
> >>> >
> >>> > Feel free to open up a ticket and link it to 4605 for adding the
> local
> >>> > table access functionality, and we can discuss the de/merits of
> adding
> >>> the
> >>> > access.
> >>> >
> >>> > -Jesse
> >>> >
> >>> > On Sat, Dec 3, 2011 at 6:24 AM, Suraj Varma <svarma...@gmail.com>
> wrote:
> >>> >
> >>> >> I'm looking at the preCheckAndPut / postCheckAndPut api with
> >>> >> coprocessors and I'm wondering ... are these pre/post checks done
> >>> >> _after_ taking the row lock or is the row lock only done within the
> >>> >> checkAndPut api.
> >>> >>
> >>> >> I'm interested in seeing if we can implement something like:
> >>> >> (in pseudo sql)
> >>> >> update table-name
> >>> >> set column-name = new-value
> >>> >> where (column-value - new-value) > threshold-value
> >>> >>
> >>> >> Basically ... I want to enhance the checkAndPut to not just compare
> >>> >> "values" ... but apply an arbitrary function on the value
> _atomically_
> >>> >> in the Put call. Multiple threads would be firing these mutations
> and
> >>> >> I'd like the threshold-value above to never be breached under any
> >>> >> circumstance.
> >>> >>
> >>> >> Is there a solution that can be implemented either via checkAndPut
> or
> >>> >> using coprocessors preCheckAndPut? If not, would this be a useful
> >>> >> feature to build in HBase?
> >>> >>
> >>> >> Thanks,
> >>> >> --Suraj
> >>> >>
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > -------------------
> >>> > Jesse Yates
> >>> > 240-888-2200
> >>> > @jesse_yates
> >>>
>
