This optimization is feasible. Cheers
On Nov 8, 2013, at 4:53 AM, James Lee <[email protected]> wrote: > Thanks for letting us know the good feature. > > Maybe I can find the way to optimize Haeinsa transaction which mutations > are in single HBase region with MultiRowMutationEndpoint. > > > On Thu, Nov 7, 2013 at 11:11 AM, Ted Yu <[email protected]> wrote: > >> Thanks for the update. >> >> Have you taken a look at the following endpoint in 0.94 ? >> >> src/main/java/org/apache/hadoop/hbase/coprocessor/MultiRowMutationEndpoint.java >> >> >> On Wed, Nov 6, 2013 at 5:40 PM, James Lee <[email protected]> wrote: >> >>> That's for optimization of performance in Haeinsa. >>> We can use fewer HBase operation in single row transaction. >>> It is related to algorithm design of Haeinsa. >>> >>> For example, consider about following raw-HBase code. (2get/2put to 2 >> rows) >>> >>> get(row1, col1) >>> get(row2, col2) >>> put(row1, col1, value1) >>> put(row2, col2, value2) >>> >>> If you use Haeinsa, this will be executed by following order. >>> (2get/5checkAndPut) >>> >>> get(row1, col1) >>> get(row2, col2) >>> // commit() here >>> checkAndPut(row1, prevLock1, {col1:value1}) >>> checkAndPut(row2, prevLock2, {col2:value2}) >>> checkAndPut(row1, prevLock1, COMMITED) >>> checkAndPut(row2, prevLock2, STABLE) >>> checkAndPut(row1, prevLock1, STABLE) >>> >>> Now, consider single row transaction. (2get/2put to single row) >>> >>> get(row1, col1) >>> get(row1, col2) >>> put(row1, col1, value1) >>> put(row1, col2, value2) >>> >>> If you use Haeinsa, this will be executed by following order. >>> (2get/1checkAndPut) >>> >>> get(row1, col1) >>> get(row1, col2) >>> // commit() here >>> checkAndPut(row1, prevLock1, {col1:value1, col2:value2, lock:STABLE}) >>> >>> Single row transaction needs much fewer operation to achieve ACID >>> properties in Haeinsa >>> >>> Thanks. >>> >>> >>> On Thu, Nov 7, 2013 at 7:40 AM, Ted Yu <[email protected]> wrote: >>> >>>> Jung-Haeng: >>>> In HaeinsaTransaction.java : >>>> >>>> /** >>>> * If there is only one rowTx and type of its mutation is >>>> HaeinsaPut. >>>> */ >>>> SINGLE_ROW_PUT_ONLY, >>>> >>>> Can you elaborate a bit on why the single row put gets special >> treatment >>> ? >>>> >>>> Cheers >>>> >>>> >>>> On Fri, Oct 11, 2013 at 6:09 PM, Ted Yu <[email protected]> wrote: >>>> >>>>> I am now able to build on Linux. >>>>> >>>>> Running test suite. >>>>> >>>>> Will look deeper into the code over the weekend. >>>>> >>>>> >>>>> On Thu, Oct 10, 2013 at 10:16 PM, Jung-Haeng Lee <[email protected] >>>>> wrote: >>>>> >>>>>> Thanks for your advice. >>>>>> I fixed to Hbase 0.94.3 on pom.xml. >>>>>> (To ensure consistency, HBASE-7051 should be solved.) >>>>>> >>>>>> I also fixed pom.xml of Haeinsa: >>>>>> Now maven-thrift-plugin is activate on linux using default profile. >>>>>> Please ensure the "thrift" executable is in your PATH. >>>>>> >>>>>> And you should build Haeinsa with jdk 1.7. >>>>>> I think System.lineSeparator() is new method of Java7. >>>>>> >>>>>> - James Lee >>>>>> >>>>>> >>>>>> On Fri, Oct 11, 2013 at 3:18 AM, Ted Yu <[email protected]> >> wrote: >>>>>> >>>>>>> Using the 'mac' profile, I got the following: >>>>>>> >>>>>>> Number of foreign imports: 1 >>>>>>> import: Entry[import from realm ClassRealm[maven.api, parent: >>> null]] >>>>>>> >>>>>>> ----------------------------------------------------- >>>>>>> >>>>>>> at >> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137) >>>>>>> ... 20 more >>>>>>> Caused by: java.lang.NoSuchMethodError: >>>>>>> java.lang.System.lineSeparator()Ljava/lang/String; >>>>>>> at >>>> io.netty.build.checkstyle.NewlineCheck.<clinit>(NewlineCheck.java:43) >>>>>>> at java.lang.Class.forName0(Native Method) >>>>>>> at java.lang.Class.forName(Class.java:249) >>>>>>> at >> com.puppycrawl.tools.checkstyle.PackageObjectFactory.createObject(PackageObjectFactory.java:111) >>>>>>> at >> com.puppycrawl.tools.checkstyle.PackageObjectFactory.doMakeObject(PackageObjectFactory.java:80) >>>>>>> at >> com.puppycrawl.tools.checkstyle.PackageObjectFactory.createModule(PackageObjectFactory.java:147) >>>>>>> at >>>> com.puppycrawl.tools.checkstyle.Checker.setupChild(Checker.java:153) >>>>>>> at >> com.puppycrawl.tools.checkstyle.api.AutomaticBean.configure(AutomaticBean.java:184) >>>>>>> at >> org.apache.maven.plugin.checkstyle.DefaultCheckstyleExecutor.executeCheckstyle(DefaultCheckstyleExecutor.java:172) >>>>>>> at >> org.apache.maven.plugin.checkstyle.CheckstyleViolationCheckMojo.execute(CheckstyleViolationCheckMojo.java:365) >>>>>>> at >> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101) >>>>>>> ... 20 more >>>>>>> >>>>>>> >>>>>>> On Thu, Oct 10, 2013 at 11:06 AM, Ted Yu <[email protected]> >>> wrote: >>>>>>> >>>>>>>> Looks like HBase 0.92.1 is used. How about upgrading to 0.94.12 >> ? >>>>>>>> >>>>>>>> I cloned the git repo. But I got some compilation errors: >>>>>>>> >>>>>>>> [ERROR] >> /grid/0/dev/ty/haeinsa/src/main/java/kr/co/vcnc/haeinsa/HaeinsaTransaction.java:[324,20] >>>>>>>> cannot find symbol >>>>>>>> [ERROR] symbol: class TRowKey >>>>>>>> [ERROR] location: class kr.co.vcnc.haeinsa.HaeinsaTransaction >>>>>>>> [ERROR] >> /grid/0/dev/ty/haeinsa/src/main/java/kr/co/vcnc/haeinsa/HaeinsaTransaction.java:[325,13] >>>>>>>> cannot find symbol >>>>>>>> [ERROR] symbol: class TRowKey >>>>>>>> [ERROR] location: class kr.co.vcnc.haeinsa.HaeinsaTransaction >>>>>>>> >>>>>>>> How do I activate maven-thrift-plugin on Linux ? >>>>>>>> >>>>>>>> Thanks >>>>>>>> >>>>>>>> On Thu, Oct 10, 2013 at 9:41 AM, Michael Segel < >>>>>>> [email protected]>wrote: >>>>>>>> >>>>>>>>> Which level of isolation do you support? >>>>>>>>> >>>>>>>>> >>>>>>>>> On Oct 10, 2013, at 12:55 AM, Myungbo Kim <[email protected]> >>>> wrote: >>>>>>>>> >>>>>>>>>> Hello everyone in the mailing list. >>>>>>>>>> >>>>>>>>>> I want to introduce an open-source library Haeinsa which >>> supports >>>>>>>>> multi-row, multi-table transaction on HBase. >>>>>>>>>> Haeinsa is client-only library that support serializability >> and >>>>>> linear >>>>>>>>> scalability. >>>>>>>>>> Here is the github repository and presentation that describes >>>>>>> mechanism >>>>>>>>> of it. >>>>>>>>>> >>>>>>>>>> github : https://github.com/VCNC/haeinsa >>>>>>>>>> presentation : https://speakerdeck.com/vcnc/haeinsa-overview >>>>>>>>>> >>>>>>>>>> It was inspired by Google's percolator, but implementation >>> detail >>>>>> is >>>>>>>>> different. >>>>>>>>>> It use two-phase commit protocol and optimistic concurrency >>>>>> control to >>>>>>>>> implement, and Haeinsa now processes more than 300M+ >> transactions >>>> per >>>>>>> day >>>>>>>>> in single cluster without any consistency problem for more >> than 2 >>>>>> month. >>>>>>>>>> I know that there has been lots of libraries and papers for >>> HBase >>>>>>>>> transaction, but within my knowledge, this is the only >>> open-source >>>>>>> library >>>>>>>>> which support serializability and linear scalability. >>>>>>>>>> There is no theoretical limit of transaction throughput. >>>>>>>>>> It was tested against cluster on AWS until 40,000 >>>> transaction/sec. >>>>>>>>> (Still testing on bigger cluster) >>>>>>>>>> If you find it interesting, please leave me comment. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Andrew Kim >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *이 정 행 / Jung-Haeng Lee* >>>>>> * >>>>>> * >>>>>> Blog: http://eincs.net >>>>>> Facebook: http://www.facebook.com/eincs >>>>>> Twitter: http://twitter.com/eincs >>>>>> LinkedIn: http://linkedin.com/in/eincs >>> >>> >>> >>> -- >>> *이 정 행 / Jung-Haeng Lee* >>> >>> Blog: http://eincs.net >>> Facebook: http://www.facebook.com/eincs >>> Twitter: http://twitter.com/eincs >>> LinkedIn: http://linkedin.com/in/eincs > > > > -- > *이 정 행 / Jung-Haeng Lee* > > Blog: http://eincs.net > Facebook: http://www.facebook.com/eincs > Twitter: http://twitter.com/eincs > LinkedIn: http://linkedin.com/in/eincs
