[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13631616#comment-13631616 ] Anoop Sam John commented on HBASE-4210: --- bq.You mean postBatchMutate? Yes.. sorry I corrected it now. bq.If it calls postBatchMutate, do we still need postDelete/postPut for each mutation (as is currently done after the mvcc forward move statement). The postPut/Delete will be called post MVCC as in old implementation. You mean we need to remove this? I think we should not. As per the use case either make use of pre/postBatchMutate or pre/postPut/Delete Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.6, 0.95.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.98.0, 0.94.6, 0.95.0 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13631791#comment-13631791 ] Himanshu Vashishtha commented on HBASE-4210: An end user should be aware that there are two places in the flow where Pre/Post hooks are called in this flow. He should write his Observers keeping that in mind. So, this is how it looks now: 1) Pre Puts/Deletes 1.1) Take updateLock.readLock() 2) PreBatchMutate 3) Memstore, Wal, sync, mvcc. 4) PostBatchMutate 4.1) Release updateLock.readLock() 5) Post Puts/Deletes Why do you need the readLock while calling pre/postBatchMutateXXX? Is it necessary? Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.6, 0.95.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.98.0, 0.94.6, 0.95.0 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13631801#comment-13631801 ] Himanshu Vashishtha commented on HBASE-4210: Anoop: I see you guys already discussed above that where do you need to put the hooks for batchMutate. What I am trying to grok is why updateLock.readLock is needed. This lock is a bit expensive one to hold as it used for ops like flushes, and one can call any arbitrary code in the preXXX hook. Hope that clarifies a bit :) Btw, thanks for updating the release note. Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.6, 0.95.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.98.0, 0.94.6, 0.95.0 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13632541#comment-13632541 ] Anoop Sam John commented on HBASE-4210: --- Himanshu The postBatchMutate will be called before the MVCC completion step. We can not release the updateLock.readLock before the MVCC completion. What we make is a way to achieve cross region atomic operation. The postBatch writes to another region, but any reader will either see both region data or none. That was the consideration in placing the hooks. Yes you are correct, the updateLock can block the flush requests. But we can ideally not expecting too time taking ops in these hooks. I mailing list we can see some people try making RPC calls from CP hooks (HTable#put() etc) It is explained in replies that this is some thing should be avoided and Andrew explained why it is an anti pattern. May be we should document this some where? Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.6, 0.95.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.98.0, 0.94.6, 0.95.0 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13630721#comment-13630721 ] Himanshu Vashishtha commented on HBASE-4210: Sorry for coming in late here. Regarding the release note: preBatchMutate hook will be called after write to memstore and append and sync to WAL. You mean postBatchMutate? If it calls postBatchMutate, do we still need postDelete/postPut for each mutation (as is currently done after the mvcc forward move statement). This looks wrong to me. Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.6, 0.95.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.98.0, 0.94.6, 0.95.0 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592063#comment-13592063 ] Anoop Sam John commented on HBASE-4210: --- Yes this is committed to 0.94 and will be available with next release (0.94.6) Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13593153#comment-13593153 ] Hudson commented on HBASE-4210: --- Integrated in HBase-0.94-security-on-Hadoop-23 #12 (See [https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/12/]) HBASE-4210 Allow coprocessor to interact with batches per region sent from a client (Anoop). ADDENDUM patch for fixing line endings (Revision 1450593) HBASE-4210 Allow coprocessor to interact with batches per region sent from a client (Anoop) (Revision 1450465) Result = FAILURE enis : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/MiniBatchOperationInProgress.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestMiniBatchOperationInProgress.java tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/MiniBatchOperationInProgress.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestMiniBatchOperationInProgress.java Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13592035#comment-13592035 ] Asaf Mesika commented on HBASE-4210: Is this going to Hbase 0.94.6? Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13588278#comment-13588278 ] Hudson commented on HBASE-4210: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #422 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/422/]) HBASE-4210 Allow coprocessor to interact with batches per region sent from a client (Anoop). ADDENDUM patch for fixing line endings (Revision 1450591) HBASE-4210 Allow coprocessor to interact with batches per region sent from a client (Anoop) (Revision 1450438) Result = FAILURE enis : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MiniBatchOperationInProgress.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMiniBatchOperationInProgress.java tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MiniBatchOperationInProgress.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMiniBatchOperationInProgress.java Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13588307#comment-13588307 ] Hudson commented on HBASE-4210: --- Integrated in hbase-0.95-on-hadoop2 #2 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/2/]) HBASE-4210 Allow coprocessor to interact with batches per region sent from a client (Anoop). ADDENDUM patch for fixing line endings (Revision 1450592) HBASE-4210 Allow coprocessor to interact with batches per region sent from a client (Anoop) (Revision 1450445) Result = FAILURE enis : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MiniBatchOperationInProgress.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMiniBatchOperationInProgress.java tedyu : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MiniBatchOperationInProgress.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMiniBatchOperationInProgress.java Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13589288#comment-13589288 ] Hudson commented on HBASE-4210: --- Integrated in HBase-0.94-security #113 (See [https://builds.apache.org/job/HBase-0.94-security/113/]) HBASE-4210 Allow coprocessor to interact with batches per region sent from a client (Anoop). ADDENDUM patch for fixing line endings (Revision 1450593) HBASE-4210 Allow coprocessor to interact with batches per region sent from a client (Anoop) (Revision 1450465) Result = FAILURE enis : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/MiniBatchOperationInProgress.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestMiniBatchOperationInProgress.java tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/MiniBatchOperationInProgress.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestMiniBatchOperationInProgress.java Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587407#comment-13587407 ] Andrew Purtell commented on HBASE-4210: --- Trunk patch v3 looks good to me, +1 Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587441#comment-13587441 ] Ted Yu commented on HBASE-4210: --- [~lhofhansl]: Are you Okay with 0.94 patch v5 ? Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587514#comment-13587514 ] Hadoop QA commented on HBASE-4210: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12571037/4210_Trunk-V3.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 9 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestFromClientSide Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4557//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4557//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4557//console This message is automatically generated. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587585#comment-13587585 ] Ted Yu commented on HBASE-4210: --- I ran the failed test with trunk patch and it passed. Integrated to 0.95 and trunk. Thanks for the patch, Anoop. Thanks for the reviews, Ram, Lars and Andy. Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587607#comment-13587607 ] Ted Yu commented on HBASE-4210: --- 0.94 test suite passed with HBASE-4210_94-V5.patch: {code} Tests run: 1275, Failures: 0, Errors: 0, Skipped: 13 [INFO] [INFO] BUILD SUCCESS {code} Integrated to 0.94 as well - I saw Lars' +1 @ 26/Feb/13 00:43 Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587634#comment-13587634 ] Lars Hofhansl commented on HBASE-4210: -- +1 on v5 for 0.94 Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587643#comment-13587643 ] Ted Yu commented on HBASE-4210: --- @Anoop: Mind adding release notes for this issue ? Thanks Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587674#comment-13587674 ] Hudson commented on HBASE-4210: --- Integrated in hbase-0.95 #2 (See [https://builds.apache.org/job/hbase-0.95/2/]) HBASE-4210 Allow coprocessor to interact with batches per region sent from a client (Anoop) (Revision 1450445) Result = FAILURE tedyu : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MiniBatchOperationInProgress.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMiniBatchOperationInProgress.java Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587845#comment-13587845 ] Sergey Shelukhin commented on HBASE-4210: - Do the files modified: hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MiniBatchOperationInProgress.java modified: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMiniBatchOperationInProgress.java have correct line endings? My git is acting weird with those Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587925#comment-13587925 ] Hudson commented on HBASE-4210: --- Integrated in HBase-TRUNK #3901 (See [https://builds.apache.org/job/HBase-TRUNK/3901/]) HBASE-4210 Allow coprocessor to interact with batches per region sent from a client (Anoop) (Revision 1450438) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MiniBatchOperationInProgress.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMiniBatchOperationInProgress.java Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587963#comment-13587963 ] Hudson commented on HBASE-4210: --- Integrated in HBase-0.94 #863 (See [https://builds.apache.org/job/HBase-0.94/863/]) HBASE-4210 Allow coprocessor to interact with batches per region sent from a client (Anoop) (Revision 1450465) Result = SUCCESS tedyu : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/MiniBatchOperationInProgress.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionObserverInterface.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestMiniBatchOperationInProgress.java Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587974#comment-13587974 ] Enis Soztutar commented on HBASE-4210: -- bq. have correct line endings? My git is acting weird with those Those files have CRLF endings. We have to recommit them as an addendum patch with LF endings. Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587978#comment-13587978 ] Ted Yu commented on HBASE-4210: --- Anoop is aware of this and preparing the addendum. Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587982#comment-13587982 ] Enis Soztutar commented on HBASE-4210: -- It was blocking me, so I did it anyway. Thanks Ted, Anoop. Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13587985#comment-13587985 ] Hadoop QA commented on HBASE-4210: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12571132/hbase-4210-addendum.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4569//console This message is automatically generated. Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13588010#comment-13588010 ] Anoop Sam John commented on HBASE-4210: --- Thanks a lot Enis. I had prepared it and about to upload. Sorry for the mistake. Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13588035#comment-13588035 ] Hudson commented on HBASE-4210: --- Integrated in HBase-TRUNK #3902 (See [https://builds.apache.org/job/HBase-TRUNK/3902/]) HBASE-4210 Allow coprocessor to interact with batches per region sent from a client (Anoop). ADDENDUM patch for fixing line endings (Revision 1450591) Result = FAILURE enis : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MiniBatchOperationInProgress.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMiniBatchOperationInProgress.java Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13588044#comment-13588044 ] Hudson commented on HBASE-4210: --- Integrated in HBase-0.94 #864 (See [https://builds.apache.org/job/HBase-0.94/864/]) HBASE-4210 Allow coprocessor to interact with batches per region sent from a client (Anoop). ADDENDUM patch for fixing line endings (Revision 1450593) Result = FAILURE enis : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/MiniBatchOperationInProgress.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestMiniBatchOperationInProgress.java Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13588046#comment-13588046 ] Enis Soztutar commented on HBASE-4210: -- bq. Sorry for the mistake. No big deal chief. Keep the patches coming. Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13588053#comment-13588053 ] Hudson commented on HBASE-4210: --- Integrated in hbase-0.95 #4 (See [https://builds.apache.org/job/hbase-0.95/4/]) HBASE-4210 Allow coprocessor to interact with batches per region sent from a client (Anoop). ADDENDUM patch for fixing line endings (Revision 1450592) Result = FAILURE enis : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MiniBatchOperationInProgress.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMiniBatchOperationInProgress.java Allow coprocessor to interact with batches per region sent from a client Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.95.0, 0.94.6 Reporter: Lars Hofhansl Assignee: Anoop Sam John Fix For: 0.95.0, 0.98.0, 0.94.6 Attachments: 4210_Trunk-V3.patch, HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_94-V5.patch, hbase-4210-addendum.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch, HBASE-4210_Trunk-V3.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586023#comment-13586023 ] Ted Yu commented on HBASE-4210: --- [~apurtell]: What do you think ? Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_Trunk.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586134#comment-13586134 ] Hadoop QA commented on HBASE-4210: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570817/HBASE-4210_Trunk-V2.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 9 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor {color:red}-1 core zombie tests{color}. There are 1 zombie test(s): Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4529//console This message is automatically generated. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586163#comment-13586163 ] Ted Yu commented on HBASE-4210: --- {code} + * @param miniBatchOp batch of Mutations getting applied to region as one batch. {code} 'as one batch' is redundant. {code} + * @param T PairMutation, Integer pair of Mutations and associated rowlocks . {code} 'associated rowlocks' - 'associated rowlock Ids' {code} + public boolean hadPreBatchMutate() { + public boolean hadPostBatchMutate() { {code} The above methods can be package-private, right ? Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586532#comment-13586532 ] Lars Hofhansl commented on HBASE-4210: -- +1 SimpleRegionObserver is a test class, I think having these public is the right approach (i.e. the RegionObserver can be used from different packages). Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586714#comment-13586714 ] Anoop Sam John commented on HBASE-4210: --- Will address Ted's comments about javadoc and give patch tonight(IST) SimpleRegionObserver - Ya IMO also it can be public. [~yuzhih...@gmail.com] fine for you? Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13586717#comment-13586717 ] Ted Yu commented on HBASE-4210: --- Should be good. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_94-V4.patch, HBASE-4210_Trunk.patch, HBASE-4210_Trunk-V2.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585369#comment-13585369 ] Hadoop QA commented on HBASE-4210: -- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12570653/HBASE-4210_Trunk.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 9 new or modified tests. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 3 warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4515//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4515//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4515//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4515//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4515//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4515//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4515//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4515//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4515//console This message is automatically generated. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_Trunk.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13585612#comment-13585612 ] Anoop Sam John commented on HBASE-4210: --- [~ram_krish] Added some unit tests. [~lhofhansl], [~yuzhih...@gmail.com] Is it good to go in now? Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch, HBASE-4210_94-V3.patch, HBASE-4210_Trunk.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13582107#comment-13582107 ] ramkrishna.s.vasudevan commented on HBASE-4210: --- Patch is more simpler. Can you add testcases also? Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13582111#comment-13582111 ] Anoop Sam John commented on HBASE-4210: --- You mean unit test for MiniBatchOperationInProgress? I think I should change the test cases in TestRegionObserverInterface also as per the new hook. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13582120#comment-13582120 ] ramkrishna.s.vasudevan commented on HBASE-4210: --- Testcase that covers the new class also. May be Unit testcase or a testcase with a wider scope. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13582912#comment-13582912 ] Lars Hofhansl commented on HBASE-4210: -- +1 on patch (both 0.94 and 0.96). MiniBatchOperationInProgress is still a bit questionable (slightly overkill. IMHO, just to hide the array offset). Also this won't allow the RegionObserver to interfere with the batch of operations (for example removing one), but I guess that would be tricky anyway. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13582915#comment-13582915 ] Anoop Sam John commented on HBASE-4210: --- Thanks [~lhofhansl] for the review. I will address the comments from Ram and give updated patch.(Adding test cases) bq.Also this won't allow the RegionObserver to interfere with the batch of operations (for example removing one) May be CP can set the status so that region can ignore the Mutation? Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13582920#comment-13582920 ] Lars Hofhansl commented on HBASE-4210: -- bq. May be CP can set the status so that region can ignore the Mutation? Ah yes, that works. Should maybe add a comment to in the RegionObserver interface. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13582921#comment-13582921 ] Anoop Sam John commented on HBASE-4210: --- RegionObserver - At new pre hook and prePut/Delete hooks also we may add javadoc regarding how this Mutation can get ignored by region. I will do that part also in the next version of patch Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch, HBASE-4210_94-V2.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13578560#comment-13578560 ] Andrew Purtell commented on HBASE-4210: --- Not sure about MiniBatchOperationInProgress. You are halfway from starting with an array and ending up with the full list contract, including proper iterators. Why not have getters and setters take an index parameter? Then there would seem no need for next(), rewind(), the IllegalStateExceptions, etc. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13578927#comment-13578927 ] Anoop Sam John commented on HBASE-4210: --- [~apurtell] Thanks for the review and comment. Ya it looks simple for me. Oops why I have not gone with that simplest way! :( So when MiniBatchOperationInProgress contains operation index [5-10) and some one call MiniBatchOperationInProgress#getOperation(1) - we can convert it into BatchOperationInProgress.operations[5+1]. I read your comment correctly? This looks just fine with me. I will update the patch tonight. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13577768#comment-13577768 ] ramkrishna.s.vasudevan commented on HBASE-4210: --- Can we avoid exposing WALedits here? Because we now expose them in WalObserver also. Should next() return boolean and hasNext() be a private implementation? Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13578135#comment-13578135 ] Anoop Sam John commented on HBASE-4210: --- bq.Should next() return boolean and hasNext() be a private implementation? Looks fine. Just like InternalScanner#next() returns hasNext. bq.Can we avoid exposing WALedits here? Because we now expose them in WalObserver also. Why I have done so is, the prePut() hook allows the CP to add/change the WALWdit. We capture the WALEdits from CP and writing. So may be some one might need it? Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13578146#comment-13578146 ] Ted Yu commented on HBASE-4210: --- {code} + +public class MiniBatchOperationInProgressT { {code} Please add javadoc for the class with explanation of type parameter T. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13578151#comment-13578151 ] Anoop Sam John commented on HBASE-4210: --- Thanks [~yuzhih...@gmail.com]. Sure I will do that. This patch is not final. I have to add some more javadocs. The approach looks fine? I am trying to avoid the overheads mentioned in above comments [https://issues.apache.org/jira/browse/HBASE-4210?focusedCommentId=13544586page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13544586] [~lhofhansl], [~apurtell] Your comments? Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Attachments: HBASE-4210_94.patch Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13576917#comment-13576917 ] Anoop Sam John commented on HBASE-4210: --- Work in progress. Will attach initial version tomorrow. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13572052#comment-13572052 ] Anoop Sam John commented on HBASE-4210: --- Got busy with some other works. Will come up with a patch next week. Also this pre hooks for the Puts will be called after the TS for the Puts are populated in HRegion so the hooks can get the actual TS which will go in the Put. Just seen the issue raised by JD HBASE-7774 Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13572142#comment-13572142 ] ramkrishna.s.vasudevan commented on HBASE-4210: --- Yes Anoop. The patch will be very useful for folks to understand what was done. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.6 Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13555475#comment-13555475 ] Andrew Purtell commented on HBASE-4210: --- bq. The main concern is how we can pass the mutations which are handled in a current Mini batch. Yes, I propose you don't do that, and instead inform the CP when a batch is starting, when the batch has completed, and by inclusion of an RPC id in hook upcalls for the individual ops in the batch, what ops below to what batch RPC (by ID). This way there is no expensive construction of a large data structure. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.5 Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1314#comment-1314 ] Lars Hofhansl commented on HBASE-4210: -- Using RPC ID to tie individual hooks together works. I also do like Anoops suggestion to pass an array of operations along with first and last. Specifically that would be batchOp.operations, firstIndex, and lastIndexExclusive. That way nothing needs to be copied and we still have the (semi) intuitive pattern that you get an array of operations along with what subset of is being handled now. The RPC ID idea is cleaner; but it is harder to use by a hook and the preHooks are actually all run before any batch. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.5 Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1370#comment-1370 ] Andrew Purtell commented on HBASE-4210: --- bq. The RPC ID idea is cleaner; but it is harder to use by a hook and the preHooks are actually all run before any batch. Agreed. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.5 Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13555823#comment-13555823 ] Anoop Sam John commented on HBASE-4210: --- I think the CP should know which all Mutations is getting applied each time. The batch of mutations might be running in N number of mini batches. So in every mini batch we need to know which all Mutations getting applied this time. This way only I can be make sure that I add the index for those Puts. This makes index 100% consistent. This one can be achieved with RPC ID alone Andrew? Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.5 Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13555825#comment-13555825 ] Andrew Purtell commented on HBASE-4210: --- bq. This one can be achieved with RPC ID alone Andrew? No. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.5 Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13555827#comment-13555827 ] Anoop Sam John commented on HBASE-4210: --- Andrew - You agree on passing the Mutation(which are getting applied in every MiniBatch) also to CP. Waiting a +1 from CP's boss. :) Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.5 Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13555847#comment-13555847 ] Andrew Purtell commented on HBASE-4210: --- Thanks Anoop but I'm not a boss and we're having a discussion here. Or more accurately you were having a discussion and I jumped in with a thought. Do what you think is best after considering the input, put up a patch, and let's see. :-) Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.5 Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13553680#comment-13553680 ] Anoop Sam John commented on HBASE-4210: --- Andrew - You propose a solution for the issue which me and Lars were discussing? The main concern is how we can pass the mutations which are handled in a current Mini batch. Sorry I didn't get you fully here. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.5 Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13553688#comment-13553688 ] Anoop Sam John commented on HBASE-4210: --- @Lars How about abstracting out a MiniBatchOperationInProgress object and pass to the CP? Underlying actual array will be that in BatchOperationInProgress only. This new object is kind of ByteBuffer position() and limit() way. Am I making sense? Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.5 Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551386#comment-13551386 ] Andrew Purtell commented on HBASE-4210: --- An option here would be to allocate an internal identifier for every operation, corresponding to the user RPC, and then no matter how many units of work that is broken down internally the RPC ID would be carried through. Then there could be a CP hook at step 3 which announces processing on a given RPC (batch). The other hooks will get the ID somehow so the CP can correlate. Then finally at step 8 an upcall notifying that processing the given RPC ID is completed (or errored out). Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.5 Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13543729#comment-13543729 ] Anoop Sam John commented on HBASE-4210: --- @Ram the issues that we found I will try to address in another Jira. That was as a result of we trying to do a cross region transaction kinda thing. Here I mean the row locks dead lock. As the preposed hooks will be called after acquiring the row locks and within the CP implementation one tries to acquire another row lock, there is chance for dead lock. Well we will document this clearly warning the users. :) Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13543732#comment-13543732 ] Anoop Sam John commented on HBASE-4210: --- bq.Todd introduced initialized variable so that we don't call preXXX hooks more than once Yes Ted I am aware of this change that had happened. But all the pre hooks for each of the Mutation will be called at 1st step itself. After that the actual mutate operation might be happening in N batches. We would like to have a pre/post hooks for each of the mini batch mutation.(One call for each mini operation) May be it would be better to have one pre/post hook for the entire batch operation also? - As per our use case we wanted a such a hook also. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13544586#comment-13544586 ] Lars Hofhansl commented on HBASE-4210: -- preHook before STEP 3 and postHook before STEP 8 makes sense. As I mentioned in my comment August 24th, though, is: What would pass to these hooks? Recreating an array of mutations just for the hooks is too expensive, but then we need to pass batchOp or maybe bathOp.operations along with the first and last index... Leaking a lot of HBase internal (which we'll likely want to change around at some point in the future) into the coprocessor code. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.5 Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13544608#comment-13544608 ] Anoop Sam John commented on HBASE-4210: --- Lars yes that is what confusing me too... Passing the index looks very odd As of now in our implementation we have created the a new array and pass the mutation in that batch.. Yes agree with you just for hook recreating the array and dealing with some steps is expensive :( I will make a simple perf test with just passing the existing array with start and end index... Just for testing and will see how much diff it will make.. This is a critical path that is my worry from beginning.. :( Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Fix For: 0.96.0, 0.94.5 Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13541988#comment-13541988 ] ramkrishna.s.vasudevan commented on HBASE-4210: --- Yes. We encountered some dead lock situation wrt to the advancing the mvcc and the order in which the flush was happening. Infact some dataloss also we encountered that time. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13541328#comment-13541328 ] Anoop Sam John commented on HBASE-4210: --- The problems we faced with prePut() during the analysis for the secondary index 1. This is not considering the mini batch nature. Hook for all the Puts called at once at the 1st step itself 2. Not able to know what is the TS which will be used for each of the Put(If no explicit TS in put from client side, then we calculate it at RS at some below lines) Now which will be the suitable location for this new hook...!! I suggest pre hook to be in doMiniBatchMutation() just before STEP 3. Write back to memstore. This place ensures the MVCC consistency. Row lock is already hold at this point so the CP implementation to be careful not making the system into a dead lock situation. Well we can clearly warn this in doc I guess. The post hook can be before STEP 8. Advance mvcc. As per the usecase in secondary index I am giving these suggestions Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13541387#comment-13541387 ] Ted Yu commented on HBASE-4210: --- Todd introduced initialized variable so that we don't call preXXX hooks more than once. Please take this into account when coming up with your patch. w.r.t. potential dead lock situation, did you encounter any when implementing secondary indexing ? Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Assignee: Anoop Sam John Priority: Minor Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13540873#comment-13540873 ] Anoop Sam John commented on HBASE-4210: --- Oh I missed this issue till now... We have implemented some thing like this while working with the secondary index. I will give more details and a patch that we had... We will discuss further based on that then :) More useful CP hooks we can give... Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Priority: Minor Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440570#comment-13440570 ] Lars Hofhansl commented on HBASE-4210: -- An example is to keep a 2ndary index up to date. In that case we definitely want to make use of the batching. (and we cannot collect in the single hooks, since we have no indication about when we're done). Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Priority: Minor Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440593#comment-13440593 ] Andrew Purtell commented on HBASE-4210: --- +1 Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Priority: Minor Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440832#comment-13440832 ] Lars Hofhansl commented on HBASE-4210: -- Then the question: Where to hook this up. Looking at HRegion.doMiniBatchMutation, we'd leak a lot of internals into the coprocessor hooks, which is bad. We also do not want rebuild in array or List or Puts or Deletes again just because there might be a coprocessor hook. I guess we have the following options: # Pass BatchOperationInProgress to these coprocessor hooks (in HRegion.doMiniBatchMutation) # Pass the original set of Mutations to the batch pre hook and the same set and their results (OperationStatus[]) to post hook. This means the post hook will have to do an extra loop now, to filter out the failed mutations) Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Priority: Minor Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13088079#comment-13088079 ] Lars Hofhansl commented on HBASE-4210: -- The motivation is generally the same as for batching on the client. There's cost that can be amortized over a set of KVs. Another reason is that inside a region we can do atomic operations (i.e. we can actually use the rowlocks here), whereas on the client rowlocks are not reliable. I have no specific use cases at this point, though. Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Priority: Minor Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[ https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13086155#comment-13086155 ] Andrew Purtell commented on HBASE-4210: --- @Lars: Can you talk a bit about the use case you are considering here? What is the motivation? Allow coprocessor to interact with batches per region sent from a client(?) --- Key: HBASE-4210 URL: https://issues.apache.org/jira/browse/HBASE-4210 Project: HBase Issue Type: New Feature Reporter: Lars Hofhansl Priority: Minor Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly one row|cell operations. It might be a good idea to allow a coprocessor to deal with batches of puts and deletes as they arrive from the client. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira