[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13226135#comment-13226135 ] Hudson commented on HBASE-5541: --- Integrated in HBase-TRUNK #2675 (See [https://builds.apache.org/job/HBase-TRUNK/2675/]) HBASE-5541 Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks (Revision 1298678) Result = SUCCESS larsh : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 5541-v2.txt, 5541.txt Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the lock is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225407#comment-13225407 ] Zhihong Yu commented on HBASE-5541: --- bq. after the log is released I guess you meant 'after the lock is released' Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the log is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[
https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225492#comment-13225492
]
Zhihong Yu commented on HBASE-5541:
---
There're two step 11 now:
{code}
+if (acquiredLocks != null) {
+ // 11. release the row lock
{code}
Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
Key: HBASE-5541
URL: https://issues.apache.org/jira/browse/HBASE-5541
Project: HBase
Issue Type: Sub-task
Components: client, regionserver
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.94.0
Attachments: 5541.txt
Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed.
Similar to what we do in doMiniBatchPut, we should create the log entry with
the lock held, but only sync the HLog after the lock is released, along with
rollback logic in case the sync'ing fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225503#comment-13225503 ] Lars Hofhansl commented on HBASE-5541: -- Whoops. Yes, that comment should've been removed. Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 5541.txt Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the lock is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225660#comment-13225660 ] Lars Hofhansl commented on HBASE-5541: -- Looks good to me. Any objections to committing this? Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 5541.txt Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the lock is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225661#comment-13225661 ] Zhihong Yu commented on HBASE-5541: --- Redundant step 11 comment would be removed, right ? Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 5541.txt Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the lock is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225667#comment-13225667 ] Lars Hofhansl commented on HBASE-5541: -- Yep :) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 5541.txt Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the lock is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225725#comment-13225725 ] Lars Hofhansl commented on HBASE-5541: -- I'll wait for an official +1. Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 5541.txt Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the lock is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225729#comment-13225729 ] Zhihong Yu commented on HBASE-5541: --- +1 from me. Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 5541-v2.txt, 5541.txt Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the lock is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225772#comment-13225772 ] Hadoop QA commented on HBASE-5541: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517655/5541-v2.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -123 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 159 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestImportTsv Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1142//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1142//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1142//console This message is automatically generated. Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 5541-v2.txt, 5541.txt Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the lock is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225775#comment-13225775 ] Hudson commented on HBASE-5541: --- Integrated in HBase-0.94 #22 (See [https://builds.apache.org/job/HBase-0.94/22/]) HBASE-5541 Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks (Revision 1298680) Result = SUCCESS larsh : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 5541-v2.txt, 5541.txt Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the lock is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225849#comment-13225849 ] Lars Hofhansl commented on HBASE-5541: -- @Stack: you OK with the explanation above? Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 5541-v2.txt, 5541.txt Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the lock is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225863#comment-13225863 ] Hudson commented on HBASE-5541: --- Integrated in HBase-TRUNK-security #132 (See [https://builds.apache.org/job/HBase-TRUNK-security/132/]) HBASE-5541 Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks (Revision 1298678) Result = FAILURE larsh : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 5541-v2.txt, 5541.txt Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the lock is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225882#comment-13225882 ] stack commented on HBASE-5541: -- Commit is fine. What about case where the edits are applied, sync succeeds, but the cp postdelete and postput fail? Then mvcc will have been updated but we have removed the edits from memstore. i suppose its ok? The read point is advanced but if we take this edit in isolation, no mutations made it through? Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 5541-v2.txt, 5541.txt Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the lock is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225898#comment-13225898 ] Lars Hofhansl commented on HBASE-5541: -- When we get to the CP post hooks all locks are released, the WAL is synched, and the mvcc is rolled forward. No edit will be removed from the memstore (that only happens when the wal sync fails). What that means is that a post hook cannot undo the operation (the same as standalone Put and Delete). I spent some time trying to poke holes in the design, I think it is tight. Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 5541-v2.txt, 5541.txt Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the lock is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225900#comment-13225900 ] stack commented on HBASE-5541: -- Grand Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 5541-v2.txt, 5541.txt Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the lock is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5541) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks
[ https://issues.apache.org/jira/browse/HBASE-5541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13225905#comment-13225905 ] Lars Hofhansl commented on HBASE-5541: -- That is not to say that there are no holes. :) Avoid holding the rowlock during HLog sync in HRegion.mutateRowWithLocks Key: HBASE-5541 URL: https://issues.apache.org/jira/browse/HBASE-5541 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Lars Hofhansl Assignee: Lars Hofhansl Fix For: 0.94.0 Attachments: 5541-v2.txt, 5541.txt Currently mutateRowsWithLocks holds the row lock while the HLog is sync'ed. Similar to what we do in doMiniBatchPut, we should create the log entry with the lock held, but only sync the HLog after the lock is released, along with rollback logic in case the sync'ing fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
