[jira] [Commented] (HBASE-7836) Create a new replay command so that recovered edits won't mess up normal coprocessing metrics
[ https://issues.apache.org/jira/browse/HBASE-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13658999#comment-13658999 ] stack commented on HBASE-7836: -- [~jeffreyz] We need this still now hbase-7006 is in? Or this is distinct from it? Create a new replay command so that recovered edits won't mess up normal coprocessing metrics - Key: HBASE-7836 URL: https://issues.apache.org/jira/browse/HBASE-7836 Project: HBase Issue Type: Sub-task Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.95.1 Attachments: hbase-7836-combined.patch, hbase-7836-combined_v2.patch, hbase-7836_v1.patch, hbase-7836_v2.patch, hbase-7837-v3.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7836) Create a new replay command so that recovered edits won't mess up normal coprocessing metrics
[ https://issues.apache.org/jira/browse/HBASE-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13659010#comment-13659010 ] Jeffrey Zhong commented on HBASE-7836: -- No, we don't need it. Thanks. Create a new replay command so that recovered edits won't mess up normal coprocessing metrics - Key: HBASE-7836 URL: https://issues.apache.org/jira/browse/HBASE-7836 Project: HBase Issue Type: Sub-task Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.95.1 Attachments: hbase-7836-combined.patch, hbase-7836-combined_v2.patch, hbase-7836_v1.patch, hbase-7836_v2.patch, hbase-7837-v3.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7836) Create a new replay command so that recovered edits won't mess up normal coprocessing metrics
[
https://issues.apache.org/jira/browse/HBASE-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13636137#comment-13636137
]
Hadoop QA commented on HBASE-7836:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579496/hbase-7837-v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5359//console
This message is automatically generated.
Create a new replay command so that recovered edits won't mess up normal
coprocessing metrics
-
Key: HBASE-7836
URL: https://issues.apache.org/jira/browse/HBASE-7836
Project: HBase
Issue Type: Sub-task
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.95.1
Attachments: hbase-7836-combined.patch, hbase-7836-combined_v2.patch,
hbase-7836_v1.patch, hbase-7836_v2.patch, hbase-7837-v3.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7836) Create a new replay command so that recovered edits won't mess up normal coprocessing metrics
[ https://issues.apache.org/jira/browse/HBASE-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634386#comment-13634386 ] Jimmy Xiang commented on HBASE-7836: I was wondering if we can reuse the replicateWALEntry RPC call. If not, how about call this method replayWALEntry and use WALEntry instead of MultiRequest? We can change the proto types such as WALEdit/WALEntry if needed. Create a new replay command so that recovered edits won't mess up normal coprocessing metrics - Key: HBASE-7836 URL: https://issues.apache.org/jira/browse/HBASE-7836 Project: HBase Issue Type: Sub-task Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.95.1 Attachments: hbase-7836-combined.patch, hbase-7836-combined_v2.patch, hbase-7836_v1.patch, hbase-7836_v2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7836) Create a new replay command so that recovered edits won't mess up normal coprocessing metrics
[ https://issues.apache.org/jira/browse/HBASE-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634423#comment-13634423 ] Jeffrey Zhong commented on HBASE-7836: -- That's a good question(my first version was piggyback on the function and multi). That causes two disadvantages: 1) We need decompose the WAL entries twice one is in the client side to see where edits should go per region(splitting) and the other decomposing is in the receiving server side. Inside, replicateWALEntry implementations, all those WALEntries are also translated to mutations. In addition, inside replicateWALEntry I'll make the replicateWALEntry a little bit 'messy' to handle replay case. 2) For better monitoring priority purpose, a separate command is also better. The cost to create one command is trivial that's why I choose a new replay command(even with piggyback this command or multi, I still need add special attributes) Create a new replay command so that recovered edits won't mess up normal coprocessing metrics - Key: HBASE-7836 URL: https://issues.apache.org/jira/browse/HBASE-7836 Project: HBase Issue Type: Sub-task Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.95.1 Attachments: hbase-7836-combined.patch, hbase-7836-combined_v2.patch, hbase-7836_v1.patch, hbase-7836_v2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7836) Create a new replay command so that recovered edits won't mess up normal coprocessing metrics
[ https://issues.apache.org/jira/browse/HBASE-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13634529#comment-13634529 ] Jimmy Xiang commented on HBASE-7836: I see. That's good. Create a new replay command so that recovered edits won't mess up normal coprocessing metrics - Key: HBASE-7836 URL: https://issues.apache.org/jira/browse/HBASE-7836 Project: HBase Issue Type: Sub-task Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Fix For: 0.95.1 Attachments: hbase-7836-combined.patch, hbase-7836-combined_v2.patch, hbase-7836_v1.patch, hbase-7836_v2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7836) Create a new replay command so that recovered edits won't mess up normal coprocessing metrics
[
https://issues.apache.org/jira/browse/HBASE-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13621885#comment-13621885
]
Hadoop QA commented on HBASE-7836:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12576932/hbase-7836-combined_v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 15 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestShell
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5130//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5130//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5130//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5130//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5130//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5130//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5130//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5130//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5130//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5130//console
This message is automatically generated.
Create a new replay command so that recovered edits won't mess up normal
coprocessing metrics
-
Key: HBASE-7836
URL: https://issues.apache.org/jira/browse/HBASE-7836
Project: HBase
Issue Type: Sub-task
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.95.1
Attachments: hbase-7836-combined.patch, hbase-7836-combined_v2.patch,
hbase-7836_v1.patch, hbase-7836_v2.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7836) Create a new replay command so that recovered edits won't mess up normal coprocessing metrics
[
https://issues.apache.org/jira/browse/HBASE-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13619982#comment-13619982
]
Ted Yu commented on HBASE-7836:
---
Test suite using patch v2 gave two test failures:
{code}
Failed tests:
testFixAssignmentsAndNoHdfsChecking(org.apache.hadoop.hbase.util.TestHBaseFsck):
expected:[NOT_DEPLOYED, HOLE_IN_REGION_CHAIN] but was:[NOT_DEPLOYED,
NOT_DEPLOYED, HOLE_IN_REGION_CHAIN, LAST_REGION_ENDKEY_NOT_EMPTY]
Tests in error:
testLogSplittingAfterMasterRecoveryDueToZKExpiry(org.apache.hadoop.hbase.TestZooKeeper):
Failed after attempts=10, exceptions:(..)
{code}
The second may be related to the patch.
Create a new replay command so that recovered edits won't mess up normal
coprocessing metrics
-
Key: HBASE-7836
URL: https://issues.apache.org/jira/browse/HBASE-7836
Project: HBase
Issue Type: Sub-task
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.95.0
Attachments: hbase-7836_v1.patch, hbase-7836_v2.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7836) Create a new replay command so that recovered edits won't mess up normal coprocessing metrics
[
https://issues.apache.org/jira/browse/HBASE-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13620050#comment-13620050
]
Ted Yu commented on HBASE-7836:
---
{code}
+ * When a regon is in recovering state, it can only accept writes not reads
{code}
typo: regon
For batchMutate(), please add javadoc for param inReplay:
{code}
+ OperationStatus[] batchMutate(
+ PairMutation, Integer[] mutationsAndLocks, boolean inReplay) throws
IOException {
{code}
In javadoc, should state that doPreMutationHook() would not be called if region
is in replay state.
Should the following existing method be extended / merged with the above new
method ?
{code}
public OperationStatus[] batchMutate(
PairMutation, Integer[] mutationsAndLocks) throws IOException {
{code}
In HRegionServer.java,
{code}
-this.recoveringRegions.add(region.getEncodedName());
+this.recoveringRegions.put(region.getEncodedName(), null);
{code}
Why the value of null is used above ?
Create a new replay command so that recovered edits won't mess up normal
coprocessing metrics
-
Key: HBASE-7836
URL: https://issues.apache.org/jira/browse/HBASE-7836
Project: HBase
Issue Type: Sub-task
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.95.0
Attachments: hbase-7836_v1.patch, hbase-7836_v2.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7836) Create a new replay command so that recovered edits won't mess up normal coprocessing metrics
[
https://issues.apache.org/jira/browse/HBASE-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13620286#comment-13620286
]
Ted Yu commented on HBASE-7836:
---
{code}
+ public MultiResponse replay(final RpcController rpcc, final MultiRequest
request)
+ throws ServiceException {
{code}
Should the method name be more informative ? How about replayHLog() or
something similar ?
{code}
* @param mutations
*/
protected void doBatchOp(final MultiResponse.Builder builder, final HRegion
region,
- final ListMutationProto mutations, final CellScanner cells) {
+ final ListMutationProto mutations, final CellScanner cells, boolean
inReplay) {
{code}
Add @param for inReplay.
{code}
-throw new IOException(Found invalid location string: + loc);
+ if(tableName.isEmpty()){
+LOG.warn(Invalid location string: + loc + found.);
}
- HTable table = this.tableNameToHTableMap.get(Bytes.toBytes(tableName));
-
+
+ HConnection hconn = getConnectionByTableName(Bytes.toBytes(tableName));
{code}
When tableName.isEmpty(), why would we go on fetching connection ?
{code}
-} catch (InterruptedException ix) {
- throw new InterruptedIOException(Interrupted when replaying wal
edits.);
+} catch (RuntimeException rx) {
{code}
I think we'd better be specific about the exceptions we catch.
{code}
+ private void replayEdits(final HRegionLocation regionLoc, final HRegionInfo
regionInfo,
+ final ListActionRow actions)
{code}
HRegionLocation contains HRegionInfo field. Why is regionInfo parameter needed
above ?
{code}
+ public MapString, HRegion getRecoveringRegions() {
+// TODO Auto-generated method stub
{code}
You can remove the TODO above.
Create a new replay command so that recovered edits won't mess up normal
coprocessing metrics
-
Key: HBASE-7836
URL: https://issues.apache.org/jira/browse/HBASE-7836
Project: HBase
Issue Type: Sub-task
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.95.0
Attachments: hbase-7836_v1.patch, hbase-7836_v2.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7836) Create a new replay command so that recovered edits won't mess up normal coprocessing metrics
[
https://issues.apache.org/jira/browse/HBASE-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13618543#comment-13618543
]
Ted Yu commented on HBASE-7836:
---
I got the following from running test suite for combined patch (7835 + 7836):
{code}
Failed tests:
testWorkerAbort(org.apache.hadoop.hbase.master.TestDistributedLogSplitting):
none of the following counters went up in 8 milliseconds -
tot_wkr_task_resigned, tot_wkr_task_err, tot_wkr_final_transition_failed,
tot_wkr_task_done, tot_wkr_preempt_task
Tests in error:
testZKClosingNodeVersionMismatch(org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionHandler)
testCloseRegion(org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionHandler)
testFailedUpdateMeta(org.apache.hadoop.hbase.regionserver.handler.TestOpenRegionHandler)
testYankingRegionFromUnderIt(org.apache.hadoop.hbase.regionserver.handler.TestOpenRegionHandler)
{code}
Create a new replay command so that recovered edits won't mess up normal
coprocessing metrics
-
Key: HBASE-7836
URL: https://issues.apache.org/jira/browse/HBASE-7836
Project: HBase
Issue Type: Sub-task
Reporter: Jeffrey Zhong
Assignee: Jeffrey Zhong
Fix For: 0.95.0
Attachments: hbase-7836_v1.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
