[jira] [Commented] (HBASE-5839) Backup master not starting up due to Bind Exception while starting HttpServer
[ https://issues.apache.org/jira/browse/HBASE-5839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258044#comment-13258044 ] Uma Maheswara Rao G commented on HBASE-5839: bq. Even before that the Data Xceviers threads (IPC handlers) are started and they are started at random port. DataXceiver server will listen on fixed port, 50010. Clients only will bind to random ports for short term. Even I have seen this situations some times in my clusters. We followed to set the fixed ports of servers at range of ~65000. It may just reduce the probability of clients to binding on the same ports. As per our observations, that random ports did not reach to this level. Not sure, is there any specific range for random ports. This problem may be unavoidable problem. Restart servers only the option. Let's see if anyone has some better idea to avoid this situation. Backup master not starting up due to Bind Exception while starting HttpServer - Key: HBASE-5839 URL: https://issues.apache.org/jira/browse/HBASE-5839 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Fix For: 0.96.0, 0.94.1 Backup master tries to bind to the info port 60010. This is done once the back up master becomes active. Even before that the Data Xceviers threads (IPC handlers) are started and they are started at random port. If already 60010 is used then when standby master comes up then it fails due to bind exception. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5831) hadoopqa builds not completing
[
https://issues.apache.org/jira/browse/HBASE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-5831:
-
Status: Patch Available (was: Open)
hadoopqa builds not completing
--
Key: HBASE-5831
URL: https://issues.apache.org/jira/browse/HBASE-5831
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Priority: Blocker
Attachments: 5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.all.mapreduce.txt, 5831.remove.all.mapreduce.txt,
5831.remove.all.mapreduce.txt, 5831.remove.all.mapreduce.txt
No test failures but build complains it has failed. trunk build seems to
have the same affliction:
{code}
Results :
Tests run: 909, Failures: 0, Errors: 0, Skipped: 9
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 41:19.273s
[INFO] Finished at: Wed Apr 18 21:54:31 UTC 2012
[INFO] Final Memory: 59M/451M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.12-TRUNK-HBASE-2:test
(secondPartTestsExecution) on project hbase: Failure or timeout - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12523250/5811+%281%29.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 6 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number
of release audit warnings.
-1 core tests. The patch failed these unit tests:
{code}
Its not apparent that any particular test is not finishing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5831) hadoopqa builds not completing
[
https://issues.apache.org/jira/browse/HBASE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-5831:
-
Attachment: 5831.remove.all.mapreduce.txt
Retry. See if can get clean build again w/ all tests passing. Last test had a
fail.
hadoopqa builds not completing
--
Key: HBASE-5831
URL: https://issues.apache.org/jira/browse/HBASE-5831
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Priority: Blocker
Attachments: 5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.all.mapreduce.txt, 5831.remove.all.mapreduce.txt,
5831.remove.all.mapreduce.txt, 5831.remove.all.mapreduce.txt
No test failures but build complains it has failed. trunk build seems to
have the same affliction:
{code}
Results :
Tests run: 909, Failures: 0, Errors: 0, Skipped: 9
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 41:19.273s
[INFO] Finished at: Wed Apr 18 21:54:31 UTC 2012
[INFO] Final Memory: 59M/451M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.12-TRUNK-HBASE-2:test
(secondPartTestsExecution) on project hbase: Failure or timeout - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12523250/5811+%281%29.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 6 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number
of release audit warnings.
-1 core tests. The patch failed these unit tests:
{code}
Its not apparent that any particular test is not finishing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5831) hadoopqa builds not completing
[
https://issues.apache.org/jira/browse/HBASE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-5831:
-
Status: Open (was: Patch Available)
hadoopqa builds not completing
--
Key: HBASE-5831
URL: https://issues.apache.org/jira/browse/HBASE-5831
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Priority: Blocker
Attachments: 5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.all.mapreduce.txt, 5831.remove.all.mapreduce.txt,
5831.remove.all.mapreduce.txt, 5831.remove.all.mapreduce.txt
No test failures but build complains it has failed. trunk build seems to
have the same affliction:
{code}
Results :
Tests run: 909, Failures: 0, Errors: 0, Skipped: 9
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 41:19.273s
[INFO] Finished at: Wed Apr 18 21:54:31 UTC 2012
[INFO] Final Memory: 59M/451M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.12-TRUNK-HBASE-2:test
(secondPartTestsExecution) on project hbase: Failure or timeout - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12523250/5811+%281%29.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 6 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number
of release audit warnings.
-1 core tests. The patch failed these unit tests:
{code}
Its not apparent that any particular test is not finishing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5840) Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status
Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status -- Key: HBASE-5840 URL: https://issues.apache.org/jira/browse/HBASE-5840 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.0 Reporter: Gopinathan A Fix For: 0.94.1 TaskMonitor Status will not be cleared in case Regions FAILED_OPEN. This will keeps showing old status. This will miss leads the user. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5840) Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status
[ https://issues.apache.org/jira/browse/HBASE-5840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-5840: -- Fix Version/s: 0.96.0 Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status -- Key: HBASE-5840 URL: https://issues.apache.org/jira/browse/HBASE-5840 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.0 Reporter: Gopinathan A Fix For: 0.96.0, 0.94.1 TaskMonitor Status will not be cleared in case Regions FAILED_OPEN. This will keeps showing old status. This will miss leads the user. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5840) Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status
[ https://issues.apache.org/jira/browse/HBASE-5840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan reassigned HBASE-5840: - Assignee: ramkrishna.s.vasudevan Open Region FAILED_OPEN doesn't clear the TaskMonitor Status, keeps showing the old status -- Key: HBASE-5840 URL: https://issues.apache.org/jira/browse/HBASE-5840 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.0 Reporter: Gopinathan A Assignee: ramkrishna.s.vasudevan Fix For: 0.96.0, 0.94.1 TaskMonitor Status will not be cleared in case Regions FAILED_OPEN. This will keeps showing old status. This will miss leads the user. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5831) hadoopqa builds not completing
[
https://issues.apache.org/jira/browse/HBASE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258066#comment-13258066
]
Hadoop QA commented on HBASE-5831:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12523458/5831.remove.all.mapreduce.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 26 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestFromClientSide
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1591//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1591//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1591//console
This message is automatically generated.
hadoopqa builds not completing
--
Key: HBASE-5831
URL: https://issues.apache.org/jira/browse/HBASE-5831
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Priority: Blocker
Attachments: 5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.all.mapreduce.txt, 5831.remove.all.mapreduce.txt,
5831.remove.all.mapreduce.txt, 5831.remove.all.mapreduce.txt
No test failures but build complains it has failed. trunk build seems to
have the same affliction:
{code}
Results :
Tests run: 909, Failures: 0, Errors: 0, Skipped: 9
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 41:19.273s
[INFO] Finished at: Wed Apr 18 21:54:31 UTC 2012
[INFO] Final Memory: 59M/451M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.12-TRUNK-HBASE-2:test
(secondPartTestsExecution) on project hbase: Failure or timeout - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12523250/5811+%281%29.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 6 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number
of release audit warnings.
-1 core tests. The patch failed these unit tests:
{code}
Its not apparent that any particular test is not finishing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5841) hbase shell translate_hbase_exceptions() rely on table name as first argument
hbase shell translate_hbase_exceptions() rely on table name as first argument
-
Key: HBASE-5841
URL: https://issues.apache.org/jira/browse/HBASE-5841
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 0.92.1, 0.94.0, 0.96.0
Reporter: Matteo Bertozzi
shell/commands.rb translate_hbase_exceptions() rely on the fact that the table
name is the first argument.
This is true for many of the commands but for example:
- grant(user, rights, table_name, family=nil, qualifier=nil
- revoke(user, table_name, family=nil, qualifier=nil)
has user as first argument, so if you specify a table that doesn't exists, or
where you don't have access you end up with a message like Unknown table
{username} and so on...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5829) Inconsistency between the regions map and the servers map in AssignmentManager
[
https://issues.apache.org/jira/browse/HBASE-5829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258074#comment-13258074
]
Maryann Xue commented on HBASE-5829:
In AssignmentManager.unassign(HRegionInfo, boolean)
// Remove from the regionsMap
synchronized (this.regions) {
this.regions.remove(region);
}
In AssignmentManager.assign(HRegionInfo, RegionState, boolean, boolean, boolean)
synchronized (this.regions) {
this.regions.put(plan.getRegionInfo(), plan.getDestination());
}
Here, not updating/removing the region from this.servers might cause the
balancer to generate incorrect region plans.
After the fix of HBASE-5563, it seems this problem won't cause endless loop of
wrong balances or a region always in transition.
Inconsistency between the regions map and the servers map in
AssignmentManager
--
Key: HBASE-5829
URL: https://issues.apache.org/jira/browse/HBASE-5829
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.90.6, 0.92.1
Reporter: Maryann Xue
There are occurrences in AM where this.servers is not kept consistent with
this.regions. This might cause balancer to offline a region from the RS that
already returned NotServingRegionException at a previous offline attempt.
In AssignmentManager.unassign(HRegionInfo, boolean)
try {
// TODO: We should consider making this look more like it does for the
// region open where we catch all throwables and never abort
if (serverManager.sendRegionClose(server, state.getRegion(),
versionOfClosingNode)) {
LOG.debug(Sent CLOSE to + server + for region +
region.getRegionNameAsString());
return;
}
// This never happens. Currently regionserver close always return true.
LOG.warn(Server + server + region CLOSE RPC returned false for +
region.getRegionNameAsString());
} catch (NotServingRegionException nsre) {
LOG.info(Server + server + returned + nsre + for +
region.getRegionNameAsString());
// Presume that master has stale data. Presume remote side just split.
// Presume that the split message when it comes in will fix up the
master's
// in memory cluster state.
} catch (Throwable t) {
if (t instanceof RemoteException) {
t = ((RemoteException)t).unwrapRemoteException();
if (t instanceof NotServingRegionException) {
if (checkIfRegionBelongsToDisabling(region)) {
// Remove from the regionsinTransition map
LOG.info(While trying to recover the table
+ region.getTableNameAsString()
+ to DISABLED state the region + region
+ was offlined but the table was in DISABLING state);
synchronized (this.regionsInTransition) {
this.regionsInTransition.remove(region.getEncodedName());
}
// Remove from the regionsMap
synchronized (this.regions) {
this.regions.remove(region);
}
deleteClosingOrClosedNode(region);
}
}
// RS is already processing this region, only need to update the
timestamp
if (t instanceof RegionAlreadyInTransitionException) {
LOG.debug(update + state + the timestamp.);
state.update(state.getState());
}
}
In AssignmentManager.assign(HRegionInfo, RegionState, boolean, boolean,
boolean)
synchronized (this.regions) {
this.regions.put(plan.getRegionInfo(), plan.getDestination());
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5842) Passing shell commands as an argument
Passing shell commands as an argument
-
Key: HBASE-5842
URL: https://issues.apache.org/jira/browse/HBASE-5842
Project: HBase
Issue Type: Improvement
Components: shell
Affects Versions: 0.94.0
Reporter: Harsh J
Priority: Minor
Many times we've required scans of .META. to analyze issues with the cluster we
work on, and to have the result in a file we can pass around we usually end up
doing something like:
{{echo scan '.META.'| hbase shell meta-scan.txt}}
This can rather be simplified as something like the following instead, with
support for a commands reading argument:
{{hbase shell -c scan '.META.'}}
[Note though: File reading is possible already, i.e. {{hbase shell file.hs}},
but then thats two steps and we usually don't keep a file around for just a
meta table scan.]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5843) Improve HBase MTTR - Mean Time To Recover
Improve HBase MTTR - Mean Time To Recover - Key: HBASE-5843 URL: https://issues.apache.org/jira/browse/HBASE-5843 Project: HBase Issue Type: Umbrella Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal A part of the approach is described here: https://docs.google.com/document/d/1z03xRoZrIJmg7jsWuyKYl6zNournF_7ZHzdi0qz_B4c/edit The ideal target is: - failure impact client applications only by an added delay to execute a query, whatever the failure. - this delay is always inferior to 1 second. We're not going to achieve that immediately... Priority will be given to the most frequent issues. Short term: - software crash - standard administrative tasks as stop/start of a cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5844) Delete the region servers znode after a regions server crash
Delete the region servers znode after a regions server crash Key: HBASE-5844 URL: https://issues.apache.org/jira/browse/HBASE-5844 Project: HBase Issue Type: Improvement Components: regionserver, scripts Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal today, if the regions server crashes, its znode is not deleted in ZooKeeper. So the recovery process will stop only after a timeout, usually 30s. By deleting the znode in start script, we remove this delay and the recovery starts immediately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5844) Delete the region servers znode after a regions server crash
[ https://issues.apache.org/jira/browse/HBASE-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5844: --- Attachment: 5844.v1.patch Delete the region servers znode after a regions server crash Key: HBASE-5844 URL: https://issues.apache.org/jira/browse/HBASE-5844 Project: HBase Issue Type: Improvement Components: regionserver, scripts Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Attachments: 5844.v1.patch today, if the regions server crashes, its znode is not deleted in ZooKeeper. So the recovery process will stop only after a timeout, usually 30s. By deleting the znode in start script, we remove this delay and the recovery starts immediately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5844) Delete the region servers znode after a regions server crash
[ https://issues.apache.org/jira/browse/HBASE-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258137#comment-13258137 ] nkeywal commented on HBASE-5844: Patch v1. Tested on a local cluster pseudo distributed, by stopping the server by kill -9. I will do some minor improvements on the java code and then test on a real cluster, but I'm interested by a feedback on the script. Delete the region servers znode after a regions server crash Key: HBASE-5844 URL: https://issues.apache.org/jira/browse/HBASE-5844 Project: HBase Issue Type: Improvement Components: regionserver, scripts Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Attachments: 5844.v1.patch today, if the regions server crashes, its znode is not deleted in ZooKeeper. So the recovery process will stop only after a timeout, usually 30s. By deleting the znode in start script, we remove this delay and the recovery starts immediately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5677) The master never does balance because duplicate openhandled the one region
[ https://issues.apache.org/jira/browse/HBASE-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258171#comment-13258171 ] xufeng commented on HBASE-5677: --- @stack Yes,we should close this issue. I will create a new issue to backport HBASE-5454 to 0.90,0.92.2 version. And submit the patch that the checkinitialized method in createTable for trunk and 0.94 version. The master never does balance because duplicate openhandled the one region -- Key: HBASE-5677 URL: https://issues.apache.org/jira/browse/HBASE-5677 Project: HBase Issue Type: Bug Affects Versions: 0.90.6 Environment: 0.90 Reporter: xufeng Assignee: xufeng Fix For: 0.90.7, 0.92.2 Attachments: 5677-proposal.txt, 5677-proposal.txt, Backport-HBASE-5454-to-90.patch, Backport-HBASE-5454-to-92.patch, HBASE-5677-90-v1.patch, surefire-report_no_patched_v1.html, surefire-report_patched_v1.html If region be assigned When the master is doing initialization(before do processFailover),the region will be duplicate openhandled. because the unassigned node in zookeeper will be handled again in AssignmentManager#processFailover() it cause the region in RIT,thus the master never does balance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2489) Make the Filesystem needs to be upgraded error message more useful.
[
https://issues.apache.org/jira/browse/HBASE-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258188#comment-13258188
]
chenning commented on HBASE-2489:
-
Hi All, I met this problem in my environment. Fortunately, I found this has
been fixed here, as a freshman, I don't know how to apply this fix into my
environment since I only see a patch file which can't be applied to my
environment, any suggestion? Thank you.
Make the Filesystem needs to be upgraded error message more useful.
-
Key: HBASE-2489
URL: https://issues.apache.org/jira/browse/HBASE-2489
Project: HBase
Issue Type: Improvement
Components: util
Reporter: Benoit Sigoure
Assignee: Benoit Sigoure
Priority: Trivial
Fix For: 0.90.0
Attachments:
0001-Improve-the-error-message-File-system-needs-to-be-up.patch
The other day, when starting HBase I got this error:
{noformat}
2010-04-23 09:38:14,847 ERROR org.apache.hadoop.hbase.master.HMaster: Failed
to start master
org.apache.hadoop.hbase.util.FileSystemVersionException: File system needs to
be upgraded. Run the '${HBASE_HOME}/bin/hbase migrate' script.
at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:187)
{noformat}
I was puzzled until I realized, after adding extra debug statements in the
code, that I forgot to properly set {{hbase.rootdir}} after re-deploying my
dev environment. I think the message above was misleading and I'm proposing
a trivial patch to make it a little bit better:
{noformat}
2010-04-23 09:48:29,000 ERROR org.apache.hadoop.hbase.master.HMaster: Failed
to start master
org.apache.hadoop.hbase.util.FileSystemVersionException: File system needs to
be upgraded. You have version null and I want version 7. Run the
'${HBASE_HOME}/bin/hbase migrate' script.
at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:189)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2489) Make the Filesystem needs to be upgraded error message more useful.
[
https://issues.apache.org/jira/browse/HBASE-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258199#comment-13258199
]
chenning commented on HBASE-2489:
-
Forget to ask, is there any way to work this problem around? Thank you.
Make the Filesystem needs to be upgraded error message more useful.
-
Key: HBASE-2489
URL: https://issues.apache.org/jira/browse/HBASE-2489
Project: HBase
Issue Type: Improvement
Components: util
Reporter: Benoit Sigoure
Assignee: Benoit Sigoure
Priority: Trivial
Fix For: 0.90.0
Attachments:
0001-Improve-the-error-message-File-system-needs-to-be-up.patch
The other day, when starting HBase I got this error:
{noformat}
2010-04-23 09:38:14,847 ERROR org.apache.hadoop.hbase.master.HMaster: Failed
to start master
org.apache.hadoop.hbase.util.FileSystemVersionException: File system needs to
be upgraded. Run the '${HBASE_HOME}/bin/hbase migrate' script.
at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:187)
{noformat}
I was puzzled until I realized, after adding extra debug statements in the
code, that I forgot to properly set {{hbase.rootdir}} after re-deploying my
dev environment. I think the message above was misleading and I'm proposing
a trivial patch to make it a little bit better:
{noformat}
2010-04-23 09:48:29,000 ERROR org.apache.hadoop.hbase.master.HMaster: Failed
to start master
org.apache.hadoop.hbase.util.FileSystemVersionException: File system needs to
be upgraded. You have version null and I want version 7. Run the
'${HBASE_HOME}/bin/hbase migrate' script.
at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:189)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5833) 0.92 build has been failing pretty consistently on TestMasterFailover....
[
https://issues.apache.org/jira/browse/HBASE-5833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5833:
--
Comment: was deleted
(was: In build #380, TestMasterFailover hung again:
{code}
Running org.apache.hadoop.hbase.master.TestMasterFailover
Running org.apache.hadoop.hbase.master.TestClockSkewDetection
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.241 sec
{code})
0.92 build has been failing pretty consistently on TestMasterFailover
-
Key: HBASE-5833
URL: https://issues.apache.org/jira/browse/HBASE-5833
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: stack
Fix For: 0.92.2
Attachments: 5833.txt
Trunk seems fine but 0.92 fails on this test pretty regularly. Running it
local it seems to hang for me.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5824) HRegion.incrementColumnValue is not used in trunk
[
https://issues.apache.org/jira/browse/HBASE-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258269#comment-13258269
]
Zhihong Yu commented on HBASE-5824:
---
Implication of this JIRA is that the type of exception thrown when certain
constraint isn't satisfied differs depending on whether auto flush is enabled.
When I changed the test case slightly:
{code}
Index: src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java
===
--- src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java
(revision 1328376)
+++ src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java
(working copy)
@@ -105,7 +105,7 @@
util.getHBaseAdmin().createTable(desc);
HTable table = new HTable(util.getConfiguration(), tableName);
-table.setAutoFlush(true);
+table.setAutoFlush(false);
// test that we do fail on violation
Put put = new Put(row1);
{code}
I got:
{code}
testConstraintFails(org.apache.hadoop.hbase.constraint.TestConstraint) Time
elapsed: 4.144 sec FAILURE!
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at
org.apache.hadoop.hbase.constraint.TestConstraint.testConstraintFails(TestConstraint.java:118)
{code}
This makes exception handling unnecessarily complicated.
HRegion.incrementColumnValue is not used in trunk
-
Key: HBASE-5824
URL: https://issues.apache.org/jira/browse/HBASE-5824
Project: HBase
Issue Type: Bug
Reporter: Elliott Clark
Assignee: Jimmy Xiang
Fix For: 0.96.0
Attachments: hbase-5824.patch, hbase-5824_v2.patch,
hbase_5824.addendum
on 0.94 a call to client.HTable#incrementColumnValue will cause
HRegion#incrementColumnValue. On trunk all calls to
HTable.incrementColumnValue got to HRegion#increment.
My guess is that HTable#incrementColumnValue and HTable#increment serialize
to the same thing over the wire so that the remote HRegionServer no longer
knows which htable method was called.
To repro I checked out trunk and put a break point in
HRegion#incrementColumnValue and then ran TestFromClientSide. The breakpoint
wasn't hit.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5831) hadoopqa builds not completing
[
https://issues.apache.org/jira/browse/HBASE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-5831:
-
Status: Open (was: Patch Available)
hadoopqa builds not completing
--
Key: HBASE-5831
URL: https://issues.apache.org/jira/browse/HBASE-5831
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Priority: Blocker
Attachments: 5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.all.mapreduce.txt, 5831.remove.all.mapreduce.txt,
5831.remove.all.mapreduce.txt, 5831.remove.all.mapreduce.txt
No test failures but build complains it has failed. trunk build seems to
have the same affliction:
{code}
Results :
Tests run: 909, Failures: 0, Errors: 0, Skipped: 9
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 41:19.273s
[INFO] Finished at: Wed Apr 18 21:54:31 UTC 2012
[INFO] Final Memory: 59M/451M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.12-TRUNK-HBASE-2:test
(secondPartTestsExecution) on project hbase: Failure or timeout - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12523250/5811+%281%29.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 6 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number
of release audit warnings.
-1 core tests. The patch failed these unit tests:
{code}
Its not apparent that any particular test is not finishing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5831) hadoopqa builds not completing
[
https://issues.apache.org/jira/browse/HBASE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-5831:
-
Status: Patch Available (was: Open)
hadoopqa builds not completing
--
Key: HBASE-5831
URL: https://issues.apache.org/jira/browse/HBASE-5831
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Priority: Blocker
Attachments: 5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.all.mapreduce.txt, 5831.remove.all.mapreduce.txt,
5831.remove.all.mapreduce.txt, 5831.remove.all.mapreduce.txt,
5831.remove.all.mapreduce.txt
No test failures but build complains it has failed. trunk build seems to
have the same affliction:
{code}
Results :
Tests run: 909, Failures: 0, Errors: 0, Skipped: 9
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 41:19.273s
[INFO] Finished at: Wed Apr 18 21:54:31 UTC 2012
[INFO] Final Memory: 59M/451M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.12-TRUNK-HBASE-2:test
(secondPartTestsExecution) on project hbase: Failure or timeout - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12523250/5811+%281%29.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 6 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number
of release audit warnings.
-1 core tests. The patch failed these unit tests:
{code}
Its not apparent that any particular test is not finishing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5831) hadoopqa builds not completing
[
https://issues.apache.org/jira/browse/HBASE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-5831:
-
Attachment: 5831.remove.all.mapreduce.txt
Retry
hadoopqa builds not completing
--
Key: HBASE-5831
URL: https://issues.apache.org/jira/browse/HBASE-5831
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Priority: Blocker
Attachments: 5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.all.mapreduce.txt, 5831.remove.all.mapreduce.txt,
5831.remove.all.mapreduce.txt, 5831.remove.all.mapreduce.txt,
5831.remove.all.mapreduce.txt
No test failures but build complains it has failed. trunk build seems to
have the same affliction:
{code}
Results :
Tests run: 909, Failures: 0, Errors: 0, Skipped: 9
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 41:19.273s
[INFO] Finished at: Wed Apr 18 21:54:31 UTC 2012
[INFO] Final Memory: 59M/451M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.12-TRUNK-HBASE-2:test
(secondPartTestsExecution) on project hbase: Failure or timeout - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12523250/5811+%281%29.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 6 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number
of release audit warnings.
-1 core tests. The patch failed these unit tests:
{code}
Its not apparent that any particular test is not finishing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5824) HRegion.incrementColumnValue is not used in trunk
[ https://issues.apache.org/jira/browse/HBASE-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258293#comment-13258293 ] Zhihong Yu commented on HBASE-5824: --- Constraint is in 0.94 This change causes regression in the way client responds to Constraint violation. Now client has to deal with two exceptions instead of only one exception (in 0.94) HRegion.incrementColumnValue is not used in trunk - Key: HBASE-5824 URL: https://issues.apache.org/jira/browse/HBASE-5824 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5824.patch, hbase-5824_v2.patch, hbase_5824.addendum on 0.94 a call to client.HTable#incrementColumnValue will cause HRegion#incrementColumnValue. On trunk all calls to HTable.incrementColumnValue got to HRegion#increment. My guess is that HTable#incrementColumnValue and HTable#increment serialize to the same thing over the wire so that the remote HRegionServer no longer knows which htable method was called. To repro I checked out trunk and put a break point in HRegion#incrementColumnValue and then ran TestFromClientSide. The breakpoint wasn't hit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5824) HRegion.incrementColumnValue is not used in trunk
[ https://issues.apache.org/jira/browse/HBASE-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258297#comment-13258297 ] stack commented on HBASE-5824: -- This patch only makes sense in trunk, not in 0.94. What are the exceptions that now are different? HRegion.incrementColumnValue is not used in trunk - Key: HBASE-5824 URL: https://issues.apache.org/jira/browse/HBASE-5824 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5824.patch, hbase-5824_v2.patch, hbase_5824.addendum on 0.94 a call to client.HTable#incrementColumnValue will cause HRegion#incrementColumnValue. On trunk all calls to HTable.incrementColumnValue got to HRegion#increment. My guess is that HTable#incrementColumnValue and HTable#increment serialize to the same thing over the wire so that the remote HRegionServer no longer knows which htable method was called. To repro I checked out trunk and put a break point in HRegion#incrementColumnValue and then ran TestFromClientSide. The breakpoint wasn't hit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5816) Balancer and ServerShutdownHandler concurrently reassigning the same region
[ https://issues.apache.org/jira/browse/HBASE-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258301#comment-13258301 ] ramkrishna.s.vasudevan commented on HBASE-5816: --- I thought more on this.. There are already some state management datastructures like RIT, regions and servers map. So adding more may again be redundant(my thought). and handling them should be clean. Balancer and ServerShutdownHandler concurrently reassigning the same region --- Key: HBASE-5816 URL: https://issues.apache.org/jira/browse/HBASE-5816 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.90.6 Reporter: Maryann Xue Assignee: ramkrishna.s.vasudevan Priority: Critical Attachments: HBASE-5816.patch The first assign thread exits with success after updating the RegionState to PENDING_OPEN, while the second assign follows immediately into assign and fails the RegionState check in setOfflineInZooKeeper(). This causes the master to abort. In the below case, the two concurrent assigns occurred when AM tried to assign a region to a dying/dead RS, and meanwhile the ShutdownServerHandler tried to assign this region (from the region plan) spontaneously. 2012-04-17 05:44:57,648 INFO org.apache.hadoop.hbase.master.HMaster: balance hri=TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b., src=hadoop05.sh.intel.com,60020,1334544902186, dest=xmlqa-clv16.sh.intel.com,60020,1334612497253 2012-04-17 05:44:57,648 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Starting unassignment of region TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b. (offlining) 2012-04-17 05:44:57,648 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Sent CLOSE to serverName=hadoop05.sh.intel.com,60020,1334544902186, load=(requests=0, regions=0, usedHeap=0, maxHeap=0) for region TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b. 2012-04-17 05:44:57,666 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Handling new unassigned node: /hbase/unassigned/fe38fe31caf40b6e607a3e6bbed6404b (region=TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b., server=hadoop05.sh.intel.com,60020,1334544902186, state=RS_ZK_REGION_CLOSING) 2012-04-17 05:52:58,984 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b. state=CLOSED, ts=1334612697672, server=hadoop05.sh.intel.com,60020,1334544902186 2012-04-17 05:52:58,984 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:6-0x236b912e9b3000e Creating (or updating) unassigned node for fe38fe31caf40b6e607a3e6bbed6404b with OFFLINE state 2012-04-17 05:52:59,096 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Using pre-existing plan for region TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b.; plan=hri=TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b., src=hadoop05.sh.intel.com,60020,1334544902186, dest=xmlqa-clv16.sh.intel.com,60020,1334612497253 2012-04-17 05:52:59,096 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Assigning region TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b. to xmlqa-clv16.sh.intel.com,60020,1334612497253 2012-04-17 05:54:19,159 DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Forcing OFFLINE; was=TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b. state=PENDING_OPEN, ts=1334613179096, server=xmlqa-clv16.sh.intel.com,60020,1334612497253 2012-04-17 05:54:59,033 WARN org.apache.hadoop.hbase.master.AssignmentManager: Failed assignment of TABLE_ORDER_CUSTOMER,,1334017820846.fe38fe31caf40b6e607a3e6bbed6404b. to serverName=xmlqa-clv16.sh.intel.com,60020,1334612497253, load=(requests=0, regions=0, usedHeap=0, maxHeap=0), trying to assign elsewhere instead; retry=0 java.net.SocketTimeoutException: Call to /10.239.47.87:60020 failed on socket timeout exception: java.net.SocketTimeoutException: 12 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/10.239.47.89:41302 remote=/10.239.47.87:60020] at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:805) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:778) at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:283) at $Proxy7.openRegion(Unknown Source) at org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:573) at
[jira] [Commented] (HBASE-5824) HRegion.incrementColumnValue is not used in trunk
[ https://issues.apache.org/jira/browse/HBASE-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258313#comment-13258313 ] Zhihong Yu commented on HBASE-5824: --- In 0.94, client only needs to deal with RetriesExhaustedWithDetailsException whose only cause is a ConstraintException With Jimmy's patch, client needs to deal with RetriesExhaustedWithDetailsException (not subclass of DoNotRetryIOException) and ConstraintException (subclass of DoNotRetryIOException). This is because there're two execution paths for Put. I am afraid Benoit would have more to complain about in HBASE-5796 :-) HRegion.incrementColumnValue is not used in trunk - Key: HBASE-5824 URL: https://issues.apache.org/jira/browse/HBASE-5824 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5824.patch, hbase-5824_v2.patch, hbase_5824.addendum on 0.94 a call to client.HTable#incrementColumnValue will cause HRegion#incrementColumnValue. On trunk all calls to HTable.incrementColumnValue got to HRegion#increment. My guess is that HTable#incrementColumnValue and HTable#increment serialize to the same thing over the wire so that the remote HRegionServer no longer knows which htable method was called. To repro I checked out trunk and put a break point in HRegion#incrementColumnValue and then ran TestFromClientSide. The breakpoint wasn't hit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5831) hadoopqa builds not completing
[
https://issues.apache.org/jira/browse/HBASE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258323#comment-13258323
]
Hadoop QA commented on HBASE-5831:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12523507/5831.remove.all.mapreduce.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 26 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1592//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1592//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1592//console
This message is automatically generated.
hadoopqa builds not completing
--
Key: HBASE-5831
URL: https://issues.apache.org/jira/browse/HBASE-5831
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Priority: Blocker
Attachments: 5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.TestLoadIncrementalHFilesSplitRecovery.txt,
5831.remove.all.mapreduce.txt, 5831.remove.all.mapreduce.txt,
5831.remove.all.mapreduce.txt, 5831.remove.all.mapreduce.txt,
5831.remove.all.mapreduce.txt
No test failures but build complains it has failed. trunk build seems to
have the same affliction:
{code}
Results :
Tests run: 909, Failures: 0, Errors: 0, Skipped: 9
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 41:19.273s
[INFO] Finished at: Wed Apr 18 21:54:31 UTC 2012
[INFO] Final Memory: 59M/451M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.12-TRUNK-HBASE-2:test
(secondPartTestsExecution) on project hbase: Failure or timeout - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12523250/5811+%281%29.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 6 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number
of release audit warnings.
-1 core tests. The patch failed these unit tests:
{code}
Its not apparent that any particular test is not finishing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5837) hbase shell deleteall to .META. allows insertion of malformed rowkey.
[
https://issues.apache.org/jira/browse/HBASE-5837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258344#comment-13258344
]
Jonathan Hsieh commented on HBASE-5837:
---
Here's an example shell command that will leave you with a borked hbase:
{code}
deleteall '.META.', 'table,not a comma'
{code}
hbase shell deleteall to .META. allows insertion of malformed rowkey.
-
Key: HBASE-5837
URL: https://issues.apache.org/jira/browse/HBASE-5837
Project: HBase
Issue Type: Bug
Components: master, shell
Affects Versions: 0.90.6
Reporter: Jonathan Hsieh
When using the hbase shell to manipulate meta entries, one is allowed to
'delete' malformed rows (entries with less than 2 ascii 44 ',' chars). When
this happens HBase servers may go down and the cluster will not be
restartable without manual intervention.
The delete results in a durable malformed rowkey in .META.'s memstore,
.META.'s HLog, and eventually .META.'s HFiles. Subsequent scans to meta
(such as when a HMaster starts) fail in the scanner because the comparator
fails. In the case of an HMaster startup, it causes an abort that kills the
HMaster process.
{code}
12/04/18 22:07:34 FATAL master.HMaster: Unhandled exception. Starting
shutdown.
org.apache.hadoop.ipc.RemoteException: java.io.IOException:
java.lang.IllegalArgumentException: No 44 in
blah,1334744821162.81f2df35c332dd2d3bb966fb5b419568., length=47, offset=54
at
org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:990)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:979)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1894)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1834)
at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
Caused by: java.lang.IllegalArgumentException: No 44 in
blah,1334744821162.81f2df35c332dd2d3bb966fb5b419568., length=47, offset=54
at
org.apache.hadoop.hbase.KeyValue.getRequiredDelimiterInReverse(KeyValue.java:1300)
at
org.apache.hadoop.hbase.KeyValue$MetaKeyComparator.compareRows(KeyValue.java:1846)
at
org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:130)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:257)
at
org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:114)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:2435)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:2391)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:2408)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1870)
... 6 more
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:771)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
at $Proxy9.next(Unknown Source)
at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:264)
at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:237)
at
org.apache.hadoop.hbase.catalog.MetaReader.fullScanOfResults(MetaReader.java:220)
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:1580)
at
org.apache.hadoop.hbase.master.AssignmentManager.processFailover(AssignmentManager.java:221)
at
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:422)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:295)
12/04/18 22:07:34 INFO master.HMaster: Aborting
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5842) Passing shell commands as an argument
[
https://issues.apache.org/jira/browse/HBASE-5842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258360#comment-13258360
]
Jonathan Hsieh commented on HBASE-5842:
---
Harsh, I would add that the output from the shell of that particular example
is too clever with formatting and ends up being cumbersome to use with standard
unix parsing tools.
For that particular command, I've hacked the HFile tool do dump contents in an
grep'able format.
Are you looking for that command in particular or are there more cases?
Maybe we should add a hbase-admin command that has shortcuts to utility methods
like HLog, HFile, and something like DumpMeta?
Passing shell commands as an argument
-
Key: HBASE-5842
URL: https://issues.apache.org/jira/browse/HBASE-5842
Project: HBase
Issue Type: Improvement
Components: shell
Affects Versions: 0.94.0
Reporter: Harsh J
Priority: Minor
Many times we've required scans of .META. to analyze issues with the cluster
we work on, and to have the result in a file we can pass around we usually
end up doing something like:
{{echo scan '.META.'| hbase shell meta-scan.txt}}
This can rather be simplified as something like the following instead, with
support for a commands reading argument:
{{hbase shell -c scan '.META.'}}
[Note though: File reading is possible already, i.e. {{hbase shell file.hs}},
but then thats two steps and we usually don't keep a file around for just a
meta table scan.]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3967) Support deletes in HFileOutputFormat based bulk import mechanism
[ https://issues.apache.org/jira/browse/HBASE-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258369#comment-13258369 ] Nicholas Telford commented on HBASE-3967: - Anyone know what the status of this issue is? It looks to have been completed in the latest patch (or perhaps even in HBASE-5440, not sure what Lars meant there) but not reviewed/accepted? Support deletes in HFileOutputFormat based bulk import mechanism Key: HBASE-3967 URL: https://issues.apache.org/jira/browse/HBASE-3967 Project: HBase Issue Type: Sub-task Reporter: Kannan Muthukkaruppan Priority: Critical Fix For: 0.96.0 Attachments: diff.patch During bulk imports, it'll be useful to be able to do delete mutations (either to delete data that already exists in HBase or was inserted earlier during this run of the import). For example, we have a use case, where we are processing a log of data which may have both inserts and deletes in the mix and we want to upload that into HBase using the bulk import mechanism. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5824) HRegion.incrementColumnValue is not used in trunk
[ https://issues.apache.org/jira/browse/HBASE-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258373#comment-13258373 ] Jimmy Xiang commented on HBASE-5824: Yes, this patch is for 0.96 only. RetriesExhaustedWithDetailsException applies to batch processing only. For single action, individual exception is used. Currently only Put is implicitly batched. Should I change single Put to use RetriesExhaustedWithDetailsException too? HRegion.incrementColumnValue is not used in trunk - Key: HBASE-5824 URL: https://issues.apache.org/jira/browse/HBASE-5824 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: 5824-addendum-v2.txt, hbase-5824.patch, hbase-5824_v2.patch, hbase_5824.addendum on 0.94 a call to client.HTable#incrementColumnValue will cause HRegion#incrementColumnValue. On trunk all calls to HTable.incrementColumnValue got to HRegion#increment. My guess is that HTable#incrementColumnValue and HTable#increment serialize to the same thing over the wire so that the remote HRegionServer no longer knows which htable method was called. To repro I checked out trunk and put a break point in HRegion#incrementColumnValue and then ran TestFromClientSide. The breakpoint wasn't hit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5824) HRegion.incrementColumnValue is not used in trunk
[ https://issues.apache.org/jira/browse/HBASE-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5824: -- Attachment: 5824-addendum-v2.txt Proposed addendum that matches the original intent of the JIRA. There is no strong reason for the change in HTable.doPut(final ListPut puts) HRegion.incrementColumnValue is not used in trunk - Key: HBASE-5824 URL: https://issues.apache.org/jira/browse/HBASE-5824 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: 5824-addendum-v2.txt, hbase-5824.patch, hbase-5824_v2.patch, hbase_5824.addendum on 0.94 a call to client.HTable#incrementColumnValue will cause HRegion#incrementColumnValue. On trunk all calls to HTable.incrementColumnValue got to HRegion#increment. My guess is that HTable#incrementColumnValue and HTable#increment serialize to the same thing over the wire so that the remote HRegionServer no longer knows which htable method was called. To repro I checked out trunk and put a break point in HRegion#incrementColumnValue and then ran TestFromClientSide. The breakpoint wasn't hit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5824) HRegion.incrementColumnValue is not used in trunk
[ https://issues.apache.org/jira/browse/HBASE-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258377#comment-13258377 ] Zhihong Yu commented on HBASE-5824: --- @Jimmy: Shall we pursue your proposal in a separate JIRA so that we can keep the original intent of this JIRA ? HRegion.incrementColumnValue is not used in trunk - Key: HBASE-5824 URL: https://issues.apache.org/jira/browse/HBASE-5824 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: 5824-addendum-v2.txt, hbase-5824.patch, hbase-5824_v2.patch, hbase_5824.addendum on 0.94 a call to client.HTable#incrementColumnValue will cause HRegion#incrementColumnValue. On trunk all calls to HTable.incrementColumnValue got to HRegion#increment. My guess is that HTable#incrementColumnValue and HTable#increment serialize to the same thing over the wire so that the remote HRegionServer no longer knows which htable method was called. To repro I checked out trunk and put a break point in HRegion#incrementColumnValue and then ran TestFromClientSide. The breakpoint wasn't hit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5621) Convert admin protocol of HRegionInterface to PB
[ https://issues.apache.org/jira/browse/HBASE-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258379#comment-13258379 ] Jimmy Xiang commented on HBASE-5621: Looking into the failed unit tests. Convert admin protocol of HRegionInterface to PB Key: HBASE-5621 URL: https://issues.apache.org/jira/browse/HBASE-5621 Project: HBase Issue Type: Sub-task Components: ipc, master, migration, regionserver Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: hbase-5621_v3.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5845) Single Put should use RetriesExhaustedWithDetailsException in case any exception
Single Put should use RetriesExhaustedWithDetailsException in case any exception Key: HBASE-5845 URL: https://issues.apache.org/jira/browse/HBASE-5845 Project: HBase Issue Type: Bug Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Due to change in HBASE-5824. Put has to exception paths now. It's better to stay the same for easy exception handling. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5824) HRegion.incrementColumnValue is not used in trunk
[ https://issues.apache.org/jira/browse/HBASE-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258381#comment-13258381 ] Jimmy Xiang commented on HBASE-5824: @Ted, I filed HBASE-5845. Thanks for pointing out the issue. Good catch. HRegion.incrementColumnValue is not used in trunk - Key: HBASE-5824 URL: https://issues.apache.org/jira/browse/HBASE-5824 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: 5824-addendum-v2.txt, hbase-5824.patch, hbase-5824_v2.patch, hbase_5824.addendum on 0.94 a call to client.HTable#incrementColumnValue will cause HRegion#incrementColumnValue. On trunk all calls to HTable.incrementColumnValue got to HRegion#increment. My guess is that HTable#incrementColumnValue and HTable#increment serialize to the same thing over the wire so that the remote HRegionServer no longer knows which htable method was called. To repro I checked out trunk and put a break point in HRegion#incrementColumnValue and then ran TestFromClientSide. The breakpoint wasn't hit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5824) HRegion.incrementColumnValue is not used in trunk
[ https://issues.apache.org/jira/browse/HBASE-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258390#comment-13258390 ] Zhihong Yu commented on HBASE-5824: --- I talked with Jimmy offline. We agree that single Put execution path isn't of high priority. We will leave the discussion and decision making to HBASE-5845. HRegion.incrementColumnValue is not used in trunk - Key: HBASE-5824 URL: https://issues.apache.org/jira/browse/HBASE-5824 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: 5824-addendum-v2.txt, hbase-5824.patch, hbase-5824_v2.patch, hbase_5824.addendum on 0.94 a call to client.HTable#incrementColumnValue will cause HRegion#incrementColumnValue. On trunk all calls to HTable.incrementColumnValue got to HRegion#increment. My guess is that HTable#incrementColumnValue and HTable#increment serialize to the same thing over the wire so that the remote HRegionServer no longer knows which htable method was called. To repro I checked out trunk and put a break point in HRegion#incrementColumnValue and then ran TestFromClientSide. The breakpoint wasn't hit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3614) Expose per-region request rate metrics
[ https://issues.apache.org/jira/browse/HBASE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258394#comment-13258394 ] Elliott Clark commented on HBASE-3614: -- @Todd the time period is configurable. It's set through the hadoop-metrics hbase.period. Whenever the thread that publishes metrics comes around the averages are re-set. Expose per-region request rate metrics -- Key: HBASE-3614 URL: https://issues.apache.org/jira/browse/HBASE-3614 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Gary Helmling Assignee: Elliott Clark Priority: Minor Fix For: 0.96.0 Attachments: HBASE-3614-0.patch, HBASE-3614-1.patch, HBASE-3614-2.patch, HBASE-3614-3.patch, HBASE-3614-4.patch, HBASE-3614-5.patch, HBASE-3614-6.patch, HBASE-3614-7.patch, HBASE-3614-8.patch, HBASE-3614-9.patch, Screen Shot 2012-04-17 at 2.41.27 PM.png We currently export metrics on request rates for each region server, and this can help with identifying uneven load at a high level. But once you see a given server under high load, you're forced to extrapolate based on your application patterns and the data it's serving what the likely culprit is. This can and should be much easier if we just exported request rate metrics per-region on each server. Dynamically updating the metrics keys based on assigned regions may pose some minor challenges, but this seems a very valuable diagnostic tool to have available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5824) HRegion.incrementColumnValue is not used in trunk
[ https://issues.apache.org/jira/browse/HBASE-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258400#comment-13258400 ] Zhihong Yu commented on HBASE-5824: --- Integrated addendum v2 to trunk. HRegion.incrementColumnValue is not used in trunk - Key: HBASE-5824 URL: https://issues.apache.org/jira/browse/HBASE-5824 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: 5824-addendum-v2.txt, hbase-5824.patch, hbase-5824_v2.patch, hbase_5824.addendum on 0.94 a call to client.HTable#incrementColumnValue will cause HRegion#incrementColumnValue. On trunk all calls to HTable.incrementColumnValue got to HRegion#increment. My guess is that HTable#incrementColumnValue and HTable#increment serialize to the same thing over the wire so that the remote HRegionServer no longer knows which htable method was called. To repro I checked out trunk and put a break point in HRegion#incrementColumnValue and then ran TestFromClientSide. The breakpoint wasn't hit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5809) Avoid move api to take the destination server same as the source server.
[ https://issues.apache.org/jira/browse/HBASE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-5809: -- Resolution: Fixed Status: Resolved (was: Patch Available) Avoid move api to take the destination server same as the source server. Key: HBASE-5809 URL: https://issues.apache.org/jira/browse/HBASE-5809 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: ramkrishna.s.vasudevan Priority: Minor Labels: patch Fix For: 0.96.0 Attachments: HBASE-5809.patch In Move currently we take any destination specified and if the destination is same as the source we still do unassign and assign. Here we can have problems due to RegionAlreadyInTransitionException and thus hanging the region in RIT for long time. We can avoid this scenario by not allowing the move to happen in this scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5845) Single Put should use RetriesExhaustedWithDetailsException in case any exception
[ https://issues.apache.org/jira/browse/HBASE-5845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-5845: --- Description: Due to change in HBASE-5824. Put has two exception paths now. It's better to stay the same for easy exception handling. (was: Due to change in HBASE-5824. Put has to exception paths now. It's better to stay the same for easy exception handling.) Single Put should use RetriesExhaustedWithDetailsException in case any exception Key: HBASE-5845 URL: https://issues.apache.org/jira/browse/HBASE-5845 Project: HBase Issue Type: Bug Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Due to change in HBASE-5824. Put has two exception paths now. It's better to stay the same for easy exception handling. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5809) Avoid move api to take the destination server same as the source server.
[ https://issues.apache.org/jira/browse/HBASE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-5809: -- Fix Version/s: (was: 0.94.1) Issue Type: Improvement (was: Bug) Committed to trunk. Made it as an improvement. Thanks for the patch Rajesh. Thanks for the review Ted and Uma. Avoid move api to take the destination server same as the source server. Key: HBASE-5809 URL: https://issues.apache.org/jira/browse/HBASE-5809 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: ramkrishna.s.vasudevan Priority: Minor Labels: patch Fix For: 0.96.0 Attachments: HBASE-5809.patch In Move currently we take any destination specified and if the destination is same as the source we still do unassign and assign. Here we can have problems due to RegionAlreadyInTransitionException and thus hanging the region in RIT for long time. We can avoid this scenario by not allowing the move to happen in this scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5845) Single Put should use RetriesExhaustedWithDetailsException in case any exception
[ https://issues.apache.org/jira/browse/HBASE-5845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5845: -- Description: In HBASE-5824, attempt was made to handle single Put execution separately. Put has two exception paths thereafter. It's better to keep one exception for easy exception handling. (was: Due to change in HBASE-5824. Put has two exception paths now. It's better to stay the same for easy exception handling.) Single Put should use RetriesExhaustedWithDetailsException in case any exception Key: HBASE-5845 URL: https://issues.apache.org/jira/browse/HBASE-5845 Project: HBase Issue Type: Bug Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor In HBASE-5824, attempt was made to handle single Put execution separately. Put has two exception paths thereafter. It's better to keep one exception for easy exception handling. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5809) Avoid move api to take the destination server same as the source server.
[ https://issues.apache.org/jira/browse/HBASE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan reassigned HBASE-5809: - Assignee: rajeshbabu Avoid move api to take the destination server same as the source server. Key: HBASE-5809 URL: https://issues.apache.org/jira/browse/HBASE-5809 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: ramkrishna.s.vasudevan Assignee: rajeshbabu Priority: Minor Labels: patch Fix For: 0.96.0 Attachments: HBASE-5809.patch In Move currently we take any destination specified and if the destination is same as the source we still do unassign and assign. Here we can have problems due to RegionAlreadyInTransitionException and thus hanging the region in RIT for long time. We can avoid this scenario by not allowing the move to happen in this scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5809) Avoid move api to take the destination server same as the source server.
[ https://issues.apache.org/jira/browse/HBASE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-5809: -- Attachment: HBASE-5809.patch This is what i finally committed. Small formatting change and corrected typo error. Avoid move api to take the destination server same as the source server. Key: HBASE-5809 URL: https://issues.apache.org/jira/browse/HBASE-5809 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: ramkrishna.s.vasudevan Assignee: rajeshbabu Priority: Minor Labels: patch Fix For: 0.96.0 Attachments: HBASE-5809.patch, HBASE-5809.patch In Move currently we take any destination specified and if the destination is same as the source we still do unassign and assign. Here we can have problems due to RegionAlreadyInTransitionException and thus hanging the region in RIT for long time. We can avoid this scenario by not allowing the move to happen in this scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5635) If getTaskList() returns null splitlogWorker is down. It wont serve any requests.
[
https://issues.apache.org/jira/browse/HBASE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258408#comment-13258408
]
ramkrishna.s.vasudevan commented on HBASE-5635:
---
@devs
I plan to commit this to trunk tomorrow. Pls provide your comments.
If getTaskList() returns null splitlogWorker is down. It wont serve any
requests.
--
Key: HBASE-5635
URL: https://issues.apache.org/jira/browse/HBASE-5635
Project: HBase
Issue Type: Bug
Components: wal
Affects Versions: 0.92.1
Reporter: Kristam Subba Swathi
Attachments: HBASE-5635.1.patch, HBASE-5635.2.patch, HBASE-5635.patch
During the hlog split operation if all the zookeepers are down ,then the
paths will be returned as null and the splitworker thread wil be exited
Now this regionserver wil not be able to acquire any other tasks since the
splitworker thread is exited
Please find the attached code for more details
{code}
private ListString getTaskList() {
for (int i = 0; i zkretries; i++) {
try {
return (ZKUtil.listChildrenAndWatchForNewChildren(this.watcher,
this.watcher.splitLogZNode));
} catch (KeeperException e) {
LOG.warn(Could not get children of znode +
this.watcher.splitLogZNode, e);
try {
Thread.sleep(1000);
} catch (InterruptedException e1) {
LOG.warn(Interrupted while trying to get task list ..., e1);
Thread.currentThread().interrupt();
return null;
}
}
}
{code}
in the org.apache.hadoop.hbase.regionserver.SplitLogWorker
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5824) HRegion.incrementColumnValue is not used in trunk
[ https://issues.apache.org/jira/browse/HBASE-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258411#comment-13258411 ] Hadoop QA commented on HBASE-5824: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12523521/5824-addendum-v2.txt against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1593//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1593//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1593//console This message is automatically generated. HRegion.incrementColumnValue is not used in trunk - Key: HBASE-5824 URL: https://issues.apache.org/jira/browse/HBASE-5824 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: 5824-addendum-v2.txt, hbase-5824.patch, hbase-5824_v2.patch, hbase_5824.addendum on 0.94 a call to client.HTable#incrementColumnValue will cause HRegion#incrementColumnValue. On trunk all calls to HTable.incrementColumnValue got to HRegion#increment. My guess is that HTable#incrementColumnValue and HTable#increment serialize to the same thing over the wire so that the remote HRegionServer no longer knows which htable method was called. To repro I checked out trunk and put a break point in HRegion#incrementColumnValue and then ran TestFromClientSide. The breakpoint wasn't hit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5635) If getTaskList() returns null splitlogWorker is down. It wont serve any requests.
[
https://issues.apache.org/jira/browse/HBASE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258414#comment-13258414
]
Lars Hofhansl commented on HBASE-5635:
--
+1 on patch. I wonder how many more problems like this we have lurking in the
worker threads.
Can you do a 0.94.1 patch as well?
If getTaskList() returns null splitlogWorker is down. It wont serve any
requests.
--
Key: HBASE-5635
URL: https://issues.apache.org/jira/browse/HBASE-5635
Project: HBase
Issue Type: Bug
Components: wal
Affects Versions: 0.92.1
Reporter: Kristam Subba Swathi
Attachments: HBASE-5635.1.patch, HBASE-5635.2.patch, HBASE-5635.patch
During the hlog split operation if all the zookeepers are down ,then the
paths will be returned as null and the splitworker thread wil be exited
Now this regionserver wil not be able to acquire any other tasks since the
splitworker thread is exited
Please find the attached code for more details
{code}
private ListString getTaskList() {
for (int i = 0; i zkretries; i++) {
try {
return (ZKUtil.listChildrenAndWatchForNewChildren(this.watcher,
this.watcher.splitLogZNode));
} catch (KeeperException e) {
LOG.warn(Could not get children of znode +
this.watcher.splitLogZNode, e);
try {
Thread.sleep(1000);
} catch (InterruptedException e1) {
LOG.warn(Interrupted while trying to get task list ..., e1);
Thread.currentThread().interrupt();
return null;
}
}
}
{code}
in the org.apache.hadoop.hbase.regionserver.SplitLogWorker
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5844) Delete the region servers znode after a regions server crash
[ https://issues.apache.org/jira/browse/HBASE-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258422#comment-13258422 ] Jean-Daniel Cryans commented on HBASE-5844: --- It is deleted automatically, it's an ephemeral znode so it takes zk.session.timeout time to see it disappear. Delete the region servers znode after a regions server crash Key: HBASE-5844 URL: https://issues.apache.org/jira/browse/HBASE-5844 Project: HBase Issue Type: Improvement Components: regionserver, scripts Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Attachments: 5844.v1.patch today, if the regions server crashes, its znode is not deleted in ZooKeeper. So the recovery process will stop only after a timeout, usually 30s. By deleting the znode in start script, we remove this delay and the recovery starts immediately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5635) If getTaskList() returns null splitlogWorker is down. It wont serve any requests.
[
https://issues.apache.org/jira/browse/HBASE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258441#comment-13258441
]
ramkrishna.s.vasudevan commented on HBASE-5635:
---
Thanks Lars. Sure i will make a patch for 0.94 as well and commit both
tomorrow.
If getTaskList() returns null splitlogWorker is down. It wont serve any
requests.
--
Key: HBASE-5635
URL: https://issues.apache.org/jira/browse/HBASE-5635
Project: HBase
Issue Type: Bug
Components: wal
Affects Versions: 0.92.1
Reporter: Kristam Subba Swathi
Attachments: HBASE-5635.1.patch, HBASE-5635.2.patch, HBASE-5635.patch
During the hlog split operation if all the zookeepers are down ,then the
paths will be returned as null and the splitworker thread wil be exited
Now this regionserver wil not be able to acquire any other tasks since the
splitworker thread is exited
Please find the attached code for more details
{code}
private ListString getTaskList() {
for (int i = 0; i zkretries; i++) {
try {
return (ZKUtil.listChildrenAndWatchForNewChildren(this.watcher,
this.watcher.splitLogZNode));
} catch (KeeperException e) {
LOG.warn(Could not get children of znode +
this.watcher.splitLogZNode, e);
try {
Thread.sleep(1000);
} catch (InterruptedException e1) {
LOG.warn(Interrupted while trying to get task list ..., e1);
Thread.currentThread().interrupt();
return null;
}
}
}
{code}
in the org.apache.hadoop.hbase.regionserver.SplitLogWorker
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5846) HBase rpm packing is broken at multiple places
HBase rpm packing is broken at multiple places
--
Key: HBASE-5846
URL: https://issues.apache.org/jira/browse/HBASE-5846
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 0.92.1
Environment: CentOS release 5.7 (Final)
Reporter: Shrijeet Paliwal
Here is how I executed rpm build:
{noformat}
MAVEN_OPTS=-Xmx2g mvn clean package assembly:single -Prpm -DskipTests
{noformat}
The issues with the rpm build are:
* There is no clean (%clean) section in the hbase.spec file . Last run can
leave stuff in RPM_BUILD_ROOT which in turn will fail build. As a fix I added
'rm -rf $RPM_BUILD_ROOT' to %clean section
* The Buildroot is set to _build_dir . The build fails with this error.
{noformat}
cp: cannot copy a directory,
`/data/9adda425-1f1e-4fe5-8a53-83bd2ce5ad45/app/jenkins/workspace/hbase.92/target/rpm/hbase/BUILD',
into itself,
`/data/9adda425-1f1e-4fe5-8a53-83bd2ce5ad45/app/jenkins/workspace/hbase.92/target/rpm/hbase/BUILD/BUILD'
{noformat}
If we set it to ' %{_tmppath}/%{name}-%{version}-root' build passes
* The src/packages/update-hbase-env.sh script will leave inconsistent state if
'yum update hbase' is executed. It deletes data from /etc/init.d/hbase* and
does not put scripts back during update.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5635) If getTaskList() returns null splitlogWorker is down. It wont serve any requests.
[
https://issues.apache.org/jira/browse/HBASE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-5635:
--
Attachment: HBASE-5635._trunk.patch
If getTaskList() returns null splitlogWorker is down. It wont serve any
requests.
--
Key: HBASE-5635
URL: https://issues.apache.org/jira/browse/HBASE-5635
Project: HBase
Issue Type: Bug
Components: wal
Affects Versions: 0.92.1
Reporter: Kristam Subba Swathi
Attachments: HBASE-5635.1.patch, HBASE-5635.2.patch,
HBASE-5635._trunk.patch, HBASE-5635.patch
During the hlog split operation if all the zookeepers are down ,then the
paths will be returned as null and the splitworker thread wil be exited
Now this regionserver wil not be able to acquire any other tasks since the
splitworker thread is exited
Please find the attached code for more details
{code}
private ListString getTaskList() {
for (int i = 0; i zkretries; i++) {
try {
return (ZKUtil.listChildrenAndWatchForNewChildren(this.watcher,
this.watcher.splitLogZNode));
} catch (KeeperException e) {
LOG.warn(Could not get children of znode +
this.watcher.splitLogZNode, e);
try {
Thread.sleep(1000);
} catch (InterruptedException e1) {
LOG.warn(Interrupted while trying to get task list ..., e1);
Thread.currentThread().interrupt();
return null;
}
}
}
{code}
in the org.apache.hadoop.hbase.regionserver.SplitLogWorker
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5635) If getTaskList() returns null splitlogWorker is down. It wont serve any requests.
[
https://issues.apache.org/jira/browse/HBASE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258447#comment-13258447
]
ramkrishna.s.vasudevan commented on HBASE-5635:
---
Reattached the patch and prepared one for 0.94 also.
If getTaskList() returns null splitlogWorker is down. It wont serve any
requests.
--
Key: HBASE-5635
URL: https://issues.apache.org/jira/browse/HBASE-5635
Project: HBase
Issue Type: Bug
Components: wal
Affects Versions: 0.92.1
Reporter: Kristam Subba Swathi
Attachments: HBASE-5635.1.patch, HBASE-5635.2.patch,
HBASE-5635._trunk.patch, HBASE-5635.patch, HBASE-5635_0.94.patch
During the hlog split operation if all the zookeepers are down ,then the
paths will be returned as null and the splitworker thread wil be exited
Now this regionserver wil not be able to acquire any other tasks since the
splitworker thread is exited
Please find the attached code for more details
{code}
private ListString getTaskList() {
for (int i = 0; i zkretries; i++) {
try {
return (ZKUtil.listChildrenAndWatchForNewChildren(this.watcher,
this.watcher.splitLogZNode));
} catch (KeeperException e) {
LOG.warn(Could not get children of znode +
this.watcher.splitLogZNode, e);
try {
Thread.sleep(1000);
} catch (InterruptedException e1) {
LOG.warn(Interrupted while trying to get task list ..., e1);
Thread.currentThread().interrupt();
return null;
}
}
}
{code}
in the org.apache.hadoop.hbase.regionserver.SplitLogWorker
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5635) If getTaskList() returns null splitlogWorker is down. It wont serve any requests.
[
https://issues.apache.org/jira/browse/HBASE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-5635:
--
Attachment: HBASE-5635_0.94.patch
If getTaskList() returns null splitlogWorker is down. It wont serve any
requests.
--
Key: HBASE-5635
URL: https://issues.apache.org/jira/browse/HBASE-5635
Project: HBase
Issue Type: Bug
Components: wal
Affects Versions: 0.92.1
Reporter: Kristam Subba Swathi
Attachments: HBASE-5635.1.patch, HBASE-5635.2.patch,
HBASE-5635._trunk.patch, HBASE-5635.patch, HBASE-5635_0.94.patch
During the hlog split operation if all the zookeepers are down ,then the
paths will be returned as null and the splitworker thread wil be exited
Now this regionserver wil not be able to acquire any other tasks since the
splitworker thread is exited
Please find the attached code for more details
{code}
private ListString getTaskList() {
for (int i = 0; i zkretries; i++) {
try {
return (ZKUtil.listChildrenAndWatchForNewChildren(this.watcher,
this.watcher.splitLogZNode));
} catch (KeeperException e) {
LOG.warn(Could not get children of znode +
this.watcher.splitLogZNode, e);
try {
Thread.sleep(1000);
} catch (InterruptedException e1) {
LOG.warn(Interrupted while trying to get task list ..., e1);
Thread.currentThread().interrupt();
return null;
}
}
}
{code}
in the org.apache.hadoop.hbase.regionserver.SplitLogWorker
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5635) If getTaskList() returns null, splitlogWorker would go down and it won't serve any requests
[
https://issues.apache.org/jira/browse/HBASE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5635:
--
Summary: If getTaskList() returns null, splitlogWorker would go down and it
won't serve any requests (was: If getTaskList() returns null splitlogWorker is
down. It wont serve any requests. )
If getTaskList() returns null, splitlogWorker would go down and it won't
serve any requests
---
Key: HBASE-5635
URL: https://issues.apache.org/jira/browse/HBASE-5635
Project: HBase
Issue Type: Bug
Components: wal
Affects Versions: 0.92.1
Reporter: Kristam Subba Swathi
Attachments: HBASE-5635.1.patch, HBASE-5635.2.patch,
HBASE-5635._trunk.patch, HBASE-5635.patch, HBASE-5635_0.94.patch
During the hlog split operation if all the zookeepers are down ,then the
paths will be returned as null and the splitworker thread wil be exited
Now this regionserver wil not be able to acquire any other tasks since the
splitworker thread is exited
Please find the attached code for more details
{code}
private ListString getTaskList() {
for (int i = 0; i zkretries; i++) {
try {
return (ZKUtil.listChildrenAndWatchForNewChildren(this.watcher,
this.watcher.splitLogZNode));
} catch (KeeperException e) {
LOG.warn(Could not get children of znode +
this.watcher.splitLogZNode, e);
try {
Thread.sleep(1000);
} catch (InterruptedException e1) {
LOG.warn(Interrupted while trying to get task list ..., e1);
Thread.currentThread().interrupt();
return null;
}
}
}
{code}
in the org.apache.hadoop.hbase.regionserver.SplitLogWorker
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5836) Backport per region metrics from HBASE-3614 to 0.94.1
[ https://issues.apache.org/jira/browse/HBASE-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-5836: - Attachment: HBASE-5836-0.patch backport of the two jira's Backport per region metrics from HBASE-3614 to 0.94.1 - Key: HBASE-5836 URL: https://issues.apache.org/jira/browse/HBASE-5836 Project: HBase Issue Type: Task Reporter: stack Assignee: Elliott Clark Fix For: 0.94.1 Attachments: HBASE-5836-0.patch This would be good to have in 0.94. Can go into 0.94.1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5824) HRegion.incrementColumnValue is not used in trunk
[ https://issues.apache.org/jira/browse/HBASE-5824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258463#comment-13258463 ] Hudson commented on HBASE-5824: --- Integrated in HBase-TRUNK #2791 (See [https://builds.apache.org/job/HBase-TRUNK/2791/]) HBASE-5824 revert changes to single Put case, preserving deprecation for ICV (Revision 1328457) Result = FAILURE tedyu : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HTable.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/constraint/TestConstraint.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerCoprocessorExceptionWithRemove.java HRegion.incrementColumnValue is not used in trunk - Key: HBASE-5824 URL: https://issues.apache.org/jira/browse/HBASE-5824 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Jimmy Xiang Fix For: 0.96.0 Attachments: 5824-addendum-v2.txt, hbase-5824.patch, hbase-5824_v2.patch, hbase_5824.addendum on 0.94 a call to client.HTable#incrementColumnValue will cause HRegion#incrementColumnValue. On trunk all calls to HTable.incrementColumnValue got to HRegion#increment. My guess is that HTable#incrementColumnValue and HTable#increment serialize to the same thing over the wire so that the remote HRegionServer no longer knows which htable method was called. To repro I checked out trunk and put a break point in HRegion#incrementColumnValue and then ran TestFromClientSide. The breakpoint wasn't hit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5809) Avoid move api to take the destination server same as the source server.
[ https://issues.apache.org/jira/browse/HBASE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258464#comment-13258464 ] Hudson commented on HBASE-5809: --- Integrated in HBase-TRUNK #2791 (See [https://builds.apache.org/job/HBase-TRUNK/2791/]) HBASE-5809 Avoid move api to take the destination server same as the source server. (Rajesh) (Revision 1328458) Result = FAILURE ramkrishna : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java Avoid move api to take the destination server same as the source server. Key: HBASE-5809 URL: https://issues.apache.org/jira/browse/HBASE-5809 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: ramkrishna.s.vasudevan Assignee: rajeshbabu Priority: Minor Labels: client Fix For: 0.96.0 Attachments: HBASE-5809.patch, HBASE-5809.patch In Move currently we take any destination specified and if the destination is same as the source we still do unassign and assign. Here we can have problems due to RegionAlreadyInTransitionException and thus hanging the region in RIT for long time. We can avoid this scenario by not allowing the move to happen in this scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5847) Support createTable(splitKeys) in Thrift
Support createTable(splitKeys) in Thrift Key: HBASE-5847 URL: https://issues.apache.org/jira/browse/HBASE-5847 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Trivial The Thrift API does not allow a user to create a table with multiple split keys. This is needed for a handful of new internal projects that are written in PHP/C++. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5846) HBase rpm packing is broken at multiple places
[
https://issues.apache.org/jira/browse/HBASE-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258467#comment-13258467
]
Shrijeet Paliwal commented on HBASE-5846:
-
Here is what happens if one runs update :
{noformat}
D: install: %post(hbase-0.92.1-2.x86_64) synchronous scriptlet start
D: install: %post(hbase-0.92.1-2.x86_64) execv(/bin/sh) pid 26772
+ /usr/share/hbase/sbin/update-hbase-env.sh --prefix=/usr --bin-dir=/usr/bin
--conf-dir=/etc/hbase --log-dir=/var/log/hbase --pid-dir=/var/run/hbase
D: install: waitpid(26772) rc 26772 status 0 secs 0.038
D: == --- hbase-0.92.1-1 x86_64-linux 0x0
D: erase: hbase-0.92.1-1 has 224 files, test = 0
D: erase: %preun(hbase-0.92.1-1.x86_64) asynchronous scriptlet start
D: erase: %preun(hbase-0.92.1-1.x86_64) execv(/bin/sh) pid 26819
+ /usr/share/hbase/sbin/update-hbase-env.sh --prefix=/usr --bin-dir=/usr/bin
--conf-dir=/etc/hbase --log-dir=/var/log/hbase --pid-dir=/var/run/hbase
--uninstall
{noformat}
This is out put of rpm -Uvv . Note how install post runs followed by preun .
preun erases all the work that was done by install post.
HBase rpm packing is broken at multiple places
--
Key: HBASE-5846
URL: https://issues.apache.org/jira/browse/HBASE-5846
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 0.92.1
Environment: CentOS release 5.7 (Final)
Reporter: Shrijeet Paliwal
Here is how I executed rpm build:
{noformat}
MAVEN_OPTS=-Xmx2g mvn clean package assembly:single -Prpm -DskipTests
{noformat}
The issues with the rpm build are:
* There is no clean (%clean) section in the hbase.spec file . Last run can
leave stuff in RPM_BUILD_ROOT which in turn will fail build. As a fix I added
'rm -rf $RPM_BUILD_ROOT' to %clean section
* The Buildroot is set to _build_dir . The build fails with this error.
{noformat}
cp: cannot copy a directory,
`/data/9adda425-1f1e-4fe5-8a53-83bd2ce5ad45/app/jenkins/workspace/hbase.92/target/rpm/hbase/BUILD',
into itself,
`/data/9adda425-1f1e-4fe5-8a53-83bd2ce5ad45/app/jenkins/workspace/hbase.92/target/rpm/hbase/BUILD/BUILD'
{noformat}
If we set it to ' %{_tmppath}/%{name}-%{version}-root' build passes
* The src/packages/update-hbase-env.sh script will leave inconsistent state
if 'yum update hbase' is executed. It deletes data from /etc/init.d/hbase*
and does not put scripts back during update.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5809) Avoid move api to take the destination server same as the source server.
[
https://issues.apache.org/jira/browse/HBASE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258474#comment-13258474
]
Zhihong Yu commented on HBASE-5809:
---
I looped the test using:
{code}
~/runtest.sh 4 TestMasterObserver#testRegionTransitionOperations
{code}
and got the following:
{code}
Failed tests:
testRegionTransitionOperations(org.apache.hadoop.hbase.coprocessor.TestMasterObserver):
Coprocessor should have been called on region move
...
TestMasterObserver#testRegionTransitionOperations failed, iteration: 2
{code}
Avoid move api to take the destination server same as the source server.
Key: HBASE-5809
URL: https://issues.apache.org/jira/browse/HBASE-5809
Project: HBase
Issue Type: Improvement
Affects Versions: 0.92.1
Reporter: ramkrishna.s.vasudevan
Assignee: rajeshbabu
Priority: Minor
Labels: client
Fix For: 0.96.0
Attachments: HBASE-5809.patch, HBASE-5809.patch
In Move currently we take any destination specified and if the destination is
same as the source we still do unassign and assign. Here we can have
problems due to RegionAlreadyInTransitionException and thus hanging the
region in RIT for long time. We can avoid this scenario by not allowing the
move to happen in this scenario.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5635) If getTaskList() returns null, splitlogWorker would go down and it won't serve any requests
[
https://issues.apache.org/jira/browse/HBASE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258492#comment-13258492
]
Hadoop QA commented on HBASE-5635:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12523534/HBASE-5635_0.94.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestMasterObserver
org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks
org.apache.hadoop.hbase.master.TestSplitLogManager
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/1594//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/1594//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/1594//console
This message is automatically generated.
If getTaskList() returns null, splitlogWorker would go down and it won't
serve any requests
---
Key: HBASE-5635
URL: https://issues.apache.org/jira/browse/HBASE-5635
Project: HBase
Issue Type: Bug
Components: wal
Affects Versions: 0.92.1
Reporter: Kristam Subba Swathi
Attachments: HBASE-5635.1.patch, HBASE-5635.2.patch,
HBASE-5635._trunk.patch, HBASE-5635.patch, HBASE-5635_0.94.patch
During the hlog split operation if all the zookeepers are down ,then the
paths will be returned as null and the splitworker thread wil be exited
Now this regionserver wil not be able to acquire any other tasks since the
splitworker thread is exited
Please find the attached code for more details
{code}
private ListString getTaskList() {
for (int i = 0; i zkretries; i++) {
try {
return (ZKUtil.listChildrenAndWatchForNewChildren(this.watcher,
this.watcher.splitLogZNode));
} catch (KeeperException e) {
LOG.warn(Could not get children of znode +
this.watcher.splitLogZNode, e);
try {
Thread.sleep(1000);
} catch (InterruptedException e1) {
LOG.warn(Interrupted while trying to get task list ..., e1);
Thread.currentThread().interrupt();
return null;
}
}
}
{code}
in the org.apache.hadoop.hbase.regionserver.SplitLogWorker
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5809) Avoid move api to take the destination server same as the source server.
[ https://issues.apache.org/jira/browse/HBASE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu reopened HBASE-5809: --- The test failure was visible on Hadoop QA as well: https://builds.apache.org/job/PreCommit-HBASE-Build/1594//testReport/org.apache.hadoop.hbase.coprocessor/TestMasterObserver/testRegionTransitionOperations/ Avoid move api to take the destination server same as the source server. Key: HBASE-5809 URL: https://issues.apache.org/jira/browse/HBASE-5809 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: ramkrishna.s.vasudevan Assignee: rajeshbabu Priority: Minor Labels: client Fix For: 0.96.0 Attachments: HBASE-5809.patch, HBASE-5809.patch In Move currently we take any destination specified and if the destination is same as the source we still do unassign and assign. Here we can have problems due to RegionAlreadyInTransitionException and thus hanging the region in RIT for long time. We can avoid this scenario by not allowing the move to happen in this scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5809) Avoid move api to take the destination server same as the source server.
[
https://issues.apache.org/jira/browse/HBASE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Yu updated HBASE-5809:
--
Comment: was deleted
(was: I think the failure was due to testRegionTransitionOperations not
checking whether the source and destination servers are the same:
{code}
master.move(firstGoodPair.getKey().getEncodedNameAsBytes(),
Bytes.toBytes(destName));
{code}
To make the test valid, destName should be chosen to be different from source
server.)
Avoid move api to take the destination server same as the source server.
Key: HBASE-5809
URL: https://issues.apache.org/jira/browse/HBASE-5809
Project: HBase
Issue Type: Improvement
Affects Versions: 0.92.1
Reporter: ramkrishna.s.vasudevan
Assignee: rajeshbabu
Priority: Minor
Labels: client
Fix For: 0.96.0
Attachments: HBASE-5809.patch, HBASE-5809.patch
In Move currently we take any destination specified and if the destination is
same as the source we still do unassign and assign. Here we can have
problems due to RegionAlreadyInTransitionException and thus hanging the
region in RIT for long time. We can avoid this scenario by not allowing the
move to happen in this scenario.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5809) Avoid move api to take the destination server same as the source server.
[ https://issues.apache.org/jira/browse/HBASE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5809: -- Attachment: 5809.addendum The addendum fixes the incorrect comparison between server names. Avoid move api to take the destination server same as the source server. Key: HBASE-5809 URL: https://issues.apache.org/jira/browse/HBASE-5809 Project: HBase Issue Type: Improvement Affects Versions: 0.92.1 Reporter: ramkrishna.s.vasudevan Assignee: rajeshbabu Priority: Minor Labels: client Fix For: 0.96.0 Attachments: 5809.addendum, HBASE-5809.patch, HBASE-5809.patch In Move currently we take any destination specified and if the destination is same as the source we still do unassign and assign. Here we can have problems due to RegionAlreadyInTransitionException and thus hanging the region in RIT for long time. We can avoid this scenario by not allowing the move to happen in this scenario. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3967) Support deletes in HFileOutputFormat based bulk import mechanism
[ https://issues.apache.org/jira/browse/HBASE-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258599#comment-13258599 ] Lars Hofhansl commented on HBASE-3967: -- HFileOutputFormat just stores KeyValues so it already handles Deletes. The question is: How do you get Deletes output from a Mapper? One solution (the one I took in HBASE-5440) is to have Mapper emit KeyValues and then use KeyValueSortReducer in the reduce phase. Another approach is this jira, which is what Facebook has internally (as far as I know). Yet another approach could be use Mutation (which is new in 0.92), and write a new SortReducer. I think the point of this jira is to get the Facebook approach into 0.94/trunk in order to make upgrading more palatable for Facebook. Kannan, correct me if I am wrong. Support deletes in HFileOutputFormat based bulk import mechanism Key: HBASE-3967 URL: https://issues.apache.org/jira/browse/HBASE-3967 Project: HBase Issue Type: Sub-task Reporter: Kannan Muthukkaruppan Priority: Critical Fix For: 0.96.0 Attachments: diff.patch During bulk imports, it'll be useful to be able to do delete mutations (either to delete data that already exists in HBase or was inserted earlier during this run of the import). For example, we have a use case, where we are processing a log of data which may have both inserts and deletes in the mix and we want to upload that into HBase using the bulk import mechanism. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5836) Backport per region metrics from HBASE-3614 to 0.94.1
[ https://issues.apache.org/jira/browse/HBASE-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-5836: - Affects Version/s: 0.94.1 Status: Patch Available (was: Open) Backport per region metrics from HBASE-3614 to 0.94.1 - Key: HBASE-5836 URL: https://issues.apache.org/jira/browse/HBASE-5836 Project: HBase Issue Type: Task Affects Versions: 0.94.1 Reporter: stack Assignee: Elliott Clark Fix For: 0.94.1 Attachments: HBASE-5836-0.patch This would be good to have in 0.94. Can go into 0.94.1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5836) Backport per region metrics from HBASE-3614 to 0.94.1
[ https://issues.apache.org/jira/browse/HBASE-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258619#comment-13258619 ] Hadoop QA commented on HBASE-5836: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12523535/HBASE-5836-0.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1595//console This message is automatically generated. Backport per region metrics from HBASE-3614 to 0.94.1 - Key: HBASE-5836 URL: https://issues.apache.org/jira/browse/HBASE-5836 Project: HBase Issue Type: Task Affects Versions: 0.94.1 Reporter: stack Assignee: Elliott Clark Fix For: 0.94.1 Attachments: HBASE-5836-0.patch This would be good to have in 0.94. Can go into 0.94.1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5732) Remove the SecureRPCEngine and merge the security-related logic in the core engine
[ https://issues.apache.org/jira/browse/HBASE-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258626#comment-13258626 ] Devaraj Das commented on HBASE-5732: bq. On review board, it is obvious where white spaces are introduced Zhihong, could you please put a link to the reviewboard for this jira please (I don't seem to be able to locate it easily).. Remove the SecureRPCEngine and merge the security-related logic in the core engine -- Key: HBASE-5732 URL: https://issues.apache.org/jira/browse/HBASE-5732 Project: HBase Issue Type: Improvement Reporter: Devaraj Das Assignee: Devaraj Das Attachments: rpcengine-merge.patch Remove the SecureRPCEngine and merge the security-related logic in the core engine. Follow up to HBASE-5727. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5732) Remove the SecureRPCEngine and merge the security-related logic in the core engine
[ https://issues.apache.org/jira/browse/HBASE-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258628#comment-13258628 ] Devaraj Das commented on HBASE-5732: I forgot to mention that I'll keep the User shim as it is (please let me know if this doesn't seem right). So most of the changes will be in HBaseServer/HBaseClient files. Will upload a patch tonight or over the weekend (sorry for the delay). Remove the SecureRPCEngine and merge the security-related logic in the core engine -- Key: HBASE-5732 URL: https://issues.apache.org/jira/browse/HBASE-5732 Project: HBase Issue Type: Improvement Reporter: Devaraj Das Assignee: Devaraj Das Attachments: rpcengine-merge.patch Remove the SecureRPCEngine and merge the security-related logic in the core engine. Follow up to HBASE-5727. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5732) Remove the SecureRPCEngine and merge the security-related logic in the core engine
[ https://issues.apache.org/jira/browse/HBASE-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258632#comment-13258632 ] Zhihong Yu commented on HBASE-5732: --- Start with https://reviews.apache.org/r/new/: select hbase for Repository and '/' for Base Directory Use Browse to locate your patch. Press 'Create New Request' button. On the next screen, enter hbase for Group. Once required fields are filled, you should be able to publish your latest patch. Remove the SecureRPCEngine and merge the security-related logic in the core engine -- Key: HBASE-5732 URL: https://issues.apache.org/jira/browse/HBASE-5732 Project: HBase Issue Type: Improvement Reporter: Devaraj Das Assignee: Devaraj Das Attachments: rpcengine-merge.patch Remove the SecureRPCEngine and merge the security-related logic in the core engine. Follow up to HBASE-5727. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5732) Remove the SecureRPCEngine and merge the security-related logic in the core engine
[ https://issues.apache.org/jira/browse/HBASE-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258634#comment-13258634 ] Devaraj Das commented on HBASE-5732: Ah I thought you had uploaded the patch on reviewboard and I could use that .. Never mind. I'll take care.. Remove the SecureRPCEngine and merge the security-related logic in the core engine -- Key: HBASE-5732 URL: https://issues.apache.org/jira/browse/HBASE-5732 Project: HBase Issue Type: Improvement Reporter: Devaraj Das Assignee: Devaraj Das Attachments: rpcengine-merge.patch Remove the SecureRPCEngine and merge the security-related logic in the core engine. Follow up to HBASE-5727. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5732) Remove the SecureRPCEngine and merge the security-related logic in the core engine
[ https://issues.apache.org/jira/browse/HBASE-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258637#comment-13258637 ] Zhihong Yu commented on HBASE-5732: --- Only the review requester can update the patch for the review. So the person composing the patch should be creating review request. Remove the SecureRPCEngine and merge the security-related logic in the core engine -- Key: HBASE-5732 URL: https://issues.apache.org/jira/browse/HBASE-5732 Project: HBase Issue Type: Improvement Reporter: Devaraj Das Assignee: Devaraj Das Attachments: rpcengine-merge.patch Remove the SecureRPCEngine and merge the security-related logic in the core engine. Follow up to HBASE-5727. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5801) [hbck] Hbck should handle case where some regions have different HTD settings in .regioninfo files (0.90 specific)
[ https://issues.apache.org/jira/browse/HBASE-5801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258649#comment-13258649 ] jirapos...@reviews.apache.org commented on HBASE-5801: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4833/ --- Review request for hbase and jmhsieh. Summary --- Added option to fix inconsistent table descriptors: 1. sideline the current .regioninfo file 2. create a new one with HTD from HBaseAdmin (meta, first entry) 3. offline the region and wait till it assigned again This addresses bug HBASE-5801. https://issues.apache.org/jira/browse/HBASE-5801 Diffs - src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java 50f9128 src/main/java/org/apache/hadoop/hbase/util/HBaseFsckRepair.java 06d2b73 src/test/java/org/apache/hadoop/hbase/util/TestHBaseFsck.java 103d8bf Diff: https://reviews.apache.org/r/4833/diff Testing --- TestHBaseFsck* are green. On live cluster, it does the fix as expected. Thanks, Jimmy [hbck] Hbck should handle case where some regions have different HTD settings in .regioninfo files (0.90 specific) --- Key: HBASE-5801 URL: https://issues.apache.org/jira/browse/HBASE-5801 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Recently, we encountered a case where some regions in a table have different HTableDescriptor settings serialized into HDFS their HRegionInfo .regioninfo file. hbck expects all HTDs within a table to be the same and currently bails out in this situation. We need to either point out a proper set of actions for the user to execute or automatically convert the region to a common HTD (likely the most common on, or possibly the first one.) Not sure if this requires reformatting data but may require closing and restarting a region. This issue is hbase 0.90.x specific -- 0.92+ keep all table info in a single .tableinfo file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5833) 0.92 build has been failing pretty consistently on TestMasterFailover....
[ https://issues.apache.org/jira/browse/HBASE-5833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258662#comment-13258662 ] stack commented on HBASE-5833: -- Looking at this more, it seems like the actual TMF tests run pretty fast but then the test gets stuck a long time finishing up on the end... like 10x more time finishing than was spent running the test. This long cleanup seems to be our 'timeout'. In TMF case, digging, there are orphan logsyncer threads outstanding trying to sync a fileystem that has been since closed. This seems to be part of the issue (There are other orphan threads -- executors). TMF does HRegion.createRegions but it doesn't pass in a WAL to use in creation. In this case, this method makes an individual WAL for the create use. In TMF, and in a bunch of other tests, this hlog is never closed leaving logsyncer threads running (and some executors). Will attach a patch that goes through all tests and does close and cleanup of wals that use this special createRegions method (there are a bunch). 0.92 build has been failing pretty consistently on TestMasterFailover - Key: HBASE-5833 URL: https://issues.apache.org/jira/browse/HBASE-5833 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.92.2 Attachments: 5833.txt Trunk seems fine but 0.92 fails on this test pretty regularly. Running it local it seems to hang for me. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5833) 0.92 build has been failing pretty consistently on TestMasterFailover....
[ https://issues.apache.org/jira/browse/HBASE-5833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5833: - Attachment: closehregions.txt This patch is for 0.92 only; thats where I'm doing investigation. Doesn't apply to trunk yet. 0.92 build has been failing pretty consistently on TestMasterFailover - Key: HBASE-5833 URL: https://issues.apache.org/jira/browse/HBASE-5833 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.92.2 Attachments: 5833.txt, closehregions.txt Trunk seems fine but 0.92 fails on this test pretty regularly. Running it local it seems to hang for me. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5833) 0.92 build has been failing pretty consistently on TestMasterFailover....
[ https://issues.apache.org/jira/browse/HBASE-5833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HBASE-5833: -- Reopen 0.92 build has been failing pretty consistently on TestMasterFailover - Key: HBASE-5833 URL: https://issues.apache.org/jira/browse/HBASE-5833 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.92.2 Attachments: 5833.txt, closehregions.txt Trunk seems fine but 0.92 fails on this test pretty regularly. Running it local it seems to hang for me. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5652) [findbugs] Fix lock release on all paths
[ https://issues.apache.org/jira/browse/HBASE-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gregory Chanan updated HBASE-5652: -- Status: Patch Available (was: Open) [findbugs] Fix lock release on all paths - Key: HBASE-5652 URL: https://issues.apache.org/jira/browse/HBASE-5652 Project: HBase Issue Type: Sub-task Components: scripts Reporter: Jonathan Hsieh Assignee: Gregory Chanan Attachments: HBASE-5652-v0.patch See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_MT_CORRECTNESS Category UL -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5652) [findbugs] Fix lock release on all paths
[ https://issues.apache.org/jira/browse/HBASE-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gregory Chanan updated HBASE-5652: -- Attachment: HBASE-5652-v0.patch * Attacked HBASE-5652-v0.patch * Attached patch that handles all the UL issues except two, which are on the exception list (I haven't tested the exception list yet, we'll see if it works in Hadoop QA). Those two are startRegionOperation and startBulkRegionOperation, which are correct (they release the lock and throw an exception if the region is closed, otherwise, hold the lock and continue processing. [findbugs] Fix lock release on all paths - Key: HBASE-5652 URL: https://issues.apache.org/jira/browse/HBASE-5652 Project: HBase Issue Type: Sub-task Components: scripts Reporter: Jonathan Hsieh Assignee: Gregory Chanan Attachments: HBASE-5652-v0.patch See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_MT_CORRECTNESS Category UL -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
Lars Hofhansl created HBASE-5848: Summary: Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5045) Add the table name and cf name for the next call int the task monitor
[ https://issues.apache.org/jira/browse/HBASE-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5045: --- Attachment: D2913.1.patch mbautin requested code review of [jira] [HBASE-5045] Annotation for Custom Param formatting and next() RPC call info. Reviewers: amirshim, Liyin, tedyu, stack, JIRA Porting Amir's fix from 89-fb. His original summary below. A method for associating pretty print classes with method calls. These allow you to get information about a method call given the params it was called with and what instance it was called on. The first use case is for getting info about a next() RPC call. TEST PLAN Original test plan from Amir: run a script that stresses a regionserver with scan and next() scans, and check that the information is show in the JSON view of the TaskMonitor REVISION DETAIL https://reviews.facebook.net/D2913 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java src/main/java/org/apache/hadoop/hbase/monitoring/MonitoredRPCHandler.java src/main/java/org/apache/hadoop/hbase/monitoring/MonitoredRPCHandlerImpl.java src/main/java/org/apache/hadoop/hbase/monitoring/MonitoredTask.java src/main/java/org/apache/hadoop/hbase/monitoring/MonitoredTaskImpl.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/main/java/org/apache/hadoop/hbase/util/ParamFormat.java src/main/java/org/apache/hadoop/hbase/util/ParamFormatHelper.java src/main/java/org/apache/hadoop/hbase/util/ParamFormatter.java src/test/java/org/apache/hadoop/hbase/util/TestParamFormatter.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/6645/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. Add the table name and cf name for the next call int the task monitor - Key: HBASE-5045 URL: https://issues.apache.org/jira/browse/HBASE-5045 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Amir Shimoni Attachments: D2913.1.patch In the task monitor, we don't have much information about the next call compared to other operations. It would be nice to add the table name and cf name for each next call in the task monitor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3792) TableInputFormat leaks ZK connections
[ https://issues.apache.org/jira/browse/HBASE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258696#comment-13258696 ] Patrick Yu commented on HBASE-3792: --- One possible workaround for this issue is to reuse the connection for all jobs. However, hbase.connection.per.config must be set to false for connection sharing to work. This flag defaults to true up until HBASE-4508, though. TableInputFormat leaks ZK connections - Key: HBASE-3792 URL: https://issues.apache.org/jira/browse/HBASE-3792 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.1 Environment: Java 1.6.0_24, Mac OS X 10.6.7 Reporter: Bryan Keller Attachments: patch0.90.4, tableinput.patch The TableInputFormat creates an HTable using a new Configuration object, and it never cleans it up. When running a Mapper, the TableInputFormat is instantiated and the ZK connection is created. While this connection is not explicitly cleaned up, the Mapper process eventually exits and thus the connection is closed. Ideally the TableRecordReader would close the connection in its close() method rather than relying on the process to die for connection cleanup. This is fairly easy to implement by overriding TableRecordReader, and also overriding TableInputFormat to specify the new record reader. The leak occurs when the JobClient is initializing and needs to retrieves the splits. To get the splits, it instantiates a TableInputFormat. Doing so creates a ZK connection that is never cleaned up. Unlike the mapper, however, my job client process does not die. Thus the ZK connections accumulate. I was able to fix the problem by writing my own TableInputFormat that does not initialize the HTable in the getConf() method and does not have an HTable member variable. Rather, it has a variable for the table name. The HTable is instantiated where needed and then cleaned up. For example, in the getSplits() method, I create the HTable, then close the connection once the splits are retrieved. I also create the HTable when creating the record reader, and I have a record reader that closes the connection when done. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5652) [findbugs] Fix lock release on all paths
[ https://issues.apache.org/jira/browse/HBASE-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258698#comment-13258698 ] Hadoop QA commented on HBASE-5652: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12523581/HBASE-5652-v0.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1596//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1596//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1596//console This message is automatically generated. [findbugs] Fix lock release on all paths - Key: HBASE-5652 URL: https://issues.apache.org/jira/browse/HBASE-5652 Project: HBase Issue Type: Sub-task Components: scripts Reporter: Jonathan Hsieh Assignee: Gregory Chanan Attachments: HBASE-5652-v0.patch See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_MT_CORRECTNESS Category UL -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5045) Add the table name and cf name for the next call int the task monitor
[ https://issues.apache.org/jira/browse/HBASE-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HBASE-5045: -- Status: Patch Available (was: Open) Add the table name and cf name for the next call int the task monitor - Key: HBASE-5045 URL: https://issues.apache.org/jira/browse/HBASE-5045 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Amir Shimoni Attachments: D2913.1.patch, D2913.2.patch In the task monitor, we don't have much information about the next call compared to other operations. It would be nice to add the table name and cf name for each next call in the task monitor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5045) Add the table name and cf name for the next call int the task monitor
[ https://issues.apache.org/jira/browse/HBASE-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5045: --- Attachment: D2913.2.patch mbautin updated the revision [jira] [HBASE-5045] Annotation for Custom Param formatting and next() RPC call info. Reviewers: amirshim, Liyin, tedyu, stack, JIRA Fixing a bug in my port (not present in the original patch) that broke RPC. REVISION DETAIL https://reviews.facebook.net/D2913 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java src/main/java/org/apache/hadoop/hbase/ipc/WritableRpcEngine.java src/main/java/org/apache/hadoop/hbase/monitoring/MonitoredRPCHandler.java src/main/java/org/apache/hadoop/hbase/monitoring/MonitoredRPCHandlerImpl.java src/main/java/org/apache/hadoop/hbase/monitoring/MonitoredTask.java src/main/java/org/apache/hadoop/hbase/monitoring/MonitoredTaskImpl.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java src/main/java/org/apache/hadoop/hbase/util/ParamFormat.java src/main/java/org/apache/hadoop/hbase/util/ParamFormatHelper.java src/main/java/org/apache/hadoop/hbase/util/ParamFormatter.java src/test/java/org/apache/hadoop/hbase/util/TestParamFormatter.java Add the table name and cf name for the next call int the task monitor - Key: HBASE-5045 URL: https://issues.apache.org/jira/browse/HBASE-5045 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Amir Shimoni Attachments: D2913.1.patch, D2913.2.patch In the task monitor, we don't have much information about the next call compared to other operations. It would be nice to add the table name and cf name for each next call in the task monitor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5045) Add the table name and cf name for the next call int the task monitor
[ https://issues.apache.org/jira/browse/HBASE-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258722#comment-13258722 ] Phabricator commented on HBASE-5045: tedyu has commented on the revision [jira] [HBASE-5045] Annotation for Custom Param formatting and next() RPC call info. WritableRpcEngine.java is going to be replaced in trunk. Would another issue be filed to make this work with PB-based RPC ? All the catch clause should be on the same line as the preceding closing brace. INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java:193 'a RPC call' - 'an RPC call' src/main/java/org/apache/hadoop/hbase/monitoring/MonitoredTask.java:22 This import should be after line 27. src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:3488 This should be package private. src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java:2217 Either return res here or move line 2216 after this line. src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java:2206 '1 and 2 parameter' - '1-parameter and 2-parameter' src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java: Can getOriginalScan() be declared in RegionScanner ? It doesn't look nice referencing an implementation class. src/main/java/org/apache/hadoop/hbase/util/ParamFormat.java:2 No year is needed. src/main/java/org/apache/hadoop/hbase/util/ParamFormat.java:31 DEFAULT can be detailed, right ? How about naming DEFAULT CONCISE ? src/main/java/org/apache/hadoop/hbase/util/ParamFormatHelper.java:2 No year, please. src/main/java/org/apache/hadoop/hbase/util/ParamFormatHelper.java:72 Insert a space after catch Put this on line 71. src/main/java/org/apache/hadoop/hbase/util/ParamFormatHelper.java:77 Put this on line 76. src/main/java/org/apache/hadoop/hbase/util/ParamFormatHelper.java:74 Insert a comma before the space of make sure src/main/java/org/apache/hadoop/hbase/util/ParamFormatter.java:2 No year, please. src/test/java/org/apache/hadoop/hbase/util/TestParamFormatter.java:1 License, please. src/test/java/org/apache/hadoop/hbase/util/TestParamFormatter.java:16 Add test category. REVISION DETAIL https://reviews.facebook.net/D2913 Add the table name and cf name for the next call int the task monitor - Key: HBASE-5045 URL: https://issues.apache.org/jira/browse/HBASE-5045 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Amir Shimoni Attachments: D2913.1.patch, D2913.2.patch In the task monitor, we don't have much information about the next call compared to other operations. It would be nice to add the table name and cf name for each next call in the task monitor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5803) [89-fb] Upgrade hbase 0.89-fb to Thrift 0.8.0 and bring Thrift server enhancements from trunk
[ https://issues.apache.org/jira/browse/HBASE-5803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin resolved HBASE-5803. --- Resolution: Fixed Committed internally. [89-fb] Upgrade hbase 0.89-fb to Thrift 0.8.0 and bring Thrift server enhancements from trunk - Key: HBASE-5803 URL: https://issues.apache.org/jira/browse/HBASE-5803 Project: HBase Issue Type: Improvement Reporter: Mikhail Bautin Assignee: Mikhail Bautin Attachments: D2811.1.patch, D2811.2.patch, D2811.3.patch, D2811.4.patch, D2811.5.patch TBoundedThreadPoolServer has been a problem for us when there is a large number of clients. We need to migrate to 0.8.0. in 89-fb and bring the relevant improvements from trunk, including supporting TThreadedSelectorServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5104) Provide a reliable intra-row pagination mechanism
[ https://issues.apache.org/jira/browse/HBASE-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258732#comment-13258732 ] Phabricator commented on HBASE-5104: mbautin has commented on the revision [jira] [HBASE-5104] Provide a reliable intra-row pagination mechanism. Ping. REVISION DETAIL https://reviews.facebook.net/D2799 Provide a reliable intra-row pagination mechanism - Key: HBASE-5104 URL: https://issues.apache.org/jira/browse/HBASE-5104 Project: HBase Issue Type: Bug Reporter: Kannan Muthukkaruppan Assignee: Madhuwanti Vaidya Attachments: D2799.1.patch, D2799.2.patch, D2799.3.patch, jira-HBASE-5104-Provide-a-reliable-intra-row-paginat-2012-04-16_12_39_42.patch, testFilterList.rb Addendum: Doing pagination (retrieving at most limit number of KVs at a particular offset) is currently supported via the ColumnPaginationFilter. However, it is not a very clean way of supporting pagination. Some of the problems with it are: * Normally, one would expect a query with (Filter(A) AND Filter(B)) to have same results as (query with Filter(A)) INTERSECT (query with Filter(B)). This is not the case for ColumnPaginationFilter as its internal state gets updated depending on whether or not Filter(A) returns TRUE/FALSE for a particular cell. * When this Filter is used in combination with other filters (e.g., doing AND with another filter using FilterList), the behavior of the query depends on the order of filters in the FilterList. This is not ideal. * ColumnPaginationFilter is a stateful filter which ends up counting multiple versions of the cell as separate values even if another filter upstream or the ScanQueryMatcher is going to reject the value for other reasons. Seems like we need a reliable way to do pagination. The particular use case that prompted this JIRA is pagination within the same rowKey. For example, for a given row key R, get columns with prefix P, starting at offset X (among columns which have prefix P) and limit Y. Some possible fixes might be: 1) enhance ColumnPrefixFilter to support another constructor which supports limit/offset. 2) Support pagination (limit/offset) at the Scan/Get API level (rather than as a filter) [Like SQL]. Original Post: Thanks Jiakai Liu for reporting this issue and doing the initial investigation. Email from Jiakai below: Assuming that we have an index column family with the following entries: tag0:001:thread1 ... tag1:001:thread1 tag1:002:thread2 ... tag1:010:thread10 ... tag2:001:thread1 tag2:005:thread5 ... To get threads with tag1 in range [5, 10), I tried the following code: ColumnPrefixFilter filter1 = new ColumnPrefixFilter(Bytes.toBytes(tag1)); ColumnPaginationFilter filter2 = new ColumnPaginationFilter(5 /* limit */, 5 /* offset */); FilterList filters = new FilterList(Operator.MUST_PASS_ALL); filters.addFilter(filter1); filters.addFilter(filter2); Get get = new Get(USER); get.addFamily(COLUMN_FAMILY); get.setMaxVersions(1); get.setFilter(filters); Somehow it didn't work as expected. It returned the entries as if the filter1 were not set. Turns out the ColumnPrefixFilter returns SEEK_NEXT_USING_HINT in some cases. The FilterList filter does not handle this return code properly (treat it as INCLUDE). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3508) LruBlockCache statistics thread should have a name
[ https://issues.apache.org/jira/browse/HBASE-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258736#comment-13258736 ] Phabricator commented on HBASE-3508: mbautin has committed the revision [jira] [HBASE-3508] [89-fb] LruBlockCache statistics thread should have a name. REVISION DETAIL https://reviews.facebook.net/D2853 COMMIT https://reviews.facebook.net/rHBASEEIGHTNINEFBBRANCH1328561 LruBlockCache statistics thread should have a name -- Key: HBASE-3508 URL: https://issues.apache.org/jira/browse/HBASE-3508 Project: HBase Issue Type: Improvement Reporter: Todd Lipcon Assignee: Todd Lipcon Priority: Trivial Fix For: 0.90.1 Attachments: D2853.1.patch, D2853.2.patch, D2853.3.patch, hbase-3508.txt Currently it's just an unnamed threadpool. It's annoying. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5803) [89-fb] Upgrade hbase 0.89-fb to Thrift 0.8.0 and bring Thrift server enhancements from trunk
[ https://issues.apache.org/jira/browse/HBASE-5803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258737#comment-13258737 ] Phabricator commented on HBASE-5803: mbautin has committed the revision [jira] [HBASE-5803] [89-fb] Upgrade hbase 0.89-fb to Thrift 0.8.0 and bring Thrift server enhancements from trunk. REVISION DETAIL https://reviews.facebook.net/D2811 COMMIT https://reviews.facebook.net/rHBASEEIGHTNINEFBBRANCH1328562 [89-fb] Upgrade hbase 0.89-fb to Thrift 0.8.0 and bring Thrift server enhancements from trunk - Key: HBASE-5803 URL: https://issues.apache.org/jira/browse/HBASE-5803 Project: HBase Issue Type: Improvement Reporter: Mikhail Bautin Assignee: Mikhail Bautin Attachments: D2811.1.patch, D2811.2.patch, D2811.3.patch, D2811.4.patch, D2811.5.patch TBoundedThreadPoolServer has been a problem for us when there is a large number of clients. We need to migrate to 0.8.0. in 89-fb and bring the relevant improvements from trunk, including supporting TThreadedSelectorServer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5849) On first cluster startup, RS aborts if root znode is not available
Enis Soztutar created HBASE-5849: Summary: On first cluster startup, RS aborts if root znode is not available Key: HBASE-5849 URL: https://issues.apache.org/jira/browse/HBASE-5849 Project: HBase Issue Type: Bug Components: master, regionserver, zookeeper Affects Versions: 0.92.2, 0.96.0, 0.94.1 Reporter: Enis Soztutar Assignee: Enis Soztutar When launching a fresh new cluster, the master has to be started first, which might create race conditions for starting master and rs at the same time. Master startup code is smt like this: - establish zk connection - create root znodes in zk (/hbase) - create ephemeral node for master /hbase/master, Region server start up code is smt like this: - establish zk connection - check whether the root znode (/hbase) is there. If not, shutdown. - wait for the master to create znodes /hbase/master So, the problem is on the very first launch of the cluster, RS aborts to start since /hbase znode might not have been created yet (only the master creates it if needed). Since /hbase/ is not deleted on cluster shutdown, on subsequent cluster starts, it does not matter which order the servers are started. So this affects only first launchs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3792) TableInputFormat leaks ZK connections
[ https://issues.apache.org/jira/browse/HBASE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258739#comment-13258739 ] Lars Hofhansl commented on HBASE-3792: -- @Patrik: In 0.92+ you can create a standalone HConnection (HConnectionManager.createConnection(...)) and then create HTables using that HConnection. See HBASE-4805. TableInputFormat leaks ZK connections - Key: HBASE-3792 URL: https://issues.apache.org/jira/browse/HBASE-3792 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.1 Environment: Java 1.6.0_24, Mac OS X 10.6.7 Reporter: Bryan Keller Attachments: patch0.90.4, tableinput.patch The TableInputFormat creates an HTable using a new Configuration object, and it never cleans it up. When running a Mapper, the TableInputFormat is instantiated and the ZK connection is created. While this connection is not explicitly cleaned up, the Mapper process eventually exits and thus the connection is closed. Ideally the TableRecordReader would close the connection in its close() method rather than relying on the process to die for connection cleanup. This is fairly easy to implement by overriding TableRecordReader, and also overriding TableInputFormat to specify the new record reader. The leak occurs when the JobClient is initializing and needs to retrieves the splits. To get the splits, it instantiates a TableInputFormat. Doing so creates a ZK connection that is never cleaned up. Unlike the mapper, however, my job client process does not die. Thus the ZK connections accumulate. I was able to fix the problem by writing my own TableInputFormat that does not initialize the HTable in the getConf() method and does not have an HTable member variable. Rather, it has a variable for the table name. The HTable is instantiated where needed and then cleaned up. For example, in the getSplits() method, I create the HTable, then close the connection once the splits are retrieved. I also create the HTable when creating the record reader, and I have a record reader that closes the connection when done. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258740#comment-13258740
]
Lars Hofhansl commented on HBASE-5848:
--
Specifically
{code}
admin.createTable(htd, byte[][] {HConstants.EMPTY_START_ROW})
{code}
Will cause the HMaster to abort.
Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to
abort
Key: HBASE-5848
URL: https://issues.apache.org/jira/browse/HBASE-5848
Project: HBase
Issue Type: Bug
Components: client
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
A coworker of mine just had this scenario. It does not make sense the
EMPTY_START_ROW as splitKey (since the region with the empty start key is
implicit), but it should not cause the HMaster to abort.
The abort happens because it tries to bulk assign the same region twice and
then runs into race conditions with ZK.
The same would (presumably) happen when two identical split keys are passed,
but the client blocks that. The simplest solution here is to also block
passed null or EMPTY_START_ROW as split key by the client.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5104) Provide a reliable intra-row pagination mechanism
[ https://issues.apache.org/jira/browse/HBASE-5104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258742#comment-13258742 ] Zhihong Yu commented on HBASE-5104: --- The previous Hadoop QA run stumbled over TestLoadIncrementalHFilesSplitRecovery Please resubmit patch for QA. Provide a reliable intra-row pagination mechanism - Key: HBASE-5104 URL: https://issues.apache.org/jira/browse/HBASE-5104 Project: HBase Issue Type: Bug Reporter: Kannan Muthukkaruppan Assignee: Madhuwanti Vaidya Attachments: D2799.1.patch, D2799.2.patch, D2799.3.patch, jira-HBASE-5104-Provide-a-reliable-intra-row-paginat-2012-04-16_12_39_42.patch, testFilterList.rb Addendum: Doing pagination (retrieving at most limit number of KVs at a particular offset) is currently supported via the ColumnPaginationFilter. However, it is not a very clean way of supporting pagination. Some of the problems with it are: * Normally, one would expect a query with (Filter(A) AND Filter(B)) to have same results as (query with Filter(A)) INTERSECT (query with Filter(B)). This is not the case for ColumnPaginationFilter as its internal state gets updated depending on whether or not Filter(A) returns TRUE/FALSE for a particular cell. * When this Filter is used in combination with other filters (e.g., doing AND with another filter using FilterList), the behavior of the query depends on the order of filters in the FilterList. This is not ideal. * ColumnPaginationFilter is a stateful filter which ends up counting multiple versions of the cell as separate values even if another filter upstream or the ScanQueryMatcher is going to reject the value for other reasons. Seems like we need a reliable way to do pagination. The particular use case that prompted this JIRA is pagination within the same rowKey. For example, for a given row key R, get columns with prefix P, starting at offset X (among columns which have prefix P) and limit Y. Some possible fixes might be: 1) enhance ColumnPrefixFilter to support another constructor which supports limit/offset. 2) Support pagination (limit/offset) at the Scan/Get API level (rather than as a filter) [Like SQL]. Original Post: Thanks Jiakai Liu for reporting this issue and doing the initial investigation. Email from Jiakai below: Assuming that we have an index column family with the following entries: tag0:001:thread1 ... tag1:001:thread1 tag1:002:thread2 ... tag1:010:thread10 ... tag2:001:thread1 tag2:005:thread5 ... To get threads with tag1 in range [5, 10), I tried the following code: ColumnPrefixFilter filter1 = new ColumnPrefixFilter(Bytes.toBytes(tag1)); ColumnPaginationFilter filter2 = new ColumnPaginationFilter(5 /* limit */, 5 /* offset */); FilterList filters = new FilterList(Operator.MUST_PASS_ALL); filters.addFilter(filter1); filters.addFilter(filter2); Get get = new Get(USER); get.addFamily(COLUMN_FAMILY); get.setMaxVersions(1); get.setFilter(filters); Somehow it didn't work as expected. It returned the entries as if the filter1 were not set. Turns out the ColumnPrefixFilter returns SEEK_NEXT_USING_HINT in some cases. The FilterList filter does not handle this return code properly (treat it as INCLUDE). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5045) Add the table name and cf name for the next call int the task monitor
[ https://issues.apache.org/jira/browse/HBASE-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258746#comment-13258746 ] Hadoop QA commented on HBASE-5045: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12523605/D2913.2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1597//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1597//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1597//console This message is automatically generated. Add the table name and cf name for the next call int the task monitor - Key: HBASE-5045 URL: https://issues.apache.org/jira/browse/HBASE-5045 Project: HBase Issue Type: Improvement Reporter: Liyin Tang Assignee: Amir Shimoni Attachments: D2913.1.patch, D2913.2.patch In the task monitor, we don't have much information about the next call compared to other operations. It would be nice to add the table name and cf name for each next call in the task monitor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5652) [findbugs] Fix lock release on all paths
[
https://issues.apache.org/jira/browse/HBASE-5652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258763#comment-13258763
]
Zhihong Yu commented on HBASE-5652:
---
{code}
+if(status != null) status.cleanup();
{code}
Please insert space between if and (.
{code}
+ try {
+this.logRollRunning = false;
+ } finally {
+this.cacheFlushLock.unlock();
+ }
{code}
The assignment wouldn't throw exception. Is the above try block needed ?
[findbugs] Fix lock release on all paths
-
Key: HBASE-5652
URL: https://issues.apache.org/jira/browse/HBASE-5652
Project: HBase
Issue Type: Sub-task
Components: scripts
Reporter: Jonathan Hsieh
Assignee: Gregory Chanan
Attachments: HBASE-5652-v0.patch
See
https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_MT_CORRECTNESS
Category UL
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5850) Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized
xufeng created HBASE-5850: - Summary: Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized Key: HBASE-5850 URL: https://issues.apache.org/jira/browse/HBASE-5850 Project: HBase Issue Type: Bug Reporter: xufeng Assignee: xufeng This issue is needed in 0.90 0.92 also. And update the hbase-5454 patch that add the checkInitialized() into HMaster#createTable(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5677) The master never does balance because duplicate openhandled the one region
[ https://issues.apache.org/jira/browse/HBASE-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258774#comment-13258774 ] xufeng commented on HBASE-5677: --- @stack see https://issues.apache.org/jira/browse/HBASE-5850 The master never does balance because duplicate openhandled the one region -- Key: HBASE-5677 URL: https://issues.apache.org/jira/browse/HBASE-5677 Project: HBase Issue Type: Bug Affects Versions: 0.90.6 Environment: 0.90 Reporter: xufeng Assignee: xufeng Fix For: 0.90.7, 0.92.2 Attachments: 5677-proposal.txt, 5677-proposal.txt, Backport-HBASE-5454-to-90.patch, Backport-HBASE-5454-to-92.patch, HBASE-5677-90-v1.patch, surefire-report_no_patched_v1.html, surefire-report_patched_v1.html If region be assigned When the master is doing initialization(before do processFailover),the region will be duplicate openhandled. because the unassigned node in zookeeper will be handled again in AssignmentManager#processFailover() it cause the region in RIT,thus the master never does balance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5850) Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized
[
https://issues.apache.org/jira/browse/HBASE-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xufeng updated HBASE-5850:
--
Attachment: backport-5454-to-90.patch
backport-5454-to-92_surefire-report.html
backport-5454-to-92.patch
backport-5454(createTable)-to-94_surefire-report.html
backport-5454(createTable)-to-94.patch
backport-5454(createTable)-to-trunk_surefire-report.html
backport-5454(createTable)-to-trunk.patch
@stack
All patch have been verified in real cluster.
The error of unit test in trunk be verified I think it is no relation whit the
patch.
I did not submit the 90 test result,because I meeted this problem:
${noformat}
Running org.apache.hadoop.hbase.master.TestMasterFailover
killed.
${noformat}
This problem also exist if there is no the patch.
Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master
is initialized
---
Key: HBASE-5850
URL: https://issues.apache.org/jira/browse/HBASE-5850
Project: HBase
Issue Type: Bug
Reporter: xufeng
Assignee: xufeng
Attachments: backport-5454(createTable)-to-94.patch,
backport-5454(createTable)-to-94_surefire-report.html,
backport-5454(createTable)-to-trunk.patch,
backport-5454(createTable)-to-trunk_surefire-report.html,
backport-5454-to-90.patch, backport-5454-to-92.patch,
backport-5454-to-92_surefire-report.html
This issue is needed in 0.90 0.92 also.
And update the hbase-5454 patch that add the checkInitialized() into
HMaster#createTable().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
