[jira] [Commented] (HBASE-8656) Rpc call may not be notified in SecureClient
[
https://issues.apache.org/jira/browse/HBASE-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672324#comment-13672324
]
cuijianwei commented on HBASE-8656:
---
[~andrew.purt...@gmail.com], Thanks for your comment. I make a patch and test
it for the following two cases.
Firstly, we modify the region server code to return Status.FATAL to
SecureClient; Secondly, we only return call id in the response to make
SecureClient receive a SocketTimeout exception after removing call. In both
cases, the SecureClient won't notify the waiting call before apply the patch.
After applying the patch, it seems ok for the two cases.
Rpc call may not be notified in SecureClient
Key: HBASE-8656
URL: https://issues.apache.org/jira/browse/HBASE-8656
Project: HBase
Issue Type: Bug
Components: Client, IPC/RPC, security
Affects Versions: 0.94.7
Reporter: cuijianwei
Attachments: HBASE-8656-0.94-v1.txt
In SecureClient.java, rpc responses will be processed by receiveResponse()
which looks like:
{code}
try {
int id = in.readInt();// try to read an id
if (LOG.isDebugEnabled())
LOG.debug(getName() + got value # + id);
Call call = calls.remove(id);
int state = in.readInt(); // read call status
if (LOG.isDebugEnabled()) {
LOG.debug(call #+id+ state is + state);
}
if (state == Status.SUCCESS.state) {
Writable value = ReflectionUtils.newInstance(valueClass, conf);
value.readFields(in); // read value
if (LOG.isDebugEnabled()) {
LOG.debug(call #+id+, response is:\n+value.toString());
}
// it's possible that this call may have been cleaned up due to a
RPC
// timeout, so check if it still exists before setting the value.
if (call != null) {
call.setValue(value);
}
} else if (state == Status.ERROR.state) {
if (call != null) {
call.setException(new
RemoteException(WritableUtils.readString(in), WritableUtils
.readString(in)));
}
} else if (state == Status.FATAL.state) {
// Close the connection
markClosed(new RemoteException(WritableUtils.readString(in),
WritableUtils.readString(in)));
}
} catch (IOException e) {
if (e instanceof SocketTimeoutException remoteId.rpcTimeout 0) {
// Clean up open calls but don't treat this as a fatal condition,
// since we expect certain responses to not make it by the specified
// {@link ConnectionId#rpcTimeout}.
closeException = e;
} else {
// Since the server did not respond within the default ping interval
// time, treat this as a fatal condition and close this connection
markClosed(e);
}
} finally {
if (remoteId.rpcTimeout 0) {
cleanupCalls(remoteId.rpcTimeout);
}
}
}
{code}
In above code, in the try block, the call will be firstly removed from call
map by:
{code}
Call call = calls.remove(id);
{code}
There may be two cases leading the call couldn't be notified and the invoking
thread will wait forever.
Firstly, if the returned status is Status.FATAL.state by:
{code}
int state = in.readInt(); // read call status
{code}
The code will come into:
{code}
} else if (state == Status.FATAL.state) {
// Close the connection
markClosed(new RemoteException(WritableUtils.readString(in),
WritableUtils.readString(in)));
}
{code}
Here, the SecureConnection is marked as closed and all rpc calls in call map
of this connection will be notified to receive an exception. However, the
current rpc call has been removed from the call map, it won't be notified.
Secondly, after the call has been removed by:
{code}
Call call = calls.remove(id);
{code}
If we encounter any exception before the 'try' block finished, the code will
come into 'catch' and 'finally' block, neither 'catch' block nor 'finally'
block will notify the rpc call because it has been removed from call map.
Compared with receiveResponse() in HBaseClient.java, it may be better to get
the rpc call from call map and remove it at the end of the 'try' block.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8652) Number of compacting KVs is not reset at the end of compaction
[
https://issues.apache.org/jira/browse/HBASE-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672331#comment-13672331
]
samar commented on HBASE-8652:
--
Can we instead just change Heading on the master status page to someting like
Total compaction KVs.
or
instead of resting to show that its done.. We could gracefully display
(totalCompactingKVs - currentCompactedKVs) which would finally become 0 after
its 100% complete
Number of compacting KVs is not reset at the end of compaction
--
Key: HBASE-8652
URL: https://issues.apache.org/jira/browse/HBASE-8652
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
Looking at master:60010/master-status#compactStas , I noticed that 'Num.
Compacting KVs' column stays unchanged at non-zero value(s).
In DefaultCompactor#compact(), we have this at the beginning:
{code}
this.progress = new CompactionProgress(fd.maxKeyCount);
{code}
But progress.totalCompactingKVs is not reset at the end of compact().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7462) TestDrainingServer is an integration test. It should be a unit test instead
[ https://issues.apache.org/jira/browse/HBASE-7462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672484#comment-13672484 ] Gustavo Anatoly commented on HBASE-7462: Hi, Nicolas. After long time, I have some code. Could you please review the test? http://goo.gl/1WVj3 Thankful. TestDrainingServer is an integration test. It should be a unit test instead --- Key: HBASE-7462 URL: https://issues.apache.org/jira/browse/HBASE-7462 Project: HBase Issue Type: Wish Components: test Affects Versions: 0.95.2 Reporter: Nicolas Liochon Priority: Trivial Labels: noob TestDrainingServer tests the function that allows to say that a regionserver should not get new regions. As it is written today, it's an integration test: it starts stops a cluster. The test would be more efficient if it would just check that the AssignmentManager does not use the drained region server; whatever the circumstances (bulk assign or not for example). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[ https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672490#comment-13672490 ] stack commented on HBASE-8645: -- Any chance of a review of this approach? Thanks lads. Change ServerName so it uses hostname only, not FQDN hostnames --- Key: HBASE-8645 URL: https://issues.apache.org/jira/browse/HBASE-8645 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Attachments: 8645.txt, 8645.txt, 8645v2.txt, 8645v3.txt, 8645v4.txt, 8645v5.txt No need of dragging around domain part of a hostname when we make ServerNames. Rather than do a.example.org,6,12345 as we currently do, just output: 1,6,12345. This will make names displayed in UI and or shell smaller. Will also tighten up our logs a little, especially where ServerName is part of a thrad name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8645) Change ServerName so it uses hostname only, not FQDN hostnames
[
https://issues.apache.org/jira/browse/HBASE-8645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672491#comment-13672491
]
stack commented on HBASE-8645:
--
The SeverName.toShortString drops the startcode and intentionally makes a
String that does not look like a ServerName#toString, rather it is a host+port
only host is missing domain. Here is sample of what logging looks like now:
{code}
2013-05-31 02:25:03,091 DEBUG [M:0;asf000:41822] zookeeper.ZKUtil(433):
master:41822-0x13ef867090c Set watcher on znode that does not yet exist,
/hbase/running
2013-05-31 02:25:03,115 WARN [M:0;asf000:41822] hbase.ZNodeClearer(57):
Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on
crash by start scripts (Longer MTTR!)
2013-05-31 02:25:03,115 INFO [M:0;asf000:41822]
master.ActiveMasterManager(170): Registered Active
Master=asf000.sp2.ygridcore.net,41822,1369967102774
2013-05-31 02:25:03,123 INFO [M:0;asf000:41822] master.SplitLogManager(201):
timeout=12, unassigned timeout=18
2013-05-31 02:25:03,124 INFO [M:0;asf000:41822] master.SplitLogManager(210):
distributedLogReplay = false
2013-05-31 02:25:03,127 INFO [M:0;asf000:41822] master.SplitLogManager(1082):
Found 0 orphan tasks and 0 rescan nodes
2013-05-31 02:25:03,182 DEBUG [M:0;asf000:41822] util.FSUtils(757): Created
cluster ID file at hdfs://localhost:52392/user/jenkins/hbase/hbase.id with ID:
df6e7248-191d-479e-86da-6bd2a844e00b
2013-05-31 02:25:03,189 INFO [pool-1-thread-1] regionserver.ShutdownHook(87):
Installed shutdown hook thread: Shutdownhook:RS:0;asf000:47701
2013-05-31 02:25:03,190 INFO [pool-1-thread-1] regionserver.ShutdownHook(87):
Installed shutdown hook thread: Shutdownhook:RS:1;asf000:48644
2013-05-31 02:25:03,192 INFO [pool-1-thread-1] regionserver.ShutdownHook(87):
Installed shutdown hook thread: Shutdownhook:RS:2;asf000:47305
2013-05-31 02:25:03,198 INFO [RS:1;asf000:48644]
zookeeper.RecoverableZooKeeper(119): Process identifier=regionserver:48644
connecting to ZooKeeper ensemble=localhost:54561
2013-05-31 02:25:03,201 INFO [RS:2;asf000:47305]
zookeeper.RecoverableZooKeeper(119): Process identifier=regionserver:47305
connecting to ZooKeeper ensemble=localhost:54561
2013-05-31 02:25:03,204 INFO [RS:0;asf000:47701]
zookeeper.RecoverableZooKeeper(119): Process identifier=regionserver:47701
connecting to ZooKeeper ensemble=localhost:54561
2013-05-31 02:25:03,207 DEBUG [RS:1;asf000:48644-EventThread]
zookeeper.ZooKeeperWatcher(307): regionserver:48644 Received ZooKeeper Event,
type=None, state=SyncConnected, path=null
2013-05-31 02:25:03,207 DEBUG [RS:2;asf000:47305-EventThread]
zookeeper.ZooKeeperWatcher(307): regionserver:47305 Received ZooKeeper Event,
type=None, state=SyncConnected, path=null
2013-05-31 02:25:03,209 DEBUG [RS:1;asf000:48644-EventThread]
zookeeper.ZooKeeperWatcher(384): regionserver:48644-0x13ef867090c0001 connected
2013-05-31 02:25:03,210 DEBUG [RS:2;asf000:47305-EventThread]
zookeeper.ZooKeeperWatcher(384): regionserver:47305-0x13ef867090c0002 connected
2013-05-31 02:25:03,211 DEBUG [RS:0;asf000:47701-EventThread]
zookeeper.ZooKeeperWatcher(307): regionserver:47701 Received ZooKeeper Event,
type=None, state=SyncConnected, path=null
2013-05-31 02:25:03,212 DEBUG [RS:0;asf000:47701-EventThread]
zookeeper.ZooKeeperWatcher(384): regionserver:47701-0x13ef867090c0003 connected
2013-05-31 02:25:03,212 DEBUG [RS:1;asf000:48644] zookeeper.ZKUtil(431):
regionserver:48644-0x13ef867090c0001 Set watcher on existing znode=/hbase/master
2013-05-31 02:25:03,213 DEBUG [RS:2;asf000:47305] zookeeper.ZKUtil(431):
regionserver:47305-0x13ef867090c0002 Set watcher on existing znode=/hbase/master
2013-05-31 02:25:03,214 DEBUG [RS:0;asf000:47701] zookeeper.ZKUtil(431):
regionserver:47701-0x13ef867090c0003 Set watcher on existing znode=/hbase/master
2013-05-31 02:25:03,221 DEBUG [RS:0;asf000:47701] zookeeper.ZKUtil(433):
regionserver:47701-0x13ef867090c0003 Set watcher on znode that does not yet
exist, /hbase/running
2013-05-31 02:25:03,221 DEBUG [RS:2;asf000:47305] zookeeper.ZKUtil(433):
regionserver:47305-0x13ef867090c0002 Set watcher on znode that does not yet
exist, /hbase/running
2013-05-31 02:25:03,222 DEBUG [RS:1;asf000:48644] zookeeper.ZKUtil(433):
regionserver:48644-0x13ef867090c0001 Set watcher on znode that does not yet
exist, /hbase/running
{code}
Change ServerName so it uses hostname only, not FQDN hostnames
---
Key: HBASE-8645
URL: https://issues.apache.org/jira/browse/HBASE-8645
Project: HBase
Issue Type: Sub-task
Reporter: stack
Assignee: stack
Attachments: 8645.txt, 8645.txt, 8645v2.txt, 8645v3.txt, 8645v4.txt,
8645v5.txt
No need of dragging around domain part of a hostname when we make
[jira] [Created] (HBASE-8674) JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
Andrew Purtell created HBASE-8674:
-
Summary: JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
Key: HBASE-8674
URL: https://issues.apache.org/jira/browse/HBASE-8674
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 0.98.0, 0.94.8, 0.95.1
Reporter: Andrew Purtell
Priority: Critical
people.apache.org can no longer host personal or transient Maven repos.
{noformat}
$ curl --connect-timeout 60 -v
http://people.apache.org/~garyh/mvn/org/apache/maven/plugins/maven-remote-resources-plugin/1.4/maven-remote-resources-plugin-1.4.pom
* About to connect() to people.apache.org port 80 (#0)
* Trying 140.211.11.9...
* Connection timed out after 60064 milliseconds
* Closing connection 0
curl: (28) Connection timed out after 60064 milliseconds
{noformat}
All builds are broken if the HBase custom junit or surefire jars are not
already in cache.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8674) JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
[
https://issues.apache.org/jira/browse/HBASE-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-8674:
--
Priority: Blocker (was: Critical)
JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
Key: HBASE-8674
URL: https://issues.apache.org/jira/browse/HBASE-8674
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 0.98.0, 0.94.8, 0.95.1
Reporter: Andrew Purtell
Priority: Blocker
people.apache.org can no longer host personal or transient Maven repos.
{noformat}
$ curl --connect-timeout 60 -v
http://people.apache.org/~garyh/mvn/org/apache/maven/plugins/maven-remote-resources-plugin/1.4/maven-remote-resources-plugin-1.4.pom
* About to connect() to people.apache.org port 80 (#0)
* Trying 140.211.11.9...
* Connection timed out after 60064 milliseconds
* Closing connection 0
curl: (28) Connection timed out after 60064 milliseconds
{noformat}
All builds are broken if the HBase custom junit or surefire jars are not
already in cache.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8674) JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
[
https://issues.apache.org/jira/browse/HBASE-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-8674:
--
Description:
people.apache.org cannot currently host personal or transient Maven repos.
{noformat}
$ curl --connect-timeout 60 -v
http://people.apache.org/~garyh/mvn/org/apache/maven/plugins/maven-remote-resources-plugin/1.4/maven-remote-resources-plugin-1.4.pom
* About to connect() to people.apache.org port 80 (#0)
* Trying 140.211.11.9...
* Connection timed out after 60064 milliseconds
* Closing connection 0
curl: (28) Connection timed out after 60064 milliseconds
{noformat}
All builds are at the moment broken if the HBase custom junit or surefire jars
are not already in cache. Even if this is a temporary condition, we should find
a new home for these artifacts, upgrade to versions that include our submitted
changes (if any), or fall back to release versions.
was:
people.apache.org can no longer host personal or transient Maven repos.
{noformat}
$ curl --connect-timeout 60 -v
http://people.apache.org/~garyh/mvn/org/apache/maven/plugins/maven-remote-resources-plugin/1.4/maven-remote-resources-plugin-1.4.pom
* About to connect() to people.apache.org port 80 (#0)
* Trying 140.211.11.9...
* Connection timed out after 60064 milliseconds
* Closing connection 0
curl: (28) Connection timed out after 60064 milliseconds
{noformat}
All builds are broken if the HBase custom junit or surefire jars are not
already in cache.
JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
Key: HBASE-8674
URL: https://issues.apache.org/jira/browse/HBASE-8674
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 0.98.0, 0.94.8, 0.95.1
Reporter: Andrew Purtell
Priority: Blocker
people.apache.org cannot currently host personal or transient Maven repos.
{noformat}
$ curl --connect-timeout 60 -v
http://people.apache.org/~garyh/mvn/org/apache/maven/plugins/maven-remote-resources-plugin/1.4/maven-remote-resources-plugin-1.4.pom
* About to connect() to people.apache.org port 80 (#0)
* Trying 140.211.11.9...
* Connection timed out after 60064 milliseconds
* Closing connection 0
curl: (28) Connection timed out after 60064 milliseconds
{noformat}
All builds are at the moment broken if the HBase custom junit or surefire
jars are not already in cache. Even if this is a temporary condition, we
should find a new home for these artifacts, upgrade to versions that include
our submitted changes (if any), or fall back to release versions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8674) JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
[
https://issues.apache.org/jira/browse/HBASE-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672628#comment-13672628
]
Andrew Purtell commented on HBASE-8674:
---
In 0.94 builds, we also have http://people.apache.org/~stack/m2/repository
JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
Key: HBASE-8674
URL: https://issues.apache.org/jira/browse/HBASE-8674
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 0.98.0, 0.94.8, 0.95.1
Reporter: Andrew Purtell
Priority: Blocker
people.apache.org cannot currently host personal or transient Maven repos.
{noformat}
$ curl --connect-timeout 60 -v
http://people.apache.org/~garyh/mvn/org/apache/maven/plugins/maven-remote-resources-plugin/1.4/maven-remote-resources-plugin-1.4.pom
* About to connect() to people.apache.org port 80 (#0)
* Trying 140.211.11.9...
* Connection timed out after 60064 milliseconds
* Closing connection 0
curl: (28) Connection timed out after 60064 milliseconds
{noformat}
All builds are at the moment broken if the HBase custom junit or surefire
jars are not already in cache. Even if this is a temporary condition, we
should find a new home for these artifacts, upgrade to versions that include
our submitted changes (if any), or fall back to release versions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8640) ServerName in master may not initialize with the configured ipc address of hbase.master.ipc.address
[
https://issues.apache.org/jira/browse/HBASE-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
rajeshbabu updated HBASE-8640:
--
Attachment: HBASE-8640_94.patch
Patch for 94
ServerName in master may not initialize with the configured ipc address of
hbase.master.ipc.address
---
Key: HBASE-8640
URL: https://issues.apache.org/jira/browse/HBASE-8640
Project: HBase
Issue Type: Bug
Components: master
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.95.2, 0.94.9
Attachments: HBASE-8640_94.patch, HBASE-8640.patch
We are starting rpc server with default interface hostname or configured ipc
address
{code}
this.rpcServer = HBaseRPC.getServer(this,
new Class?[]{HMasterInterface.class, HMasterRegionInterface.class},
initialIsa.getHostName(), // This is bindAddress if set else it's
hostname
initialIsa.getPort(),
numHandlers,
0, // we dont use high priority handlers in master
conf.getBoolean(hbase.rpc.verbose, false), conf,
0); // this is a DNC w/o high priority handlers
{code}
But we are initialzing servername with default hostname always master znode
also have this hostname.
{code}
String hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
conf.get(hbase.master.dns.interface, default),
conf.get(hbase.master.dns.nameserver, default)));
...
this.serverName = new ServerName(hostname,
this.isa.getPort(), System.currentTimeMillis());
{code}
If both default interface hostname and configured ipc address are not same
clients will get MasterNotRunningException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8640) ServerName in master may not initialize with the configured ipc address of hbase.master.ipc.address
[
https://issues.apache.org/jira/browse/HBASE-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672632#comment-13672632
]
Hadoop QA commented on HBASE-8640:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585773/HBASE-8640_94.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5906//console
This message is automatically generated.
ServerName in master may not initialize with the configured ipc address of
hbase.master.ipc.address
---
Key: HBASE-8640
URL: https://issues.apache.org/jira/browse/HBASE-8640
Project: HBase
Issue Type: Bug
Components: master
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.95.2, 0.94.9
Attachments: HBASE-8640_94.patch, HBASE-8640.patch
We are starting rpc server with default interface hostname or configured ipc
address
{code}
this.rpcServer = HBaseRPC.getServer(this,
new Class?[]{HMasterInterface.class, HMasterRegionInterface.class},
initialIsa.getHostName(), // This is bindAddress if set else it's
hostname
initialIsa.getPort(),
numHandlers,
0, // we dont use high priority handlers in master
conf.getBoolean(hbase.rpc.verbose, false), conf,
0); // this is a DNC w/o high priority handlers
{code}
But we are initialzing servername with default hostname always master znode
also have this hostname.
{code}
String hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
conf.get(hbase.master.dns.interface, default),
conf.get(hbase.master.dns.nameserver, default)));
...
this.serverName = new ServerName(hostname,
this.isa.getPort(), System.currentTimeMillis());
{code}
If both default interface hostname and configured ipc address are not same
clients will get MasterNotRunningException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8656) Rpc call may not be notified in SecureClient
[
https://issues.apache.org/jira/browse/HBASE-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8656:
-
Priority: Critical (was: Major)
Rpc call may not be notified in SecureClient
Key: HBASE-8656
URL: https://issues.apache.org/jira/browse/HBASE-8656
Project: HBase
Issue Type: Bug
Components: Client, IPC/RPC, security
Affects Versions: 0.94.7
Reporter: cuijianwei
Priority: Critical
Fix For: 0.94.9
Attachments: HBASE-8656-0.94-v1.txt
In SecureClient.java, rpc responses will be processed by receiveResponse()
which looks like:
{code}
try {
int id = in.readInt();// try to read an id
if (LOG.isDebugEnabled())
LOG.debug(getName() + got value # + id);
Call call = calls.remove(id);
int state = in.readInt(); // read call status
if (LOG.isDebugEnabled()) {
LOG.debug(call #+id+ state is + state);
}
if (state == Status.SUCCESS.state) {
Writable value = ReflectionUtils.newInstance(valueClass, conf);
value.readFields(in); // read value
if (LOG.isDebugEnabled()) {
LOG.debug(call #+id+, response is:\n+value.toString());
}
// it's possible that this call may have been cleaned up due to a
RPC
// timeout, so check if it still exists before setting the value.
if (call != null) {
call.setValue(value);
}
} else if (state == Status.ERROR.state) {
if (call != null) {
call.setException(new
RemoteException(WritableUtils.readString(in), WritableUtils
.readString(in)));
}
} else if (state == Status.FATAL.state) {
// Close the connection
markClosed(new RemoteException(WritableUtils.readString(in),
WritableUtils.readString(in)));
}
} catch (IOException e) {
if (e instanceof SocketTimeoutException remoteId.rpcTimeout 0) {
// Clean up open calls but don't treat this as a fatal condition,
// since we expect certain responses to not make it by the specified
// {@link ConnectionId#rpcTimeout}.
closeException = e;
} else {
// Since the server did not respond within the default ping interval
// time, treat this as a fatal condition and close this connection
markClosed(e);
}
} finally {
if (remoteId.rpcTimeout 0) {
cleanupCalls(remoteId.rpcTimeout);
}
}
}
{code}
In above code, in the try block, the call will be firstly removed from call
map by:
{code}
Call call = calls.remove(id);
{code}
There may be two cases leading the call couldn't be notified and the invoking
thread will wait forever.
Firstly, if the returned status is Status.FATAL.state by:
{code}
int state = in.readInt(); // read call status
{code}
The code will come into:
{code}
} else if (state == Status.FATAL.state) {
// Close the connection
markClosed(new RemoteException(WritableUtils.readString(in),
WritableUtils.readString(in)));
}
{code}
Here, the SecureConnection is marked as closed and all rpc calls in call map
of this connection will be notified to receive an exception. However, the
current rpc call has been removed from the call map, it won't be notified.
Secondly, after the call has been removed by:
{code}
Call call = calls.remove(id);
{code}
If we encounter any exception before the 'try' block finished, the code will
come into 'catch' and 'finally' block, neither 'catch' block nor 'finally'
block will notify the rpc call because it has been removed from call map.
Compared with receiveResponse() in HBaseClient.java, it may be better to get
the rpc call from call map and remove it at the end of the 'try' block.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8656) Rpc call may not be notified in SecureClient
[
https://issues.apache.org/jira/browse/HBASE-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8656:
-
Fix Version/s: 0.94.9
Rpc call may not be notified in SecureClient
Key: HBASE-8656
URL: https://issues.apache.org/jira/browse/HBASE-8656
Project: HBase
Issue Type: Bug
Components: Client, IPC/RPC, security
Affects Versions: 0.94.7
Reporter: cuijianwei
Fix For: 0.94.9
Attachments: HBASE-8656-0.94-v1.txt
In SecureClient.java, rpc responses will be processed by receiveResponse()
which looks like:
{code}
try {
int id = in.readInt();// try to read an id
if (LOG.isDebugEnabled())
LOG.debug(getName() + got value # + id);
Call call = calls.remove(id);
int state = in.readInt(); // read call status
if (LOG.isDebugEnabled()) {
LOG.debug(call #+id+ state is + state);
}
if (state == Status.SUCCESS.state) {
Writable value = ReflectionUtils.newInstance(valueClass, conf);
value.readFields(in); // read value
if (LOG.isDebugEnabled()) {
LOG.debug(call #+id+, response is:\n+value.toString());
}
// it's possible that this call may have been cleaned up due to a
RPC
// timeout, so check if it still exists before setting the value.
if (call != null) {
call.setValue(value);
}
} else if (state == Status.ERROR.state) {
if (call != null) {
call.setException(new
RemoteException(WritableUtils.readString(in), WritableUtils
.readString(in)));
}
} else if (state == Status.FATAL.state) {
// Close the connection
markClosed(new RemoteException(WritableUtils.readString(in),
WritableUtils.readString(in)));
}
} catch (IOException e) {
if (e instanceof SocketTimeoutException remoteId.rpcTimeout 0) {
// Clean up open calls but don't treat this as a fatal condition,
// since we expect certain responses to not make it by the specified
// {@link ConnectionId#rpcTimeout}.
closeException = e;
} else {
// Since the server did not respond within the default ping interval
// time, treat this as a fatal condition and close this connection
markClosed(e);
}
} finally {
if (remoteId.rpcTimeout 0) {
cleanupCalls(remoteId.rpcTimeout);
}
}
}
{code}
In above code, in the try block, the call will be firstly removed from call
map by:
{code}
Call call = calls.remove(id);
{code}
There may be two cases leading the call couldn't be notified and the invoking
thread will wait forever.
Firstly, if the returned status is Status.FATAL.state by:
{code}
int state = in.readInt(); // read call status
{code}
The code will come into:
{code}
} else if (state == Status.FATAL.state) {
// Close the connection
markClosed(new RemoteException(WritableUtils.readString(in),
WritableUtils.readString(in)));
}
{code}
Here, the SecureConnection is marked as closed and all rpc calls in call map
of this connection will be notified to receive an exception. However, the
current rpc call has been removed from the call map, it won't be notified.
Secondly, after the call has been removed by:
{code}
Call call = calls.remove(id);
{code}
If we encounter any exception before the 'try' block finished, the code will
come into 'catch' and 'finally' block, neither 'catch' block nor 'finally'
block will notify the rpc call because it has been removed from call map.
Compared with receiveResponse() in HBaseClient.java, it may be better to get
the rpc call from call map and remove it at the end of the 'try' block.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8626) RowMutations fail when Delete and Put on same columnFamily/column/row
[ https://issues.apache.org/jira/browse/HBASE-8626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672702#comment-13672702 ] Vinod commented on HBASE-8626: -- Lars, the approach you suggest does not work for my use-case because of the race-condition mentioned by Ted in his comment above at: https://issues.apache.org/jira/browse/HBASE-8626?focusedCommentId=13669464page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13669464 Can we extend current RowMutations to support something like what Andrew stated in his previous comment here: https://issues.apache.org/jira/browse/HBASE-8626?focusedCommentId=13669152page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13669152 That would solve my use-case perfectly. RowMutations fail when Delete and Put on same columnFamily/column/row - Key: HBASE-8626 URL: https://issues.apache.org/jira/browse/HBASE-8626 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.7, 0.95.0 Environment: Ubuntu 12.04, HBase 0.94.7 Reporter: Vinod Assignee: Ted Yu Attachments: 8626-v1.txt, TestRowMutations.java, tests_for_row_mutations1.patch When RowMutations have a Delete followed by Put to same column family or columns or rows, only the Delete is happening while the Put is ignored so atomicity of RowMutations is broken for such cases. Attached is a unit test where the following tests are failing: - testDeleteCFThenPutInSameCF: Delete a column family and then Put to same column family. - testDeleteColumnThenPutSameColumn: Delete a column and then Put to same column. - testDeleteRowThenPutSameRow: Delete a row and then Put to same row -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7055) port HBASE-6371 tier-based compaction from 0.89-fb to trunk (with changes)
[ https://issues.apache.org/jira/browse/HBASE-7055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672713#comment-13672713 ] Sergey Shelukhin commented on HBASE-7055: - My take on this is that there appears to be at least few scenarios that sound viable. E.g. old data is not as important and read rarely, so we don't care if it stays uncompacted and assume blooms will help us ignore these files for the keys that are actually being read; or new data is in cache so again there's no reason to compact the latest files. Or both. However, testing if these actually work require setting up a complicated test scenario with specific read patterns, over large amounts of data, or a very good simulation tool that would estimate read impact (cache usage, how many files touched, what is going on in parallel); the will for either of these is presently lacking. On the other hand, patch is extremely low impact and will be off by default. So yeah maybe we can keep it for 0.96.X. port HBASE-6371 tier-based compaction from 0.89-fb to trunk (with changes) -- Key: HBASE-7055 URL: https://issues.apache.org/jira/browse/HBASE-7055 Project: HBase Issue Type: Task Components: Compaction Affects Versions: 0.95.2 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.95.1 Attachments: HBASE-6371-squashed.patch, HBASE-6371-v2-squashed.patch, HBASE-6371-v3-refactor-only-squashed.patch, HBASE-6371-v4-refactor-only-squashed.patch, HBASE-6371-v5-refactor-only-squashed.patch, HBASE-7055-v0.patch, HBASE-7055-v1.patch, HBASE-7055-v2.patch, HBASE-7055-v3.patch, HBASE-7055-v4.patch, HBASE-7055-v5.patch, HBASE-7055-v6.patch, HBASE-7055-v7.patch, HBASE-7055-v7.patch See HBASE-6371 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8626) RowMutations fail when Delete and Put on same columnFamily/column/row
[ https://issues.apache.org/jira/browse/HBASE-8626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672737#comment-13672737 ] Lars Hofhansl commented on HBASE-8626: -- You can always have the races when multiple clients issue requests at the same time. Two clients could issue a request within the same ms (or in the case of the patch within 1 ms of each other) and the result would still be not what you want - basically there *is* no correct order if things happen at the same time. The race Ted outline is not worth than the one you get with the patch (well the chance might be slightly reduce since writes to the same rows are serialized, so the chance is lessened that the region server does them in the same ms, but it can still happen). RowMutations fail when Delete and Put on same columnFamily/column/row - Key: HBASE-8626 URL: https://issues.apache.org/jira/browse/HBASE-8626 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.7, 0.95.0 Environment: Ubuntu 12.04, HBase 0.94.7 Reporter: Vinod Assignee: Ted Yu Attachments: 8626-v1.txt, TestRowMutations.java, tests_for_row_mutations1.patch When RowMutations have a Delete followed by Put to same column family or columns or rows, only the Delete is happening while the Put is ignored so atomicity of RowMutations is broken for such cases. Attached is a unit test where the following tests are failing: - testDeleteCFThenPutInSameCF: Delete a column family and then Put to same column family. - testDeleteColumnThenPutSameColumn: Delete a column and then Put to same column. - testDeleteRowThenPutSameRow: Delete a row and then Put to same row -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7055) port HBASE-6371 tier-based compaction from 0.89-fb to trunk (with changes)
[ https://issues.apache.org/jira/browse/HBASE-7055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672738#comment-13672738 ] Lars Hofhansl commented on HBASE-7055: -- I think we should start some collaboration on this. At Salesforce we're working on a feature to separate old and new data for scenarios where we have to store very many versions for the odd historical query, but most queries are for the latest data. port HBASE-6371 tier-based compaction from 0.89-fb to trunk (with changes) -- Key: HBASE-7055 URL: https://issues.apache.org/jira/browse/HBASE-7055 Project: HBase Issue Type: Task Components: Compaction Affects Versions: 0.95.2 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.95.1 Attachments: HBASE-6371-squashed.patch, HBASE-6371-v2-squashed.patch, HBASE-6371-v3-refactor-only-squashed.patch, HBASE-6371-v4-refactor-only-squashed.patch, HBASE-6371-v5-refactor-only-squashed.patch, HBASE-7055-v0.patch, HBASE-7055-v1.patch, HBASE-7055-v2.patch, HBASE-7055-v3.patch, HBASE-7055-v4.patch, HBASE-7055-v5.patch, HBASE-7055-v6.patch, HBASE-7055-v7.patch, HBASE-7055-v7.patch See HBASE-6371 for details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8590) [0.94] BlockingMetaScannerVisitor should check for parent meta entry while waiting for split daughter
[ https://issues.apache.org/jira/browse/HBASE-8590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672740#comment-13672740 ] Lars Hofhansl commented on HBASE-8590: -- Patch looks reasonable to me. Wanna commit [~enis]? [0.94] BlockingMetaScannerVisitor should check for parent meta entry while waiting for split daughter - Key: HBASE-8590 URL: https://issues.apache.org/jira/browse/HBASE-8590 Project: HBase Issue Type: Bug Components: regionserver Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.94.9 Attachments: hbase-8590_v1.patch This was discovered after HBASE-8505 went in, which introduces a test sporadically triggering this bug. From comments at HBASE-8505: From the logs at https://builds.apache.org/job/HBase-0.94-security/ws/trunk/target/surefire-reports/org.apache.hadoop.hbase.client.TestMetaScanner-output.txt, I think I understand what is going on: BlockingMetaScannerVisitor blocks and wait for the split daughter to appear when it sees a parent region (HBASE-5986). CatalogJanitor on the other hand will order the regions in a (kind-of) topological sort (based on parent child relation) so that it will guarantee parents are not GC'd before daughters. What is happening in this issue is not related to the patch in this jira, but the test triggers this extremely rare case by running concurrent catalogjanitor, splits and metascanners. We have parent, splita and splitb regions, and catalogjanitor decides to delete parent first and splitb in one run. While there is a concurrent metascanner which will go over the parent, and sees that it is split, but before being able to read the split daughter, catalog janitor will delete both the parent and the child, which will lead to metascanner blocking until timeout and failing the test. On solution might be to also check whether the parent is still there in BlockingMetaScannerVisitor while we are blocking for the daughter. Good thing is that with HBASE-7721, we don't need any of this in trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8001) Avoid unnecessary lazy seek
[ https://issues.apache.org/jira/browse/HBASE-8001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672771#comment-13672771 ] Raymond Liu commented on HBASE-8001: [~lhofhansl], [~ted_yu]: sorry for late on this issue. busy for other staff. I do this test again in with single RS. I still use M/R job, while the table only have one region. and is 2M rows ,1CF, 18col, without any compression or encoding. size about 3G on disk. And I don't use blockcache,every time the data is read from disk by a real seek. but as we discussed before, use of blockcache will only led to more gain with this patch. with this patch, a 18col full table scan cost 99-101s, while without this patch it will cost 108-109s. still noticeable difference. I test it for several times on each case. the result is pretty stable. Do you mind to take a end2end test? I am not sure is there any other thing might still have impact upon your test case. might be that the data size is too small? Avoid unnecessary lazy seek --- Key: HBASE-8001 URL: https://issues.apache.org/jira/browse/HBASE-8001 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.94.5 Reporter: Raymond Liu Assignee: Raymond Liu Fix For: 0.98.0 Attachments: HBASE-8001_onescanner.patch, HBASE-8001_onescanner_v2.patch Lazy seek helps to reduce the real seek needed for multi hfile, when the kv from newer hfile is enough to satisfy the query. While in many case, it just push the real seek later, and do not reduce the number of real seek. e.g. there are only one hfile, or storefilescanner is closed and only one left, or the scan need to go through all the versions, or there are only one version of row and a sequence scan is performed. In these case, lazy seek just bring extra overhead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8642) [Snapshot] List and delete snapshot by table
[ https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672774#comment-13672774 ] Julian Zhou commented on HBASE-8642: Hi, [~mbertozzi], I see that last slide of HBASE-7806 for future, is that to seperate .data and .meta to enable possible file sharing among tables. One byte (256 items could be attached) for each node anchor under .data tree. This could be a new catalog system for hbase. [Snapshot] List and delete snapshot by table Key: HBASE-8642 URL: https://issues.apache.org/jira/browse/HBASE-8642 Project: HBase Issue Type: Improvement Components: snapshots Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Reporter: Julian Zhou Assignee: Julian Zhou Priority: Minor Fix For: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Attachments: 8642-trunk-0.95-v0.patch, 8642-trunk-0.95-v1.patch Support list and delete snapshot by table name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8642) [Snapshot] List and delete snapshot by table
[
https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672778#comment-13672778
]
Matteo Bertozzi commented on HBASE-8642:
{quote}is that to seperate data and meta to enable possible file sharing among
tables. This could be a new catalog system for hbase.{quote}
Yeah that's the idea... it not just file sharing between table/snapshots is
also the ability to switch to managing blocks directly and avoid to rewrite
everything during compactions.
{quote}One byte (256 items could be attached) for each node anchor under .data
tree.{quote}
Not sure what you mean with this... The file placement using the first bytes of
the hash/name is just to avoid the problem of too many files in a directory
that old filesystems have, and a different way to store stuff removing the
knowledge of table/region/cf that today is hardcoded. This doesn't really have
effect on all the capabilities that you'll gain from the richer metadata system.
(This stuff is starting being off topic here, since is not related 100% with
this jira, if you've other question/suggestions/... just write it on HBASE-7806)
[Snapshot] List and delete snapshot by table
Key: HBASE-8642
URL: https://issues.apache.org/jira/browse/HBASE-8642
Project: HBase
Issue Type: Improvement
Components: snapshots
Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2
Reporter: Julian Zhou
Assignee: Julian Zhou
Priority: Minor
Fix For: 0.98.0, 0.95.0, 0.95.1, 0.95.2
Attachments: 8642-trunk-0.95-v0.patch, 8642-trunk-0.95-v1.patch
Support list and delete snapshot by table name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8642) [Snapshot] List and delete snapshot by table
[ https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julian Zhou updated HBASE-8642: --- Attachment: 8642-trunk-0.95-v2.patch Updated with new ruby module name. Could we go ahead with this? [Snapshot] List and delete snapshot by table Key: HBASE-8642 URL: https://issues.apache.org/jira/browse/HBASE-8642 Project: HBase Issue Type: Improvement Components: snapshots Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Reporter: Julian Zhou Assignee: Julian Zhou Priority: Minor Fix For: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Attachments: 8642-trunk-0.95-v0.patch, 8642-trunk-0.95-v1.patch, 8642-trunk-0.95-v2.patch Support list and delete snapshot by table name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4811) Support reverse Scan
[ https://issues.apache.org/jira/browse/HBASE-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chunhui shen updated HBASE-4811: Attachment: hbase-4811-trunkv4.patch Addressing Ted's review comments Support reverse Scan Key: HBASE-4811 URL: https://issues.apache.org/jira/browse/HBASE-4811 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.20.6, 0.94.7 Reporter: John Carrino Assignee: Liang Xie Attachments: HBase-4811-0.94.3modified.txt, HBase-4811-0.94-v2.txt, hbase-4811-trunkv1.patch, hbase-4811-trunkv4.patch All the documentation I find about HBase says that if you want forward and reverse scans you should just build 2 tables and one be ascending and one descending. Is there a fundamental reason that HBase only supports forward Scan? It seems like a lot of extra space overhead and coding overhead (to keep them in sync) to support 2 tables. I am assuming this has been discussed before, but I can't find the discussions anywhere about it or why it would be infeasible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8674) JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
[
https://issues.apache.org/jira/browse/HBASE-8674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672791#comment-13672791
]
stack commented on HBASE-8674:
--
The stack repository was for a zookeeper and hadoop 1.0.0 RCs. It can be
removed.
[~nkeywal] Can we let go of the surefire or junit custom dependency? IIRC, we
can move to official release of one of these artifacts only?
That leaves us wanting a home for whatever is left. I could host it on
personal space (Our Dave Wang is asking if it can be hosted by Cloudera).
JUnit and Surefire TRUNK-HBASE-2 plugins need a new home
Key: HBASE-8674
URL: https://issues.apache.org/jira/browse/HBASE-8674
Project: HBase
Issue Type: Bug
Components: build
Affects Versions: 0.98.0, 0.94.8, 0.95.1
Reporter: Andrew Purtell
Priority: Blocker
people.apache.org cannot currently host personal or transient Maven repos.
{noformat}
$ curl --connect-timeout 60 -v
http://people.apache.org/~garyh/mvn/org/apache/maven/plugins/maven-remote-resources-plugin/1.4/maven-remote-resources-plugin-1.4.pom
* About to connect() to people.apache.org port 80 (#0)
* Trying 140.211.11.9...
* Connection timed out after 60064 milliseconds
* Closing connection 0
curl: (28) Connection timed out after 60064 milliseconds
{noformat}
All builds are at the moment broken if the HBase custom junit or surefire
jars are not already in cache. Even if this is a temporary condition, we
should find a new home for these artifacts, upgrade to versions that include
our submitted changes (if any), or fall back to release versions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8656) Rpc call may not be notified in SecureClient
[
https://issues.apache.org/jira/browse/HBASE-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672794#comment-13672794
]
stack commented on HBASE-8656:
--
We do not have this issue in trunk/0.95?
Rpc call may not be notified in SecureClient
Key: HBASE-8656
URL: https://issues.apache.org/jira/browse/HBASE-8656
Project: HBase
Issue Type: Bug
Components: Client, IPC/RPC, security
Affects Versions: 0.94.7
Reporter: cuijianwei
Priority: Critical
Fix For: 0.94.9
Attachments: HBASE-8656-0.94-v1.txt
In SecureClient.java, rpc responses will be processed by receiveResponse()
which looks like:
{code}
try {
int id = in.readInt();// try to read an id
if (LOG.isDebugEnabled())
LOG.debug(getName() + got value # + id);
Call call = calls.remove(id);
int state = in.readInt(); // read call status
if (LOG.isDebugEnabled()) {
LOG.debug(call #+id+ state is + state);
}
if (state == Status.SUCCESS.state) {
Writable value = ReflectionUtils.newInstance(valueClass, conf);
value.readFields(in); // read value
if (LOG.isDebugEnabled()) {
LOG.debug(call #+id+, response is:\n+value.toString());
}
// it's possible that this call may have been cleaned up due to a
RPC
// timeout, so check if it still exists before setting the value.
if (call != null) {
call.setValue(value);
}
} else if (state == Status.ERROR.state) {
if (call != null) {
call.setException(new
RemoteException(WritableUtils.readString(in), WritableUtils
.readString(in)));
}
} else if (state == Status.FATAL.state) {
// Close the connection
markClosed(new RemoteException(WritableUtils.readString(in),
WritableUtils.readString(in)));
}
} catch (IOException e) {
if (e instanceof SocketTimeoutException remoteId.rpcTimeout 0) {
// Clean up open calls but don't treat this as a fatal condition,
// since we expect certain responses to not make it by the specified
// {@link ConnectionId#rpcTimeout}.
closeException = e;
} else {
// Since the server did not respond within the default ping interval
// time, treat this as a fatal condition and close this connection
markClosed(e);
}
} finally {
if (remoteId.rpcTimeout 0) {
cleanupCalls(remoteId.rpcTimeout);
}
}
}
{code}
In above code, in the try block, the call will be firstly removed from call
map by:
{code}
Call call = calls.remove(id);
{code}
There may be two cases leading the call couldn't be notified and the invoking
thread will wait forever.
Firstly, if the returned status is Status.FATAL.state by:
{code}
int state = in.readInt(); // read call status
{code}
The code will come into:
{code}
} else if (state == Status.FATAL.state) {
// Close the connection
markClosed(new RemoteException(WritableUtils.readString(in),
WritableUtils.readString(in)));
}
{code}
Here, the SecureConnection is marked as closed and all rpc calls in call map
of this connection will be notified to receive an exception. However, the
current rpc call has been removed from the call map, it won't be notified.
Secondly, after the call has been removed by:
{code}
Call call = calls.remove(id);
{code}
If we encounter any exception before the 'try' block finished, the code will
come into 'catch' and 'finally' block, neither 'catch' block nor 'finally'
block will notify the rpc call because it has been removed from call map.
Compared with receiveResponse() in HBaseClient.java, it may be better to get
the rpc call from call map and remove it at the end of the 'try' block.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8640) ServerName in master may not initialize with the configured ipc address of hbase.master.ipc.address
[
https://issues.apache.org/jira/browse/HBASE-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-8640:
-
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk, 0.95 and to 0.94 (Pardon me if I presumed too much
[~lhofhansl])
ServerName in master may not initialize with the configured ipc address of
hbase.master.ipc.address
---
Key: HBASE-8640
URL: https://issues.apache.org/jira/browse/HBASE-8640
Project: HBase
Issue Type: Bug
Components: master
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.95.2, 0.94.9
Attachments: HBASE-8640_94.patch, HBASE-8640.patch
We are starting rpc server with default interface hostname or configured ipc
address
{code}
this.rpcServer = HBaseRPC.getServer(this,
new Class?[]{HMasterInterface.class, HMasterRegionInterface.class},
initialIsa.getHostName(), // This is bindAddress if set else it's
hostname
initialIsa.getPort(),
numHandlers,
0, // we dont use high priority handlers in master
conf.getBoolean(hbase.rpc.verbose, false), conf,
0); // this is a DNC w/o high priority handlers
{code}
But we are initialzing servername with default hostname always master znode
also have this hostname.
{code}
String hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
conf.get(hbase.master.dns.interface, default),
conf.get(hbase.master.dns.nameserver, default)));
...
this.serverName = new ServerName(hostname,
this.isa.getPort(), System.currentTimeMillis());
{code}
If both default interface hostname and configured ipc address are not same
clients will get MasterNotRunningException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8607) Run HBase server in an OSGi container
[ https://issues.apache.org/jira/browse/HBASE-8607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672799#comment-13672799 ] stack commented on HBASE-8607: -- [~andrew.purt...@gmail.com] I like the direction you suggest; would proof out two concepts in the one go (osgi and moving cp hosting out of the regionserver). Run HBase server in an OSGi container - Key: HBASE-8607 URL: https://issues.apache.org/jira/browse/HBASE-8607 Project: HBase Issue Type: New Feature Components: regionserver Reporter: James Taylor Run the HBase server in an OSGi container to support updating custom filters and coprocessor updates without requiring a region server reboot. Typically, applications that use coprocessors and custom filters also have shared classes underneath, so putting the burden on the user to include some kind of version name in the class is not adequate. Including the version name in the package might work in some cases (at least until dependent jars start to change as well), but is cumbersome and overburdens the app developer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8656) Rpc call may not be notified in SecureClient
[
https://issues.apache.org/jira/browse/HBASE-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672800#comment-13672800
]
cuijianwei commented on HBASE-8656:
---
[~saint@gmail.com], in trunk/0.95, SecureClient may be replaced by
HBaseSaslRpcClient. The code changes a lot, I'am not sure whether
HBaseSaslRpcClient has the same problem.
Rpc call may not be notified in SecureClient
Key: HBASE-8656
URL: https://issues.apache.org/jira/browse/HBASE-8656
Project: HBase
Issue Type: Bug
Components: Client, IPC/RPC, security
Affects Versions: 0.94.7
Reporter: cuijianwei
Priority: Critical
Fix For: 0.94.9
Attachments: HBASE-8656-0.94-v1.txt
In SecureClient.java, rpc responses will be processed by receiveResponse()
which looks like:
{code}
try {
int id = in.readInt();// try to read an id
if (LOG.isDebugEnabled())
LOG.debug(getName() + got value # + id);
Call call = calls.remove(id);
int state = in.readInt(); // read call status
if (LOG.isDebugEnabled()) {
LOG.debug(call #+id+ state is + state);
}
if (state == Status.SUCCESS.state) {
Writable value = ReflectionUtils.newInstance(valueClass, conf);
value.readFields(in); // read value
if (LOG.isDebugEnabled()) {
LOG.debug(call #+id+, response is:\n+value.toString());
}
// it's possible that this call may have been cleaned up due to a
RPC
// timeout, so check if it still exists before setting the value.
if (call != null) {
call.setValue(value);
}
} else if (state == Status.ERROR.state) {
if (call != null) {
call.setException(new
RemoteException(WritableUtils.readString(in), WritableUtils
.readString(in)));
}
} else if (state == Status.FATAL.state) {
// Close the connection
markClosed(new RemoteException(WritableUtils.readString(in),
WritableUtils.readString(in)));
}
} catch (IOException e) {
if (e instanceof SocketTimeoutException remoteId.rpcTimeout 0) {
// Clean up open calls but don't treat this as a fatal condition,
// since we expect certain responses to not make it by the specified
// {@link ConnectionId#rpcTimeout}.
closeException = e;
} else {
// Since the server did not respond within the default ping interval
// time, treat this as a fatal condition and close this connection
markClosed(e);
}
} finally {
if (remoteId.rpcTimeout 0) {
cleanupCalls(remoteId.rpcTimeout);
}
}
}
{code}
In above code, in the try block, the call will be firstly removed from call
map by:
{code}
Call call = calls.remove(id);
{code}
There may be two cases leading the call couldn't be notified and the invoking
thread will wait forever.
Firstly, if the returned status is Status.FATAL.state by:
{code}
int state = in.readInt(); // read call status
{code}
The code will come into:
{code}
} else if (state == Status.FATAL.state) {
// Close the connection
markClosed(new RemoteException(WritableUtils.readString(in),
WritableUtils.readString(in)));
}
{code}
Here, the SecureConnection is marked as closed and all rpc calls in call map
of this connection will be notified to receive an exception. However, the
current rpc call has been removed from the call map, it won't be notified.
Secondly, after the call has been removed by:
{code}
Call call = calls.remove(id);
{code}
If we encounter any exception before the 'try' block finished, the code will
come into 'catch' and 'finally' block, neither 'catch' block nor 'finally'
block will notify the rpc call because it has been removed from call map.
Compared with receiveResponse() in HBaseClient.java, it may be better to get
the rpc call from call map and remove it at the end of the 'try' block.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-8673) Make code involving class.getMethod() more amenable to mocking
[
https://issues.apache.org/jira/browse/HBASE-8673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-8673.
--
Resolution: Invalid
Resolving trivial, issue w/o description of why it needs to be fixed nor of how
it is to be fixed.
I also believe this a non-issue. See
hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestFSHDFSUtils.java
for example of how to mock a DFS with a method that may or may not be present.
Make code involving class.getMethod() more amenable to mocking
--
Key: HBASE-8673
URL: https://issues.apache.org/jira/browse/HBASE-8673
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Priority: Minor
In some place, such as the following (FSUtils#isInSafeMode), we retrieve
method using getMethod() with hardcoded class name:
{code}
private static boolean isInSafeMode(DistributedFileSystem dfs) throws
IOException {
boolean inSafeMode = false;
try {
Method m = DistributedFileSystem.class.getMethod(setSafeMode, new
Class? []{
{code}
The above code is not friendly to mocking.
The test class may define its own DistributedFileSystem descendent class and
pass an instance to FSUtils#isInSafeMode(). In that case, assertion similar
to the following would fail:
{code}
Mockito.verify(dfs, Mockito.times(1)).setSafeMode();
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8675) Two active Hmaster for AUTH_FAILED in secure hbase cluster
Liu Shaohui created HBASE-8675: -- Summary: Two active Hmaster for AUTH_FAILED in secure hbase cluster Key: HBASE-8675 URL: https://issues.apache.org/jira/browse/HBASE-8675 Project: HBase Issue Type: Bug Components: master Reporter: Liu Shaohui Priority: Critical In our product cluster, because of the net problem to kerberos server, the ZooKeeperWatcher in active hmaster fails to Auth , gets a connection Event of AUTH_FAILED and loose the master lock. But the zookeeper watcher ignores the event, so the old active hmaster keeps to be active. After the net problem is fixed, the backup hmaster gets the master lock and becomes active. There are two two active hmasters in the cluster. 2013-05-30 09:44:21,004 ERROR org.apache.zookeeper.client.ZooKeeperSaslClient: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: krb1.xiaomi.net)]) occurred when evaluating Zookeeper Quorum Member's received SASL token. Zookeeper Client will go to AUTH_FAILED state. 2013-05-30 09:54:07,755 WARN org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x3e10d98be405bc Unable to set watcher on znode /hbase/master org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = AuthFailed for /hbase/master at org.apache.zookeeper.KeeperException.create(KeeperException.java:123) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1036) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:166) at org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:231) at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:76) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:595) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:850) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:825) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:286) at org.apache.hadoop.hbase.client.HTable.init(HTable.java:201) at org.apache.hadoop.hbase.catalog.MetaReader.getHTable(MetaReader.java:200) at org.apache.hadoop.hbase.catalog.MetaReader.getMetaHTable(MetaReader.java:226) at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:705) at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:183) at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:168) at org.apache.hadoop.hbase.master.CatalogJanitor.getSplitParents(CatalogJanitor.java:123) at org.apache.hadoop.hbase.master.CatalogJanitor.scan(CatalogJanitor.java:134) at org.apache.hadoop.hbase.master.CatalogJanitor.chore(CatalogJanitor.java:92) at org.apache.hadoop.hbase.Chore.run(Chore.java:67) at java.lang.Thread.run(Thread.java:662) I want to just abort the hmaster server if AuthFailed or SaslAuthenticated. Any better idea about this issue? For ZookeeperWatcher is used in many classes, will the aborting will bring more problems? Any more problems we need consider? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8642) [Snapshot] List and delete snapshot by table
[
https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672817#comment-13672817
]
Hadoop QA commented on HBASE-8642:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585796/8642-trunk-0.95-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5907//console
This message is automatically generated.
[Snapshot] List and delete snapshot by table
Key: HBASE-8642
URL: https://issues.apache.org/jira/browse/HBASE-8642
Project: HBase
Issue Type: Improvement
Components: snapshots
Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2
Reporter: Julian Zhou
Assignee: Julian Zhou
Priority: Minor
Fix For: 0.98.0, 0.95.0, 0.95.1, 0.95.2
Attachments: 8642-trunk-0.95-v0.patch, 8642-trunk-0.95-v1.patch,
8642-trunk-0.95-v2.patch
Support list and delete snapshot by table name.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8640) ServerName in master may not initialize with the configured ipc address of hbase.master.ipc.address
[
https://issues.apache.org/jira/browse/HBASE-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672832#comment-13672832
]
Hudson commented on HBASE-8640:
---
Integrated in HBase-0.94-security #157 (See
[https://builds.apache.org/job/HBase-0.94-security/157/])
HBASE-8640 ServerName in master may not initialize with the configured ipc
address of hbase.master.ipc.address (Revision 1488837)
Result = SUCCESS
stack :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
ServerName in master may not initialize with the configured ipc address of
hbase.master.ipc.address
---
Key: HBASE-8640
URL: https://issues.apache.org/jira/browse/HBASE-8640
Project: HBase
Issue Type: Bug
Components: master
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.95.2, 0.94.9
Attachments: HBASE-8640_94.patch, HBASE-8640.patch
We are starting rpc server with default interface hostname or configured ipc
address
{code}
this.rpcServer = HBaseRPC.getServer(this,
new Class?[]{HMasterInterface.class, HMasterRegionInterface.class},
initialIsa.getHostName(), // This is bindAddress if set else it's
hostname
initialIsa.getPort(),
numHandlers,
0, // we dont use high priority handlers in master
conf.getBoolean(hbase.rpc.verbose, false), conf,
0); // this is a DNC w/o high priority handlers
{code}
But we are initialzing servername with default hostname always master znode
also have this hostname.
{code}
String hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
conf.get(hbase.master.dns.interface, default),
conf.get(hbase.master.dns.nameserver, default)));
...
this.serverName = new ServerName(hostname,
this.isa.getPort(), System.currentTimeMillis());
{code}
If both default interface hostname and configured ipc address are not same
clients will get MasterNotRunningException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8675) Two active Hmasters for AUTH_FAILED in secure hbase cluster
[ https://issues.apache.org/jira/browse/HBASE-8675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Shaohui updated HBASE-8675: --- Summary: Two active Hmasters for AUTH_FAILED in secure hbase cluster (was: Two active Hmaster for AUTH_FAILED in secure hbase cluster) Two active Hmasters for AUTH_FAILED in secure hbase cluster --- Key: HBASE-8675 URL: https://issues.apache.org/jira/browse/HBASE-8675 Project: HBase Issue Type: Bug Components: master Reporter: Liu Shaohui Priority: Critical In our product cluster, because of the net problem to kerberos server, the ZooKeeperWatcher in active hmaster fails to Auth , gets a connection Event of AUTH_FAILED and loose the master lock. But the zookeeper watcher ignores the event, so the old active hmaster keeps to be active. After the net problem is fixed, the backup hmaster gets the master lock and becomes active. There are two two active hmasters in the cluster. 2013-05-30 09:44:21,004 ERROR org.apache.zookeeper.client.ZooKeeperSaslClient: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: krb1.xiaomi.net)]) occurred when evaluating Zookeeper Quorum Member's received SASL token. Zookeeper Client will go to AUTH_FAILED state. 2013-05-30 09:54:07,755 WARN org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection-0x3e10d98be405bc Unable to set watcher on znode /hbase/master org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = AuthFailed for /hbase/master at org.apache.zookeeper.KeeperException.create(KeeperException.java:123) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1036) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:166) at org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:231) at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:76) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:595) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:850) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:825) at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:286) at org.apache.hadoop.hbase.client.HTable.init(HTable.java:201) at org.apache.hadoop.hbase.catalog.MetaReader.getHTable(MetaReader.java:200) at org.apache.hadoop.hbase.catalog.MetaReader.getMetaHTable(MetaReader.java:226) at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:705) at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:183) at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:168) at org.apache.hadoop.hbase.master.CatalogJanitor.getSplitParents(CatalogJanitor.java:123) at org.apache.hadoop.hbase.master.CatalogJanitor.scan(CatalogJanitor.java:134) at org.apache.hadoop.hbase.master.CatalogJanitor.chore(CatalogJanitor.java:92) at org.apache.hadoop.hbase.Chore.run(Chore.java:67) at java.lang.Thread.run(Thread.java:662) I want to just abort the hmaster server if AuthFailed or SaslAuthenticated. Any better idea about this issue? For ZookeeperWatcher is used in many classes, will the aborting will bring more problems? Any more problems we need consider? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8640) ServerName in master may not initialize with the configured ipc address of hbase.master.ipc.address
[
https://issues.apache.org/jira/browse/HBASE-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672842#comment-13672842
]
Hudson commented on HBASE-8640:
---
Integrated in HBase-TRUNK #4155 (See
[https://builds.apache.org/job/HBase-TRUNK/4155/])
HBASE-8640 ServerName in master may not initialize with the configured ipc
address of hbase.master.ipc.address (Revision 1488834)
Result = SUCCESS
stack :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
ServerName in master may not initialize with the configured ipc address of
hbase.master.ipc.address
---
Key: HBASE-8640
URL: https://issues.apache.org/jira/browse/HBASE-8640
Project: HBase
Issue Type: Bug
Components: master
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.95.2, 0.94.9
Attachments: HBASE-8640_94.patch, HBASE-8640.patch
We are starting rpc server with default interface hostname or configured ipc
address
{code}
this.rpcServer = HBaseRPC.getServer(this,
new Class?[]{HMasterInterface.class, HMasterRegionInterface.class},
initialIsa.getHostName(), // This is bindAddress if set else it's
hostname
initialIsa.getPort(),
numHandlers,
0, // we dont use high priority handlers in master
conf.getBoolean(hbase.rpc.verbose, false), conf,
0); // this is a DNC w/o high priority handlers
{code}
But we are initialzing servername with default hostname always master znode
also have this hostname.
{code}
String hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
conf.get(hbase.master.dns.interface, default),
conf.get(hbase.master.dns.nameserver, default)));
...
this.serverName = new ServerName(hostname,
this.isa.getPort(), System.currentTimeMillis());
{code}
If both default interface hostname and configured ipc address are not same
clients will get MasterNotRunningException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8640) ServerName in master may not initialize with the configured ipc address of hbase.master.ipc.address
[
https://issues.apache.org/jira/browse/HBASE-8640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13672841#comment-13672841
]
Hudson commented on HBASE-8640:
---
Integrated in hbase-0.95 #224 (See
[https://builds.apache.org/job/hbase-0.95/224/])
HBASE-8640 ServerName in master may not initialize with the configured ipc
address of hbase.master.ipc.address (Revision 1488836)
Result = SUCCESS
stack :
Files :
*
/hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
ServerName in master may not initialize with the configured ipc address of
hbase.master.ipc.address
---
Key: HBASE-8640
URL: https://issues.apache.org/jira/browse/HBASE-8640
Project: HBase
Issue Type: Bug
Components: master
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.95.2, 0.94.9
Attachments: HBASE-8640_94.patch, HBASE-8640.patch
We are starting rpc server with default interface hostname or configured ipc
address
{code}
this.rpcServer = HBaseRPC.getServer(this,
new Class?[]{HMasterInterface.class, HMasterRegionInterface.class},
initialIsa.getHostName(), // This is bindAddress if set else it's
hostname
initialIsa.getPort(),
numHandlers,
0, // we dont use high priority handlers in master
conf.getBoolean(hbase.rpc.verbose, false), conf,
0); // this is a DNC w/o high priority handlers
{code}
But we are initialzing servername with default hostname always master znode
also have this hostname.
{code}
String hostname = Strings.domainNamePointerToHostName(DNS.getDefaultHost(
conf.get(hbase.master.dns.interface, default),
conf.get(hbase.master.dns.nameserver, default)));
...
this.serverName = new ServerName(hostname,
this.isa.getPort(), System.currentTimeMillis());
{code}
If both default interface hostname and configured ipc address are not same
clients will get MasterNotRunningException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
